DeveloperJobs.io
← Back to all jobs

Job Description

Robotics Technologies is hiring a Data Engineer for an onsite role in Quincy, MA. This position is set for a 12-month engagement with the possibility of extension and focuses on building and optimizing Databricks-driven data pipelines.

Role Overview

In this Data Engineer role, you will deliver end-to-end data engineering work using Databricks. The work includes ETL/ELT development, SQL-based processing, and hands-on implementation using Scala and Spark. You will also support performance tuning and enhancement of data workflows to ensure reliable, efficient processing across Spark jobs and streaming workloads.

Key Responsibilities

  • Build and maintain Databricks-based data pipelines using ETL/ELT patterns.
  • Develop and optimize Spark jobs, data transformations, and data processing workflows within Databricks.
  • Implement solutions using Scala and Spark, including work with data frames and notebooks and SQL.
  • Perform performance tuning and enhancements across Scala/PySpark and SQL components.
  • Apply and support Databricks Lakehouse concepts, including delta lake storage and relevant architecture patterns.
  • Support streaming data processing using Spark structured streaming.
  • Work with Databricks orchestration to manage and run data processing jobs.

Required Experience

  • Minimum 8+ years of extensive data engineering experience in Databricks, including ETL/ELT using data pipelines.
  • Strong SQL and processor-related experience.
  • 8+ years hands-on development experience implementing Databricks solutions using Scala and Spark, including data frames and notebooks/SQL.
  • 8+ years hands-on development and performance tuning/enhancement experience using Scala/PySpark and SQL.
  • Strong Databricks experience developing and optimizing spark jobs, data transformations, and data processing workflows.

Databricks and Data Platform Expertise

  • Understanding of Delta Lake storage.
  • Experience with Unity catalog.
  • Experience with Autoloader.
  • Knowledge of Lakehouse architecture and Medallion architecture.
  • Familiarity with Databricks serverless options.
  • Working knowledge of Dataset and data frame concepts.
  • Experience with Databricks clusters configuration and sizing.
  • Knowledge of Databricks job orchestration.
  • Experience with Spark structured streaming.

Location and Engagement Details

  • Location: Quincy, MA (onsite)
  • Duration: 12 Months + Extension

Compensation

  • Salary: USD 86 per hour

Similar Jobs