DeveloperJobs.io
← Back to all jobs

Job Description

Robotics Technologies is hiring a Databricks Data Engineer for an onsite role in Atlanta, GA with a 24-month assignment and potential extension. The position focuses on building and operating enterprise-scale data processing solutions using Databricks and Apache Spark, with lakehouse patterns and strong attention to governance and pipeline reliability.

Key Responsibilities

  • Design and develop scalable data engineering solutions using Databricks.
  • Build batch and streaming data pipelines using Apache Spark.
  • Develop data transformations with PySpark and Spark SQL.
  • Implement lakehouse architecture using Delta Lake.
  • Manage data access, governance, and permissions through Unity Catalog.
  • Create and maintain automated workflows using Databricks Workflows.
  • Implement incremental data processing and CDC pipelines.
  • Optimize Spark jobs to improve processing performance.
  • Integrate Databricks with cloud storage and enterprise databases.
  • Develop data validation and quality monitoring processes.
  • Collaborate with data scientists and analytics teams.
  • Troubleshoot production issues and ensure pipeline reliability.

Required Qualifications

  • Strong hands-on experience with Databricks.
  • Advanced programming skills in Python and PySpark.
  • Strong knowledge of Apache Spark and Spark SQL.
  • Experience with Delta Lake and lakehouse architecture.
  • Knowledge of Unity Catalog and data governance.
  • Experience with Delta Live Tables and Lakeflow.
  • Experience building ETL and ELT pipelines.
  • Knowledge of AWS, Azure, or GCP cloud platforms.
  • Experience with SQL, data modeling, and data warehousing.
  • Familiarity with Git, CI/CD, and workflow orchestration.

Preferred Qualifications

  • Experience with Databricks Lakeflow and Lakeflow Connect.
  • Knowledge of Apache Kafka and real-time data processing.
  • Experience with Terraform and cloud infrastructure automation.
  • Knowledge of MLflow and machine learning workflows.
  • Experience with enterprise data migration projects.

Technologies

  • Databricks
  • Apache Spark
  • PySpark
  • Delta Lake
  • Delta Live Tables
  • Lakeflow
  • Unity Catalog
  • Spark SQL
  • Databricks Workflows
  • AWS
  • Azure
  • GCP
  • Git
  • CI/CD

Location and Compensation

Location: Atlanta, GA (onsite)

Compensation: USD 76 per hour

Assignment: 24 Months + Extension

Similar Jobs