DeveloperJobs.io
← Back to all jobs

Job Description

Design and deliver scalable Azure Lakehouse solutions as an Azure Databricks Data Engineer.

Responsibilities

  • Develop and review module-level code, including creation of replication patterns and root-cause analysis for recurring issues
  • Collaborate with internal and external stakeholders to ensure stable, well-documented deliverables aligned to business objectives
  • Gather requirements for specific modules and maintain clear, structured documentation
  • Work across multiple business processes to capture linkages, dependencies, and required changes
  • Create application solution designs for assigned module(s) under guidance of the Project Manager, aligned to business requirements and technical standards
  • Review developed code, resolve technical questions for the assigned module, and ensure replication patterns match project standards
  • Collaborate with project managers across all phases of the software development life cycle
  • Troubleshoot and perform root-cause analysis for recurring technical issues
  • Modify software codes to resolve errors, adapt to new hardware and software, enhance performance, and upgrade interfaces
  • Validate configurations with technical teams, facilitate integration of new applications, and address deployment dependencies
  • Prepare user training documents and related frameworks to support adoption and reduce transition risk
  • Design Databricks Lakehouse architecture using Delta Lake and Bronze/Silver/Gold Medallion layers
  • Build data pipelines using Databricks Workflows, Delta Live Tables, Auto Loader, Azure Data Factory, and Databricks SQL
  • Implement enterprise data solutions with Azure Data Lake Storage Gen2, Synapse, PySpark, Spark SQL, and SQL
  • Optimize Spark workloads via partitioning, cluster sizing, data skew handling, broadcast joins, and Photon acceleration
  • Lead migration from SAS, Hadoop, Teradata, Oracle, SQL Server, Synapse, and legacy ETL platforms
  • Define data modeling and domain architecture for batch/streaming, event-driven processing, and modern governance/security
  • Establish governance controls including lineage and access controls

Requirements

  • Hands-on experience in cloud data architecture, Databricks platform design, performance optimization, migration, data modeling, streaming, and security
  • Mandatory Skills: DataBricks - Data Engineering
  • Experience: 5-8 years

Technologies

  • Azure
  • Databricks
  • Delta Lake
  • PySpark
  • Spark SQL
  • ADLS Gen2
  • Azure Data Factory
  • Synapse
  • Databricks Workflows
  • Delta Live Tables
  • Auto Loader
  • Databricks SQL
  • Bronze/Silver/Gold Medallion
  • Photon acceleration
  • SAS
  • Hadoop
  • Teradata
  • Oracle
  • SQL Server
  • SQL
  • Legacy ETL platforms

Benefits

  • Full range of medical and dental benefits options
  • Disability insurance
  • Paid time off (inclusive of sick leave)
  • Other paid and unpaid leave options

Location: Minneapolis, Minnesota (onsite)

Compensation: USD 60,000 - 135,000 per yearly

Similar Jobs