Databricks Data Engineer
Azure Data Engineer
Azure Databricks
Big Data
Bigdata
Bronze Silver Gold
Cloud Data Engineering
Cloud Data Platform
Cloud Platform
Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Integration
Data Integration Tools
Data Lake
Data Lakehouse
Data Pipeline
Data Pipelines
Data Platform
Data Processing
Data Warehouse
Database
Databases
Databricks
Databricks Dlt
Databricks Lakehouse
Databricks Lakehouse Platform
Databricks Pyspark
Databricks Spark
Databricks Workflows
Delta Lake
ETL
Informatica
Spark
SQL
Job Description
Design and deliver scalable Azure Lakehouse solutions as an Azure Databricks Data Engineer.
Responsibilities
- Develop and review module-level code, including creation of replication patterns and root-cause analysis for recurring issues
- Collaborate with internal and external stakeholders to ensure stable, well-documented deliverables aligned to business objectives
- Gather requirements for specific modules and maintain clear, structured documentation
- Work across multiple business processes to capture linkages, dependencies, and required changes
- Create application solution designs for assigned module(s) under guidance of the Project Manager, aligned to business requirements and technical standards
- Review developed code, resolve technical questions for the assigned module, and ensure replication patterns match project standards
- Collaborate with project managers across all phases of the software development life cycle
- Troubleshoot and perform root-cause analysis for recurring technical issues
- Modify software codes to resolve errors, adapt to new hardware and software, enhance performance, and upgrade interfaces
- Validate configurations with technical teams, facilitate integration of new applications, and address deployment dependencies
- Prepare user training documents and related frameworks to support adoption and reduce transition risk
- Design Databricks Lakehouse architecture using Delta Lake and Bronze/Silver/Gold Medallion layers
- Build data pipelines using Databricks Workflows, Delta Live Tables, Auto Loader, Azure Data Factory, and Databricks SQL
- Implement enterprise data solutions with Azure Data Lake Storage Gen2, Synapse, PySpark, Spark SQL, and SQL
- Optimize Spark workloads via partitioning, cluster sizing, data skew handling, broadcast joins, and Photon acceleration
- Lead migration from SAS, Hadoop, Teradata, Oracle, SQL Server, Synapse, and legacy ETL platforms
- Define data modeling and domain architecture for batch/streaming, event-driven processing, and modern governance/security
- Establish governance controls including lineage and access controls
Requirements
- Hands-on experience in cloud data architecture, Databricks platform design, performance optimization, migration, data modeling, streaming, and security
- Mandatory Skills: DataBricks - Data Engineering
- Experience: 5-8 years
Technologies
- Azure
- Databricks
- Delta Lake
- PySpark
- Spark SQL
- ADLS Gen2
- Azure Data Factory
- Synapse
- Databricks Workflows
- Delta Live Tables
- Auto Loader
- Databricks SQL
- Bronze/Silver/Gold Medallion
- Photon acceleration
- SAS
- Hadoop
- Teradata
- Oracle
- SQL Server
- SQL
- Legacy ETL platforms
Benefits
- Full range of medical and dental benefits options
- Disability insurance
- Paid time off (inclusive of sick leave)
- Other paid and unpaid leave options
Location: Minneapolis, Minnesota (onsite)
Compensation: USD 60,000 - 135,000 per yearly