Medallion Architecture Developer Jobs

1 job found

1mo ago

Databricks Developer

Coda Staffing Chicago, IL

Chicago, IL

1mo ago

$60 - $70/hr Full time Python Senior AWS

Apply Now

Job Description

At Coda Staffing, the Databricks Developer will architect and optimize data pipelines in Azure, leveraging Databricks PySpark and Python to turn healthcare data into reliable, governable insights. The role emphasizes modern data lakehouse patterns, governance, and CI/CD practices to support compliant analytics in healthcare environments.

Responsibilities

Design, implement, and maintain scalable data pipelines using Databricks PySpark and Python.
Build and optimize ETL/ELT processes within Azure cloud environments.
Implement data models following Data Lakehouse principles, including Medallion architecture.
Ensure data quality, consistency, and performance across ingestion, staging, and curated layers.
Collaborate with data architects, analysts, and business stakeholders to translate healthcare data requirements into technical solutions.
Develop reusable data transformation logic and modular processing components.
Support deployment processes aligned with CI/CD and DevOps best practices.
Monitor and optimize data workflows for performance, scalability, and reliability.
Contribute to data governance, security, and compliance practices relevant to healthcare environments.

Requirements

Current knowledge of modern data tools such as Databricks, FiveTran, Data Fabric and others; core experience with data architecture, data integrations, data warehousing, and ETL/ELT processes.
Hands-on experience developing and deploying custom wheel packages or in-session notebook scripts for parallel execution across executor and worker nodes.
Experience with SQL, stored procedures, and PySpark based on area of data platform specialization.
Strong knowledge of cloud and hybrid relational database systems, such as MS SQL Server, PostgreSQL, Oracle, Azure SQL, AWS RDS, Aurora, or a comparable engine.
Strong experience with batch and streaming data processing techniques and file compaction strategies.
Hands-on experience with Databricks in Azure environments.
Advanced proficiency in Python and PySpark for distributed data processing.
Experience building and optimizing data pipelines in Azure (Azure Data Factory, Azure SQL, Data Lake Storage, etc.).
Solid understanding of data warehousing, data lakehouse concepts, and ETL/ELT frameworks.
Experience working with relational databases such as SQL Server, PostgreSQL, Oracle, or similar.
Knowledge of batch and streaming data processing patterns.
Experience working with large, complex datasets in cloud-based distributed environments.
Strong analytical and problem-solving skills; ability to work effectively in cross-functional and distributed teams.
Clear communication skills, with the ability to explain technical concepts to non-technical stakeholders.
Proactive mindset with a strong sense of ownership and a commitment to delivering high-quality, reliable data solutions.

Technologies

Databricks
PySpark
Python
Azure
Azure Data Factory
Data Lake Storage
FiveTran
Data Fabric
Medallion architecture
SQL
Stored Procedures
MS SQL Server
PostgreSQL
Oracle
Azure SQL
AWS RDS
Aurora

Location

Chicago, IL (onsite). Location options include Chicago or Cape Girardeau.

Work location

Hybrid remote in Chicago, IL 60617.

Compensation

USD 60 - 70 per hour.

View full job page →

Get Job Alerts

New jobs delivered to your inbox.