Databricks Engineer & AI Enablement Architect - Data Engineer III
Manager
Artificial Intelligence
Big Data
Bigdata
CI/CD
Cloud Data Engineering
Cloud Data Platform
Cloud Platform
Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Governance
Data Integration
Data Integration Tools
Data Lake
Data Lakehouse
Data Lineage
Data Management
Data Pipeline
Data Platform
Data Processing
Data Science
Data Science Ops
Data Security
Data Warehouse
Database
Databases
Databricks
Databricks Lakehouse
Delta Lake
DevOps
Devops Tools
ETL
Informatica
Information Technology (IT)
Integration
Service Principals
Software Development
Software Engineering
Spark
SQL
Job Description
Design and run secure, scalable Databricks data and AI foundations for federal healthcare data with Deloitte’s AI & Engineering team.
Responsibilities
- Own governance, architecture, reliability, and day-to-day operations for secure, scalable Databricks environments supporting the Data Aggregation Contract (CDAC)
- Set up workspace governance including Unity Catalog structures, access controls, service principals, and policies to protect PHI and other sensitive healthcare data
- Define medallion architecture standards and build reusable ingestion and transformation frameworks for CMS data domains (claims, clinical, provider, quality, oncology)
- Increase data quality and production reliability using validation, reconciliation, lineage, observability, SLA monitoring, dashboards, and proactive alerting
- Establish and enforce CI/CD standards, including review of code, data models, and pipeline designs
- Evaluate GenAI and LLM use cases for production readiness
Requirements
- Bachelor’s degree
- Ability to obtain and maintain the required clearance for the role
- 4+ years of Databricks engineering experience
- 4+ years with Python, PySpark, and SQL, plus Delta Lake and Medallion Architecture
- 2+ years of experience in databricks architecture
- Ability to travel 10–15% on average, based on work and client/industry needs
- Legally authorized to work in the United States without employer sponsorship, now or in the future
Technologies
- Databricks, Unity Catalog, Python, PySpark, SQL, Delta Lake
- Medallion Architecture, CI/CD, GenAI, LLM
- Service principals
Preferred
- Experience supporting federal healthcare clients and/or working with CMS claims (Medicaid/Medicare), clinical/provider data, quality measures, or longitudinal patient data
- Databricks certifications: Certified Data Engineer Professional, Data Analyst Associate, Machine Learning Professional, or Generative AI Engineer Associate
- Experience administering Databricks platforms, including Unity Catalog, workspace administration, cluster policies, service principals, secrets management, and role-based access controls
- Experience with Databricks Asset Bundles and Git-based CI/CD across development, test, and production
Team
- Deloitte’s Government & Public Services (GPS) practice supports federal, state, and local government clients and public higher education institutions
- Deloitte’s AI & Data offering covers designing, developing, and operating Data and AI platforms, products, insights, and services
Location and Compensation
- Location: Washington, DC (onsite)
- Salary range: USD 113,000 to 188,400 per year
- Incentive program: Possible discretionary annual incentive subject to program rules; eligibility depends on individual and organizational performance