Lead hands-on Databricks data engineering work across ETL, data models, security/compliance, and scalable pipeline performance.

Design, build, and maintain robust data pipelines within Databricks, contributing independently and providing technical leadership and mentorship to the team
Create and optimize end-to-end data flows
Own extract, transform, load (ETL) processes
Design data models to support reliable analytics and query performance
Ensure data security and compliance
Collaborate with cross-functional teams
Continuously improve performance and scalability of Databricks and AWS data workloads
Mentor a multi-disciplinary team of analysts, data engineers, developers, and data consumers in a fast-paced, agile environment
Oversee assessment, design, building, and maintenance of scalable platforms for clients

8+ years of experience designing, developing, deploying, and managing ETL processes and data pipelines, including comprehensive logging and robust error handling for troubleshooting
3+ years building, debugging, and optimizing data pipelines using Databricks notebooks, clusters, and job orchestration tools
Hands-on Databricks experience including Delta Lake, Databricks Delta, and Unity Catalog
3+ years writing Python and PySpark for ETL scripts and custom transformations
Experience with data modeling, including when to use dimensional vs relation approaches within a medallion architecture (Bronze, Silver, Gold)
Experience with AWS services, including AWS Glue, Amazon S3, and Amazon RDS
Knowledge of ETL performance optimization techniques such as partitioning, bucketing, caching, and query-performance troubleshooting
Ability to design and implement data models from defining schemas to optimizing data structures for efficient queries and analytics
Public Trust
Bachelor’s degree in Computer Science, Computer Engineering, Engineering, Mathematics, Physics, Information Systems, or Information Technology

Experience with AWS cloud-native data services such as AWS Lambda or CloudWatch for serverless data processing architectures
Experience with tools such as AWS Glue Data Catalog or Collibra for metadata management, data lineage, and cataloging data assets
Knowledge of data governance practices, data lineage, and compliance requirements (including GDPR or CCPA) supporting data quality, privacy, and security
Ability to explain complex technical concepts to a non-technical audience, including architectural design tradeoffs

Health, life, disability, financial, and retirement benefits
Paid leave
Professional development
Tuition assistance
Work-life programs
Dependent care
Recognition awards program acknowledging employees for exceptional performance and demonstration of company values
Full-time and part-time employees working at least 20 hours a week on a regular basis are eligible to participate in Booz Allen’s benefit programs

Applicants selected will be subject to a government investigation and may need to meet eligibility requirements of the U.S. government client
Public Trust determination is required

Projected compensation range: USD $77,600 to $176,000 per year (annualized).

Complete an identity verification process that leverages advanced biometrics and artificial intelligence to ensure authenticity and protect against identity fraud
Be expected to be on camera during interviews and assessments
We reserve the right to take your picture to verify your identity and prevent fraud

Use of artificial intelligence (AI) or other tools to assist with responses during interviews (in-person or virtual) is prohibited unless permission is explicitly provided

Onsite: Work will primarily be performed full-time at a customer facility, with collaboration required with colleagues and customers as needed

Location: Bethesda, MD (onsite)

Databricks Data Engineer, Lead

Job Description