Senior Data Engineer - Databricks
Analytics
Apache Airflow
Azure Databricks
Big Data
Bigdata
Business Intelligence
CI/CD
Cloud Data Engineering
Cloud Data Platform
Cloud Platform
Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Governance
Data Integration
Data Integration Tools
Data Lake
Data Lakehouse
Data Management
Data Pipeline
Data Platform
Data Processing
Data Security
Data Visualization
Data Warehouse
Database
Databases
Databricks
Databricks Lakehouse
Databricks Lakehouse Platform
Databricks Marketplace
Databricks Pyspark
Delta Lake
Delta Sharing
DevOps
Devops Tools
Digital Marketing
ETL
Gcp Data Engineer
Informatica
Information Technology (IT)
Programming
Programming Language
Programming Languages
Reporting and Analytics
Software Development
Software Engineering
Spark
SQL
Visual Design
Job Description
Capgemini is hiring a Senior Data Engineer to design, build, and optimize scalable data solutions on the Databricks Lakehouse Platform. This onsite role in Atlanta, GA supports hands-on engineering work across the full data lifecycle, from architecture and modeling through production delivery with modern Databricks capabilities.
Compensation: USD 89,400 to 166,398 per year. Experience level: 6+ years required.
What you’ll do
- Define and deploy an end-to-end data architecture that includes Databricks medallion architecture
- Build and optimize scalable data solutions using the Databricks Lakehouse Platform, including hands-on ETL/ELT development with Apache Spark (PySpark) and Delta Lake
- Lead large teams of engineers and developers to deploy Databricks solutions
- Manage and collaborate with senior client stakeholders throughout delivery
Skills and experience needed
- 6+ years of experience in data engineering
- At least 4 years designing and delivering data engineering solutions with Databricks
- Expert hands-on development knowledge in PySpark/Python
- Expert hands-on SQL development skills
- Hands-on cloud platform experience in two of: Azure, AWS, Google Cloud
- Prior expertise deploying Delta Lake solutions into production
- Experience extracting value from large, complex datasets from multiple sources and databases
- Solid grasp of database engineering and design principles
- Familiarity with CI/CD methods (desired)
- Background working with orchestration tools, such as Airflow
- Strong knowledge of Unity Catalog
- Past or current experience with ETL tools such as Infomatica, Talend, Matillion (desired)
- Delta Sharing and Marketplace expertise
- Experience working with Databricks Apps
- Working knowledge of Databricks Genie, AI/BI, and Databricks Data Visualization capabilities
- Conceptual, logical, and physical data modeling experience (preferred)
Technologies you’ll work with
- Databricks Lakehouse Platform, Apache Spark, PySpark, Python
- Delta Lake, SQL, Unity Catalog
- Azure, AWS, GCP
- Airflow, CI/CD
- Infomatica, Talend, Matillion
- Delta Sharing, Databricks Marketplace, Databricks Apps
- Databricks Genie, AI/BI, Databricks Data Visualization
Benefits
- Paid time off based on employee grade (A-F): Vacation 12-25 days depending on grade, plus company-paid holidays, personal days, and sick leave
- Medical, dental, and vision coverage (or provincial healthcare coordination in Canada)
- Retirement savings plans (for example, 401(k) in the U.S., RRSP in Canada)
- Life and disability insurance
- Employee assistance programs
- Other benefits as provided by local policy and eligibility