Sr Databricks Data Engineer

This onsite role, positioned as a Senior Consultant - Databricks Engineer with Deloitte, centers on designing, building, and optimizing cloud-based data engineering solutions to modernize data platforms and enable analytics and AI at enterprise scale.

Responsibilities

• Promote and formalize best practices for data architecture, integration, and modeling across the organization.
• Own the design, development, and ongoing maintenance of scalable data pipelines and architectures to support large-scale enterprise data needs.
• Drive improvements in data quality, operational efficiency, and the scalability of data processes.
• Lead and mentor teams of data engineers and architects; assess, pilot, and integrate new big data and analytics technologies to keep the organization at the cutting edge.
• Advise on data governance, security, and compliance strategies tailored to modern cloud data ecosystems.
• Translate technical concepts into business value for executives, business leads, and technology stakeholders.
• Oversee DevOps and automation practices to enable CI/CD pipelines with tools such as Azure DevOps, AWS CodePipeline, Jenkins, TFS, and PowerShell.
• Provide clear technical guidance to colleagues and project teams.

Requirements

• Ability to work independently and collaboratively as part of a team.
• Effective written and verbal communication skills.
• Meticulous attention to detail and a commitment to high-quality work product.
• Ability to build and sustain professional relationships across stakeholders.
• Experience to lead projects or workstreams with accountability for outcomes.
• Capacity to manage and prioritize multiple tasks in a fast-paced, dynamic environment.
• Strong interpersonal skills and a professional demeanor.
• Proven ability to meet deadlines.

Technologies

• Databricks, AWS, Azure, and Google Cloud Platform (GCP)
• Delta Lake, Apache Spark, PySpark
• Unity Catalog, Delta Live Tables, Autoloader
• Structured Streaming, Databricks Workflows
• Apache Airflow
• Azure DevOps, AWS CodePipeline, Jenkins, TFS, PowerShell
• Databricks Lakehouse concepts and related tooling

Education and Experience

• Qualifications Required
• Bachelor's degree in Computer Science, Engineering, or a related field
• 5+ years of hands-on data engineering experience with a focus on Databricks on AWS, Azure, or GCP
• Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms
• Experience with data warehousing, 3NF, dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark
• 1+ year leading complex, cross-functional data projects and technical teams, including Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, CI/CD pipelines, and performance optimization
• Ability to travel 50 percent on average based on client engagement
• Limited immigration sponsorship may be available

Preferred Qualifications

• Master’s degree in Computer Science, Engineering, or a related field
• Experience across one or more cloud ecosystems (AWS, Azure, GCP) and associated big data services
• Experience tuning and optimizing performance in Databricks and Apache Spark environments
• Experience with Databricks Lakeflow
• Experience with artificial intelligence and machine learning solutions

Location

Location: Sacramento, CA (onsite)

Compensation

Salary: USD 137,500 - 193,600 per year

Travel and Work Scope

Travel: Up to 50% on average, depending on client engagements and project requirements.