Autoloader Developer Jobs

1 job found

16d ago

Sr Databricks Data Engineer

Deloitte Sacramento, CA

Sacramento, CA

16d ago

$138k - $194k/yr Full time Autoloader Big Data Bigdata

Apply Now

Job Description

This onsite role, positioned as a Senior Consultant - Databricks Engineer with Deloitte, centers on designing, building, and optimizing cloud-based data engineering solutions to modernize data platforms and enable analytics and AI at enterprise scale.

Responsibilities

Promote and formalize best practices for data architecture, integration, and modeling across the organization.
Own the design, development, and ongoing maintenance of scalable data pipelines and architectures to support large-scale enterprise data needs.
Drive improvements in data quality, operational efficiency, and the scalability of data processes.
Lead and mentor teams of data engineers and architects; assess, pilot, and integrate new big data and analytics technologies to keep the organization at the cutting edge.
Advise on data governance, security, and compliance strategies tailored to modern cloud data ecosystems.
Translate technical concepts into business value for executives, business leads, and technology stakeholders.
Oversee DevOps and automation practices to enable CI/CD pipelines with tools such as Azure DevOps, AWS CodePipeline, Jenkins, TFS, and PowerShell.
Provide clear technical guidance to colleagues and project teams.

Requirements

Ability to work independently and collaboratively as part of a team.
Effective written and verbal communication skills.
Meticulous attention to detail and a commitment to high-quality work product.
Ability to build and sustain professional relationships across stakeholders.
Experience to lead projects or workstreams with accountability for outcomes.
Capacity to manage and prioritize multiple tasks in a fast-paced, dynamic environment.
Strong interpersonal skills and a professional demeanor.
Proven ability to meet deadlines.

Technologies

Databricks, AWS, Azure, and Google Cloud Platform (GCP)
Delta Lake, Apache Spark, PySpark
Unity Catalog, Delta Live Tables, Autoloader
Structured Streaming, Databricks Workflows
Apache Airflow
Azure DevOps, AWS CodePipeline, Jenkins, TFS, PowerShell
Databricks Lakehouse concepts and related tooling

Education and Experience

Qualifications Required
Bachelor's degree in Computer Science, Engineering, or a related field
5+ years of hands-on data engineering experience with a focus on Databricks on AWS, Azure, or GCP
Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms
Experience with data warehousing, 3NF, dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark
1+ year leading complex, cross-functional data projects and technical teams, including Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, CI/CD pipelines, and performance optimization
Ability to travel 50 percent on average based on client engagement
Limited immigration sponsorship may be available

Preferred Qualifications

Master’s degree in Computer Science, Engineering, or a related field
Experience across one or more cloud ecosystems (AWS, Azure, GCP) and associated big data services
Experience tuning and optimizing performance in Databricks and Apache Spark environments
Experience with Databricks Lakeflow
Experience with artificial intelligence and machine learning solutions

Location

Location: Sacramento, CA (onsite)

Compensation

Salary: USD 137,500 - 193,600 per year

Travel and Work Scope

Travel: Up to 50% on average, depending on client engagements and project requirements.

View full job page →

Get Autoloader Developer Jobs alerts

New Autoloader Developer Jobs, daily.