DeveloperJobs.io
← Back to all jobs

Job Description

Senior Databricks Data Engineer in Pittsburgh, onsite with Deloitte, offering a salary range of USD 116,200 to 229,100 per year.

Responsibilities

  • Champion best in class approaches for data architecture, integration, and modeling, and document them for organizational adoption.
  • Own the design, development, and ongoing maintenance of scalable data pipelines and architectures that support enterprise data needs.
  • Drive initiatives to improve data quality, enhance operational efficiency, and scale data processes.
  • Provide team and technology leadership by evaluating and piloting new big data and analytics tools, coaching data engineers and architects, and ensuring successful project delivery.
  • Design and implement governance, security, and compliance strategies tailored to modern cloud data ecosystems.
  • Translate technical concepts and business value for executives, business leads, and technology teams.
  • Oversee DevOps practices and automation, enabling CI/CD with tools such as Azure DevOps, AWS Code Pipeline, Jenkins, TFS, or PowerShell.

Requirements

  • Bachelor's degree in Computer Science, Engineering, or a related field
  • 5+ years of hands-on data engineering experience with a focus on Databricks across AWS, Microsoft Azure, or Google Cloud Platform (GCP)
  • Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms
  • Experience with data warehousing, third normal form (3NF), dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark
  • 1+ year leading complex, cross-functional data projects and technical teams, including Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of data pipelines, code, and compute resources
  • Ability to travel 50%, on average, based on the work and client engagements
  • Limited immigration sponsorship may be available

Technologies

  • Databricks
  • AWS, Microsoft Azure, Google Cloud Platform (GCP)
  • Apache Spark
  • Delta Lake
  • Unity Catalog, Delta Live Tables, Autoloader
  • Structured Streaming, Databricks Workflows
  • Apache Airflow
  • CI/CD tools: Azure DevOps, AWS Code Pipeline, Jenkins, TFS, PowerShell
  • PySpark
  • Databricks Lakeflow
  • Experience with AI and machine learning solutions

Benefits

  • Discretionary annual incentive program
  • Benefits package aligned with Core Talent Model

Qualifications Required

  • Bachelor's degree in Computer Science, Engineering, or a related field
  • 5+ years of hands-on data engineering experience with Databricks on AWS, Azure, or GCP
  • Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms
  • Experience with data warehousing, 3NF, dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark
  • 1+ year leading complex, cross-functional data projects and technical teams, including experience with Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, CI/CD pipelines, and performance optimization of data pipelines, code, and compute resources
  • Willingness to travel 50 percent on average, depending on client needs
  • Limited immigration sponsorship may be available

Preferred

  • Master's degree in Computer Science, Engineering, or a related field
  • Experience in one or more of AWS, Azure, and GCP cloud ecosystems and associated big data services
  • Experience tuning and optimizing performance in Databricks and Apache Spark environments
  • Experience with Databricks Lakeflow
  • Experience with artificial intelligence and machine learning solutions

Similar Jobs