DeveloperJobs.io
← Back to all jobs

Job Description

Join Deloitte’s AI & Engineering team to build and optimize Databricks Lakehouse data pipelines while supporting governance and performance.

Responsibilities

  • Develop and optimize data pipelines using Python, PySpark, and SQL on the Databricks Lakehouse platform
  • Use Medallion architecture patterns to structure, transform, and manage data across data layers
  • Perform performance tuning for Databricks workloads and SQL queries
  • Configure row-level and column-level security
  • Manage data lineage and data sharing using Unity Catalog
  • Implement CI/CD and testing processes for Databricks workflows

Requirements

  • Bachelor’s degree
  • 2+ years of Databricks engineering experience
  • 4+ years experience with Python, PySpark, and SQL
  • Ability to obtain and maintain the required clearance for the role
  • Legal authorization to work in the United States without employer sponsorship, now or at any time in the future
  • Ability to travel 10-15% on average, based on work you do and client/industry needs

Technologies

  • Python
  • PySpark
  • SQL
  • Databricks Lakehouse
  • Medallion architecture
  • Unity Catalog
  • CI/CD

What a Successful Candidate Can Demonstrate

  • Ability to work independently and collaborate within a team
  • Effective written and verbal communication skills
  • Meticulous attention to detail and quality of work product
  • Ability to build and sustain professional relationships
  • Ability to lead projects or workstreams
  • Ability to manage and prioritize multiple tasks in a fast-paced, dynamic environment
  • Strong interpersonal skills and professional demeanor
  • Ability to meet deadlines
  • Ability to provide clear guidance to others

Preferred

  • Experience using Medallion architecture in the Databricks Lakehouse platform
  • Experience configuring row-level and column-level security, data lineage, and data sharing in Unity Catalog
  • Experience implementing CI/CD and testing processes for Databricks
  • Experience with performance tuning in Databricks

Location: Washington, DC (onsite)

Compensation: USD 107,700 - 179,500 per year

Similar Jobs