Structured Streaming Developer Jobs

1 job found

1d ago

Deloitte Pittsburgh, PA

Pittsburgh, PA

1d ago

$116k - $229k/yr Full time Apache Airflow Automation Big Data

Senior Databricks Data Engineer in Pittsburgh, onsite with Deloitte, offering a salary range of USD 116,200 to 229,100 per year.

Champion best in class approaches for data architecture, integration, and modeling, and document them for organizational adoption.
Own the design, development, and ongoing maintenance of scalable data pipelines and architectures that support enterprise data needs.
Drive initiatives to improve data quality, enhance operational efficiency, and scale data processes.
Provide team and technology leadership by evaluating and piloting new big data and analytics tools, coaching data engineers and architects, and ensuring successful project delivery.
Design and implement governance, security, and compliance strategies tailored to modern cloud data ecosystems.
Translate technical concepts and business value for executives, business leads, and technology teams.
Oversee DevOps practices and automation, enabling CI/CD with tools such as Azure DevOps, AWS Code Pipeline, Jenkins, TFS, or PowerShell.

Bachelor's degree in Computer Science, Engineering, or a related field
5+ years of hands-on data engineering experience with a focus on Databricks across AWS, Microsoft Azure, or Google Cloud Platform (GCP)
Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms
Experience with data warehousing, third normal form (3NF), dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark
1+ year leading complex, cross-functional data projects and technical teams, including Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of data pipelines, code, and compute resources
Ability to travel 50%, on average, based on the work and client engagements
Limited immigration sponsorship may be available

Bachelor's degree in Computer Science, Engineering, or a related field
5+ years of hands-on data engineering experience with Databricks on AWS, Azure, or GCP
Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms
Experience with data warehousing, 3NF, dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark
1+ year leading complex, cross-functional data projects and technical teams, including experience with Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, CI/CD pipelines, and performance optimization of data pipelines, code, and compute resources
Willingness to travel 50 percent on average, depending on client needs
Limited immigration sponsorship may be available

Master's degree in Computer Science, Engineering, or a related field
Experience in one or more of AWS, Azure, and GCP cloud ecosystems and associated big data services
Experience tuning and optimizing performance in Databricks and Apache Spark environments
Experience with Databricks Lakeflow
Experience with artificial intelligence and machine learning solutions

New Structured Streaming Developer Jobs, daily.