Structured Streaming

Structured Streaming Developer Jobs

2 jobs found

1mo ago

Senior Apache Spark Developer

Experis Jersey City, NJ

Jersey City, NJ

1mo ago Closed

$68 - $68/hr Full time Backend Developer Apache Spark Aws Data Lake

This position is no longer accepting applications (closed Oct 23, 2026).

Job Description

Experis is seeking a Senior Apache Spark Developer for a hybrid role based in Jersey City, NJ. This position supports enterprise data engineering initiatives, with a focus on migrating and optimizing existing Spark workloads to Databricks on AWS and building data processing solutions using Kafka and Spark.

Role Overview

You will design and develop Spark-based applications for large-scale processing, improve the performance of existing ETL and data transformation pipelines, and help implement cloud-native data solutions. The work includes migration efforts from on-premises platforms to Databricks, along with building batch and streaming capabilities.

Responsibilities

Design, develop, and optimize Spark-based applications for large-scale data processing.
Build streaming and batch data solutions using Apache Kafka and Apache Spark.
Develop reusable frameworks and components to support enterprise data engineering initiatives.
Support migration activities from on-premises data platforms to Databricks and AWS.
Collaborate with architects and platform engineers to implement cloud-native data solutions.

Required Qualifications

5+ years of hands-on development experience with Apache Spark.
Strong expertise in Spark SQL, Structured Streaming, and DataFrames and DataSets, including performance tuning.
Experience integrating Spark with Apache Kafka.
Strong development skills in Java and Python.
Experience designing and supporting large-scale distributed systems.

Technology Stack

Apache Spark
Databricks
AWS
Apache Kafka
Spark SQL
Structured Streaming
DataFrames and DataSets
Java and Python
On-premises data platforms and cloud storage

Location

Jersey City, New Jersey (hybrid)

Compensation

USD 68.26/hr W2 (may be some flex). Weekly pay is provided.

Benefits

Medical and Prescription Drug Plans
Dental Plan
Vision Plan
Health Savings Account
Health Flexible Spending Account
Dependent Care Flexible Spending Account
Supplemental Life Insurance
Short Term and Long Term Disability Insurance
Business Travel Insurance
401(k), Plus Match
Weekly Pay

Work Authorization / Staffing Requirement

Due to client rules, Experis cannot work with 3rd party subcontractors. All candidates must be able to work on Experis W2 directly.

View full job page →

3mo ago

Sr Databricks Data Engineer

Deloitte Pittsburgh, PA

Pittsburgh, PA

3mo ago Closed

$116k - $229k/yr Full time Apache Airflow Automation Big Data

This position is no longer accepting applications (closed Aug 19, 2026).

Job Description

Senior Databricks Data Engineer in Pittsburgh, onsite with Deloitte, offering a salary range of USD 116,200 to 229,100 per year.

Responsibilities

Champion best in class approaches for data architecture, integration, and modeling, and document them for organizational adoption.
Own the design, development, and ongoing maintenance of scalable data pipelines and architectures that support enterprise data needs.
Drive initiatives to improve data quality, enhance operational efficiency, and scale data processes.
Provide team and technology leadership by evaluating and piloting new big data and analytics tools, coaching data engineers and architects, and ensuring successful project delivery.
Design and implement governance, security, and compliance strategies tailored to modern cloud data ecosystems.
Translate technical concepts and business value for executives, business leads, and technology teams.
Oversee DevOps practices and automation, enabling CI/CD with tools such as Azure DevOps, AWS Code Pipeline, Jenkins, TFS, or PowerShell.

Requirements

Bachelor's degree in Computer Science, Engineering, or a related field
5+ years of hands-on data engineering experience with a focus on Databricks across AWS, Microsoft Azure, or Google Cloud Platform (GCP)
Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms
Experience with data warehousing, third normal form (3NF), dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark
1+ year leading complex, cross-functional data projects and technical teams, including Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, automated CI/CD pipelines, and performance optimization of data pipelines, code, and compute resources
Ability to travel 50%, on average, based on the work and client engagements
Limited immigration sponsorship may be available

Technologies

Databricks
AWS, Microsoft Azure, Google Cloud Platform (GCP)
Apache Spark
Delta Lake
Unity Catalog, Delta Live Tables, Autoloader
Structured Streaming, Databricks Workflows
Apache Airflow
CI/CD tools: Azure DevOps, AWS Code Pipeline, Jenkins, TFS, PowerShell
PySpark
Databricks Lakeflow
Experience with AI and machine learning solutions

Benefits

Discretionary annual incentive program
Benefits package aligned with Core Talent Model

Qualifications Required

Bachelor's degree in Computer Science, Engineering, or a related field
5+ years of hands-on data engineering experience with Databricks on AWS, Azure, or GCP
Experience with Lakehouse architecture, Apache Spark, Delta Lake, cloud-native databases, storage solutions, and distributed compute platforms
Experience with data warehousing, 3NF, dimensional modeling, enterprise data lakes, incremental data loads, and metadata-driven ingestion and data quality frameworks using PySpark
1+ year leading complex, cross-functional data projects and technical teams, including experience with Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, Apache Airflow, Unity Catalog, CI/CD pipelines, and performance optimization of data pipelines, code, and compute resources
Willingness to travel 50 percent on average, depending on client needs
Limited immigration sponsorship may be available

Preferred

Master's degree in Computer Science, Engineering, or a related field
Experience in one or more of AWS, Azure, and GCP cloud ecosystems and associated big data services
Experience tuning and optimizing performance in Databricks and Apache Spark environments
Experience with Databricks Lakeflow
Experience with artificial intelligence and machine learning solutions

View full job page →

Structured Streaming Developer Jobs

Senior Apache Spark Developer

Sr Databricks Data Engineer

Get Structured Streaming Developer Jobs alerts