J
Lead Data Engineer
Manager
Big Data
Bigdata
Cloud
Cloud Data Engineering
Cloud Data Platform
Cloud Data Warehouse
Cloud Data Warehouse
Cloud Platform
Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Engineering Lead
Data Integration
Data Lake
Data Lakehouse
Data Pipeline
Data Pipelines
Data Platform
Data Processing
Data Warehouse
Data Warehousing
Database
Databases
Databricks
Engineer
ETL
Flink
Google Cloud Dataflow
Informatica
Information Technology (IT)
Integration
Kafka
Lead Data Engineering
Programming Language
Programming Languages
Snowflake
Spark
SQL
Stream Processing
Streaming Data
Job Description
Lead Data Engineer role in JPMorganChase’s Corporate Technology team focused on building and operating scalable data pipelines for ingestion and processing.
Responsibilities
- Design and develop scalable, secure distributed architectures for data ingestion and processing using cloud native technologies and services
- Design, implement, and maintain data pipelines to collect, process, and store large data volumes while ensuring timeliness, quality, and completeness
- Ensure data solutions comply with data residency and privacy regulations; apply security best practices for data at rest and in transit aligned with financial regulations and firm-wide policies
- Partner with technical teams and business stakeholders to discuss and propose approaches that meet current and future needs
- Define the technical target state for the product and drive execution against the strategy
- Review and evaluate recommendations, providing feedback on new technologies
- Execute creative software design and development to deliver production data solutions
Requirements
- Comfortable with Java/Python, including sound testing and code review practices
- SQL expertise: joins, aggregations, subqueries, and window functions
- Experience designing, building, and optimizing production ETL/ELT pipelines for batch and streaming using a framework such as Spark, Flink, or Dataflow
- Hands-on with Kafka (topics, keys, partitions, consumer groups), including at-least-once semantics and schema registry fundamentals
- Experience with data modelling plus partitioning and clustering
- Hands-on with Snowflake, Databricks, or similar, and cloud storage or HDFS
- Production experience with at least one major cloud provider (GCP/AWS) using native data services
- FinOps-aware approach to cost-effective design
- Experience with data quality checks, backfills, and incorporating SLIs with observability and reporting
- Experience with lakehouse platforms and table formats such as Delta, Iceberg, Avro, and Parquet, including time-travel
Technologies
- Java, Python, SQL
- Spark, Flink, Dataflow
- Kafka, schema registry
- Snowflake, Databricks
- HDFS, GCP, AWS
- Delta, Iceberg, Avro, Parquet
Preferred Qualifications, Capabilities, and Skills
- Experience with Kafka, Flink, or other streaming technologies
- Familiarity with AI/ML technologies including LLMs, prompt engineering, vector search, and responsible AI; experience using AI-assisted software development tools such as GitHub Copilot or Claude
- Financial services industry experience and familiarity with large-scale enterprise data environments
- Experience mentoring engineers and leading technical delivery initiatives
Location: Chicago, IL (onsite)
Salary: USD 133,000 - 175,000 per year
Similar Jobs
J
J
J
J
