Lead Software Engineer - Data Engineer

JPMorganChase offers a benefits-forward environment onsite in Jersey City, NJ, with a salary range of USD 152,000 to 215,000 per year. The CIB Regulatory Reporting Team designs and delivers data engineering solutions using Spark-based ETL/ELT pipelines and production Python/PySpark code within an agile setting, while embracing AI-assisted development practices and ensuring security, stability, and scalability. This role supports a culture that values learning, inclusion, and collaboration and provides avenues for ongoing professional growth.

Responsibilities

• Design and implement creative data solutions with a focus on Spark-based ETL/ELT workflows and complex data problems.
• Produce secure, high-quality production code in Python and PySpark, review peers' Spark jobs and SQL logic, and troubleshoot data issues end-to-end.
• Drive adoption of enterprise AI-assisted engineering practices to improve code quality, delivery speed, and operational outcomes; establish validation standards and promote reusable patterns across the team.
• Leverage the Software Development Life Cycle toolchain, including AI-assisted development, to enhance automation and overall delivery value.
• Identify opportunities to automate recurring remediation tasks to improve pipeline reliability and lakehouse maintenance automation.
• Lead evaluation sessions with external vendors and internal teams to probe architectural designs and their fit within existing systems and information architecture.
• Lead communities of practice across Software Engineering to advance Spark performance, Iceberg best practices, and data platform operations.
• Contribute to a team culture that emphasizes opportunity, inclusion, and respect.

Requirements

• Formal training or certification in software engineering with 5+ years of applied experience.
• 5+ years building production data engineering and/or software engineering solutions, including design, development, testing, and operations.
• Hands-on experience delivering system design, application development, testing, and operational stability for large-scale data pipelines.
• Advanced proficiency in Python and PySpark, with strong capabilities in Spark SQL and SQL fundamentals.
• Proven experience leading the use of approved AI-assisted software development tools for coding, code review, test acceleration, and troubleshooting, with ability to set team expectations for AI outputs.
• Solid understanding of responsible AI use in engineering workflows, including data sensitivity, secure handling of inputs and outputs, and adherence to resiliency and security requirements; experience coaching engineers on safe adoption.
• Experience with AWS data management patterns (S3 and AWS Glue Data Catalog) or other cloud data platforms, including metadata governance and schema hygiene.
• Platform experience delivering and operating Spark workloads on EMR and/or Databricks, including tuning, troubleshooting, monitoring, and cost/performance optimization.
• Lakehouse expertise with Apache Iceberg, covering table design, partitioning strategy, schema evolution and compatibility controls, compaction, small-file mitigation, snapshot retention, metadata maintenance, safe backfills and reprocessing patterns.
• Proficiency in automation and continuous delivery for data pipelines, including CI/CD and automated testing.

Technologies

• Python
• PySpark
• Spark SQL
• Apache Spark
• Databricks
• EMR
• AWS S3
• AWS Glue Data Catalog
• Apache Iceberg
• Delta Lake
• Kafka
• Java
• GitHub Copilot
• Claude

Benefits

• Comprehensive health care coverage
• On-site health and wellness centers
• Retirement savings plan
• Backup childcare
• Tuition reimbursement
• Mental health support
• Financial coaching
• Commission-based pay
• Discretionary incentive compensation (cash and/or forfeitable equity)

About JPMorganChase

JPMorganChase is one of the oldest financial institutions, delivering innovative financial solutions to consumers, small businesses, and a global corporate, institutional, and government client base. With a history spanning more than two centuries, the company remains a leader across investment banking, consumer and small business banking, commercial banking, financial transaction processing, and asset management.

About the Team

JPMorganChase's Commercial & Investment Bank operates across banking, markets, securities services and payments, serving corporations, governments, and institutions in more than 100 countries. The team provides strategic advisory, capital formation, risk management, and liquidity solutions worldwide.