Software Engineer III - Senior Databricks/Spark/AWS Data Engineer

Build secure, scalable data engineering solutions at JPMorganChase in Columbus, OH (onsite). This Software Engineer III role supports modernization to a Databricks-on-AWS lakehouse and focuses on delivering high-quality production pipelines using Apache Spark (PySpark). You will collaborate with stakeholders, apply strong engineering practices, and help enable reliable analytics for BI partners working in Sigma, Tableau, and Alteryx.

What you’ll be doing

• Design, build, and maintain new data pipelines on Databricks using PySpark, creating secure, production-ready code and reviewing/debugging processes implemented by others.
• Optimize and tune PySpark jobs and Databricks clusters for performance, scalability, and cost efficiency, including partitioning, caching, and resource management.
• Design and implement scalable data frameworks using medallion lakehouse patterns (bronze/silver/gold) for workforce data analytics end-to-end pipelines.
• Implement data quality checks and validation processes using Delta Lake and Delta Live Tables expectations to support accuracy and reliability.
• Deliver robust monitoring and alerting for data ingestion issues by leveraging Databricks and AWS CloudWatch to improve performance and throughput.
• Identify recurring operational issues and work to eliminate or automate remediation using Databricks Workflows and AWS-native automation.
• Use AI and Agentic AI to accelerate data pipeline development and adopt AI-assisted engineering tools such as Claude and GitHub Copilot to improve developer productivity and code quality.
• Provision and deliver curated, reliable data sets to BI partners so they can support reporting and analytics use cases in Sigma, Tableau, and Alteryx.
• Partner with business stakeholders to understand requirements, design appropriate solutions, and produce architecture and design artifacts for complex applications.
• Contribute to software engineering communities of practice focused on new and emerging technologies, supporting a culture that values diversity, opportunity, inclusion, and respect.

What you’ll need

• Formal training or certification in software engineering concepts plus 3+ years of applied experience in data engineering, including design, application development, testing, and operational stability.
• Advanced, hands-on expertise in Apache Spark (PySpark) for large-scale distributed processing and strong proficiency building and operating production pipelines on Databricks (including Delta Lake and lakehouse patterns).
• Strong knowledge of the AWS data ecosystem: S3, EMR, Glue, Lambda, Athena, along with AWS storage and compute services; experience with Parquet and Iceberg data formats.
• Strong programming skills in Python for data processing and application development (Java or Scala is a plus).
• Proficiency with automation and continuous delivery, using CI/CD pipelines and tools such as Git/Bitbucket, Jenkins, or Spinnaker for automated deployment and version control.
• Hands-on experience across system design, application development, testing, and operational stability, including advanced understanding of agile methodologies, application resiliency, and security.
• In-depth knowledge of the financial services industry and their IT systems.
• Solid SQL and data modeling skills for efficient data management and retrieval (experience with relational databases such as Oracle is a plus).
• Experience with scheduling tools such as Airflow and Autosys to automate and manage job scheduling.

Benefits

• Competitive total rewards package including base salary determined based on role, experience, skill set, and location.
• In eligible roles, commission-based pay and/or discretionary incentive compensation in cash and/or forfeitable equity.
• Comprehensive health care coverage.
• On-site health and wellness centers.
• Retirement savings plan.
• Backup childcare.
• Tuition reimbursement.
• Mental health support and financial coaching.
• Additional details about total compensation and benefits provided during the hiring process.
• Equal opportunity employer with high value on diversity and inclusion.

Preferred qualifications

• Databricks certifications (for example, Databricks Certified Data Engineer Associate/Professional).
• Familiarity with Generative AI and Agentic AI frameworks, including AI coding assistants such as Claude and GitHub Copilot in an engineering workflow.
• Deeper expertise in the AWS cloud platform and broader service catalog.

About the team

JPMorganChase corporate functions span areas including finance and risk, human resources, and marketing. These corporate teams are essential to helping set businesses, clients, customers, and employees up for success.