Software Engineer III

JPMorganChase is hiring a Software Engineer III for an onsite role in New York, NY, joining a high-visibility platform team within Enterprise Technology, AI and Machine Learning, and Data Platforms. The team focuses on building platform capabilities that help other groups develop, evaluate, and run agent-based solutions powered by LLMs, with end-to-end ownership and responsible delivery.

What you’ll do

• Deliver end-to-end platform features for agent-based artificial intelligence use cases, covering technical design through production deployment, monitoring, and iterative improvements.
• Build and maintain reusable services and components that support creating, evaluating, and operating LLM-powered agents at scale.
• Implement evaluation and testing for LLM systems, including quality measurement, regression testing, and error analysis to improve reliability over time.
• Develop observability capabilities using tracing, metrics, logs, and analytics to enable stable operations and faster troubleshooting.
• Apply security, safety, and governance-by-design practices, including guardrails, access controls, and audit-ready operational patterns aligned to security and governance requirements.
• Work with product managers and stakeholders to define success metrics, prioritize efforts, and deliver measurable business impact.
• Partner across engineering, data, security, and risk and compliance teams to support secure, stable, and scalable solutions.
• Create clear documentation and reference implementations to accelerate adoption and promote responsible, consistent engineering practices.

What you’ll bring

• 3+ years applied experience plus formal training or certification in software engineering concepts.
• Proficiency in Python with strong software engineering fundamentals, including testing practices, version control, and code review.
• Experience building and operating production services, including incident readiness, performance tuning, and operational stability for data-intensive systems.
• Hands-on experience taking machine learning and/or LLM-enabled features into production, including monitoring and post-deployment iteration.
• Practical experience with prompt engineering and retrieval-augmented generation (RAG), including evaluation methods and quality measurement.
• Ability to design maintainable systems and make sound technical decisions under ambiguity, balancing delivery speed, risk, and long-term sustainability.
• Strong communication skills, including explaining technical trade-offs to both technical and non-technical stakeholders.
• Strong collaboration skills and a demonstrated ability to partner across teams to deliver outcomes.

Technologies you may work with

• Python
• Retrieval-augmented generation (RAG)
• Prompt engineering
• LangGraph
• LlamaIndex
• CI/CD
• Docker
• Kubernetes
• Vector databases
• Embedding pipelines
• Amazon Web Services
• Databricks

Benefits

• Comprehensive health care coverage
• On-site health and wellness centers
• Retirement savings plan
• Backup childcare
• Tuition reimbursement
• Mental health support
• Financial coaching

Preferred qualifications

• Experience with agent orchestration frameworks such as LangGraph, LlamaIndex, or comparable approaches, plus LLM evaluation tooling.
• Experience with CI/CD practices and production deployments using Docker and Kubernetes.
• Familiarity with vector databases, embedding pipelines, and related components used in RAG solutions.
• Experience with cloud and machine learning platforms such as Amazon Web Services or Databricks (or comparable platforms).
• Experience contributing to governance, validation approaches, or guardrail frameworks for enterprise-scale AI solutions.

Location: New York, NY (onsite). Salary: USD 137,750 - 185,000 per year. Experience: minimum 3 years.