Build enterprise agentic AI that stays reliable at Fortune 500 scale. In ServiceNow’s Agent Orchestration organization, this Staff Machine Learning Engineer role focuses on the systems that power how agents execute, observe, and improve in production. You will help shape the execution harness, runtime capabilities, and the evaluation and prompt infrastructure that keep agent behavior stable as models and configurations evolve. The team benefits from a strong engineering culture around production discipline, reliability, and measurable quality.
Location: Santa Clara, CA (onsite)
Compensation: USD 176,100 - 308,200 per yearly (base pay; plus equity when applicable, variable/incentive compensation, and benefits)
Responsibilities
- Own significant parts of the agent execution harness, the infrastructure layer enabling AI agents to reason over real enterprise data, take action across workflows, and run safely at Fortune 500 scale.
- Design and build the orchestration layer for multi-step agentic workflows, including routing inputs, managing context, invoking tools, handling retries, and surfacing execution state.
- Own runtime fault tolerance, latency, and throughput, ensuring enterprise workflows cannot fail silently or unpredictably.
- Instrument the harness with tracing, cost attribution, and latency visibility to support production debugging and proactive failure detection.
- Build prompt management systems including versioning, templating, and systematic evaluation to keep behavior stable across model and configuration changes.
- Design evaluation frameworks such as unit evals, integration evals, and production monitors to measure quality, catch regressions, and guide data-informed decisions.
- Integrate with and abstract over frontier LLMs by managing model routing, fallback strategies, cost, and latency trade-offs in production.
- Set technical standards through architecture decisions, code reviews, and coaching, especially on agentic design patterns and production AI discipline.
- Define where agent logic lives (tool call, sub-agent, hardcoded path, or human escalation) and establish design standards across the team.
Requirements
- 7+ years building production software systems with a track record in reliability, performance, and scalability.
- Hands-on experience shipping generative AI products, with ownership of AI-powered features relied upon by production users.
- Strong depth in how large language models work, including failure modes, context constraints, and prompt design impact at scale.
- Experience with Python, Kubeflow, and SageMaker, Vertex AI, or AzureML, plus SQL.
- Practical prompt engineering experience, including designing, versioning, and evaluating prompts across model updates and A/B evaluation cycles.
- A real track record in eval engineering, including building and deploying evaluation suites used to drive production AI quality decisions.
- Cost and efficiency awareness at the system level, including reasoning about model routing, inference cost, and latency trade-offs.
- Solid software engineering fundamentals, including distributed systems, API design, and testing discipline.
- Comfort operating in fast-moving, ambiguous, startup-like AI product environments.
Technologies
- Python
- Kubeflow
- SageMaker
- Vertex AI
- AzureML
- SQL
- LangChain
- LlamaIndex
Benefits
- Health plans, including flexible spending accounts
- 401(k) Plan with company match
- ESPP
- Matching donations
- Flexible time away plan
- Family leave programs
- Base pay of $176,100 - $308,200, plus equity (when applicable), variable/incentive compensation and benefits
Nice to Have
- Experience with multi-agent coordination patterns (A2A, MCP)
- Familiarity with agent frameworks (LangChain, LlamaIndex, or similar)
- Prior experience shipping AI systems in enterprise software
- Experience with AI observability tooling (tracing, cost tracking, LLM-specific monitoring)
- Familiarity with cloud-native infrastructure, service observability, logging, monitoring, reliability engineering, and production troubleshooting
Work persona: Work personas (flexible, remote, or required in office) are categories assigned based on the nature of work and assigned work location. ServiceNow may confirm the distance between primary residence and the closest ServiceNow office using a third-party service to determine eligibility for a work persona.
Equal Opportunity Employer
ServiceNow is an equal opportunity employer. All qualified applicants will receive consideration without regard to race, color, religion, sex, sexual orientation, national origin, age, disability, gender identity, veteran status, or any other category protected by law. Qualified applicants with arrest or conviction records will be considered for employment in accordance with legal requirements.
Accommodations
If you require a reasonable accommodation to complete any part of the application process, or are unable to use the online application and need an alternative method to apply, contact globaltalentss@servicenow.com for assistance.
Export Control Regulations
For positions requiring access to controlled technology subject to export control regulations, ServiceNow may be required to obtain export control approval from government authorities for certain individuals. All employment is contingent upon ServiceNow obtaining any export license or other approval that may be required by relevant export control authorities.
