Software Engineer, ML Serving Platform

DoorDash is hiring a Software Engineer to build a self-serve ML Serving Platform connecting request routing and online feature retrieval to model inference on CPU and GPU infrastructure (onsite).

Responsibilities

• Build next-generation ML serving infrastructure that ties together request routing, online feature retrieval, and model inference on CPU and GPU
• Address infrastructure challenges across latency, reliability, resource efficiency, and scale
• Develop self-serve tools and workflows to make serving capabilities easier for internal teams to use
• Drive defined projects end to end: technical design, implementation, testing, rollout, and production support
• Partner with internal teams to understand evolving modeling requirements, reduce adoption barriers, and deliver measurable improvements from open source advances
• Build tools, APIs, and workflows that enable teams to deploy, configure, validate, and operate models independently
• Evaluate and integrate evolving inference frameworks and enable open source models, turning promising capabilities into reliable, efficient services at scale
• Improve systems that route prediction requests, retrieve online features, and execute models to support tens of millions of predictions per second
• Create modular components so routing, feature retrieval, and model execution can evolve and scale independently as workloads and modeling requirements change
• Ensure workloads respond to changing traffic while meeting latency and availability goals and using CPU/GPU resources efficiently
• Build integrations, validation, and migration tooling to help teams adopt new serving capabilities and a unified global platform with confidence
• Use benchmarking, profiling, metrics, and tracing to find bottlenecks; participate in on-call and improve automation and runbooks

Requirements

• 2+ years of software engineering experience building and maintaining production services or infrastructure
• Strong computer science fundamentals and proficiency in a backend or systems language: Java, Kotlin, Go, C++, or Python
• Knowledge of distributed systems fundamentals: concurrency, networking, timeouts, failure handling, and performance trade-offs
• Ability to independently turn a defined problem into technical design, tested implementation, and a safe production rollout
• Experience debugging production systems and using operational data to improve reliability, performance, or cost
• Care about the engineers using your platform; collaborate effectively with internal partners to help complex infrastructure adoption
• Degree in Computer Science or related field, or equivalent practical experience

Technologies

• Java, Kotlin, Go, C++, Python
• Kubernetes
• CPU, GPU

Nice to Have

• Experience with ML inference infrastructure, online feature retrieval, or other latency-sensitive distributed services
• Experience operating containerized workloads on Kubernetes, including deployment, resource management, or autoscaling
• Experience building self-serve developer platforms, APIs, or automation that helps other engineers move faster
• Experience integrating open source infrastructure or models and validating behavior under production workloads
• Familiarity with CPU/GPU performance profiling, inference runtimes, or model deployment workflows

Location: San Francisco, CA (onsite)
Compensation: USD 130,600 - 192,000 per year
Experience: 2+ years
Education: Degree in Computer Science or related field (or equivalent practical experience)