Senior Machine Learning Engineer

Design and productionize machine learning inference across Cloudflare’s global network, from frontier open LLMs and real-time speech models to customer-deployed inference on heterogeneous accelerators.

Responsibilities

• Develop, optimize, and productionize machine learning models for Cloudflare’s serverless inference platform, prioritizing performance, reliability, and model quality.
• Build benchmarking and evaluation frameworks to measure latency, throughput, and cost efficiency, plus model behavior across LLMs, speech, vision, and other model families.
• Improve inference performance using techniques such as quantization, batching, caching, model compilation, and accelerator-aware runtime tuning.
• Work with systems engineers to integrate models into Cloudflare’s distributed inference infrastructure across a heterogeneous fleet of GPUs and next-generation accelerators.
• Advance model deployment workflows, including validation, rollout safety, observability, regression testing, and operational readiness.
• Collaborate with product and engineering teams to translate customer requirements into scalable ML capabilities for Workers AI.
• Mentor engineers, contribute to technical direction, and raise the quality bar for production ML practices.

Requirements

• Experience building, optimizing, and operating machine learning models in production.
• Strong proficiency in Python and modern ML frameworks such as PyTorch, TensorFlow, JAX, or equivalent.
• Hands-on inference optimization for large-scale models, including quantization, batching, caching, compilation, and serving runtime tuning.
• Experience with large-scale inference serving frameworks or runtimes such as SGLang, vLLM, TensorRT-LLM, ONNX Runtime, Triton, llama.cpp, or similar.
• Familiarity with modern deep learning architectures including LLMs, speech models, vision models, embeddings, multimodal models, and retrieval-augmented generation.
• Experience optimizing models for GPUs or specialized accelerators.
• Understanding of production ML concerns: evaluation, monitoring, model regressions, rollout safety, and reliability.
• Ability to work across ML and systems boundaries, with familiarity in distributed systems, networking, or serverless platforms.
• Track record leading complex technical projects and mentoring engineers.

Technologies

• Python
• PyTorch
• TensorFlow
• JAX
• SGLang
• vLLM
• TensorRT-LLM
• ONNX Runtime
• Triton
• llama.cpp
• Workers AI

Compensation

• Compensation may be adjusted depending on work location.
• Bay Area based hires: estimated annual salary $194,000 - $266,000.
• New York City, New Jersey, Washington, Washington DC, and California (excluding Bay Area) based hires: estimated annual salary $185,000 - $254,000.

Equity

• Eligible to participate in Cloudflare’s equity plan.

Benefits

• Medical/Rx Insurance
• Dental Insurance
• Vision Insurance
• Flexible Spending Accounts
• Commuter Spending Accounts
• Fertility & Family Forming Benefits
• On-demand mental health support and Employee Assistance Program
• Global Travel Medical Insurance
• Short and Long Term Disability Insurance
• Life & Accident Insurance
• 401(k) Retirement Savings Plan
• Employee Stock Participation Plan
• Flexible paid time off covering vacation and sick leave
• Leave programs, including parental, pregnancy health, medical, and bereavement leave

Bonus Points

• Experience contributing to open source ML tooling, model serving frameworks, or inference runtimes.

Location: New York, NY (hybrid)

Salary Range Listed: USD 194,000 - 266,000 per yearly