Llm Performance Developer Jobs

1 job found

Today

Senior Machine Learning Engineer

Cloudflare New York, NY

New York, NY

Today

$194k - $266k/yr Full time Artificial Intelligence Engineer Generative AI

Apply Now

Job Description

Design and productionize machine learning inference across Cloudflare’s global network, from frontier open LLMs and real-time speech models to customer-deployed inference on heterogeneous accelerators.

Responsibilities

Develop, optimize, and productionize machine learning models for Cloudflare’s serverless inference platform, prioritizing performance, reliability, and model quality.
Build benchmarking and evaluation frameworks to measure latency, throughput, and cost efficiency, plus model behavior across LLMs, speech, vision, and other model families.
Improve inference performance using techniques such as quantization, batching, caching, model compilation, and accelerator-aware runtime tuning.
Work with systems engineers to integrate models into Cloudflare’s distributed inference infrastructure across a heterogeneous fleet of GPUs and next-generation accelerators.
Advance model deployment workflows, including validation, rollout safety, observability, regression testing, and operational readiness.
Collaborate with product and engineering teams to translate customer requirements into scalable ML capabilities for Workers AI.
Mentor engineers, contribute to technical direction, and raise the quality bar for production ML practices.

Requirements

Experience building, optimizing, and operating machine learning models in production.
Strong proficiency in Python and modern ML frameworks such as PyTorch, TensorFlow, JAX, or equivalent.
Hands-on inference optimization for large-scale models, including quantization, batching, caching, compilation, and serving runtime tuning.
Experience with large-scale inference serving frameworks or runtimes such as SGLang, vLLM, TensorRT-LLM, ONNX Runtime, Triton, llama.cpp, or similar.
Familiarity with modern deep learning architectures including LLMs, speech models, vision models, embeddings, multimodal models, and retrieval-augmented generation.
Experience optimizing models for GPUs or specialized accelerators.
Understanding of production ML concerns: evaluation, monitoring, model regressions, rollout safety, and reliability.
Ability to work across ML and systems boundaries, with familiarity in distributed systems, networking, or serverless platforms.
Track record leading complex technical projects and mentoring engineers.

Technologies

Python
PyTorch
TensorFlow
JAX
SGLang
vLLM
TensorRT-LLM
ONNX Runtime
Triton
llama.cpp
Workers AI

Compensation

Compensation may be adjusted depending on work location.
Bay Area based hires: estimated annual salary $194,000 - $266,000.
New York City, New Jersey, Washington, Washington DC, and California (excluding Bay Area) based hires: estimated annual salary $185,000 - $254,000.

Equity

Eligible to participate in Cloudflare’s equity plan.

Benefits

Medical/Rx Insurance
Dental Insurance
Vision Insurance
Flexible Spending Accounts
Commuter Spending Accounts
Fertility & Family Forming Benefits
On-demand mental health support and Employee Assistance Program
Global Travel Medical Insurance
Short and Long Term Disability Insurance
Life & Accident Insurance
401(k) Retirement Savings Plan
Employee Stock Participation Plan
Flexible paid time off covering vacation and sick leave
Leave programs, including parental, pregnancy health, medical, and bereavement leave

Bonus Points

Experience contributing to open source ML tooling, model serving frameworks, or inference runtimes.

Location: New York, NY (hybrid)

Salary Range Listed: USD 194,000 - 266,000 per yearly

View full job page →

Get Llm Performance Developer Jobs alerts

New Llm Performance Developer Jobs, daily.