Kv Cache Developer Jobs

1 job found

2d ago

Systems Software Engineer

Micron Richardson, TX

Richardson, TX

2d ago

Full time C/c++ Cuda Engineer

Apply Now

Job Description

Micron’s Storage and Memory Systems (SMS) group in the ITP Pathfinding organization seeks a Principal Systems Software Engineer to architect and build software across the AI and hardware interface.

Responsibilities

Design and modify AI inference software stacks to enable proof-of-concept hardware capabilities
Develop software that quantifies and demonstrates the value of proposed system features
Lead software architecture and development efforts independently and as a technical lead for small teams
Research emerging AI software topics to identify memory and storage system challenges and opportunities
Develop and modify Linux applications and libraries using C, C++, Python, Rust, CUDA, and/or SYCL for advanced heterogeneous computing systems
Develop and use emulators to forecast system-level performance and functional impacts before hardware availability
Debug system-level problems across applications, libraries, operating systems, kernels, system configuration, networks, and hardware/software interfaces
Use emerging AI tools to improve productivity
Collaborate across Micron organizations and with external partners
Mentor interns and earlier-career engineers
Present technical findings, insights, and innovations to internal or external customers

Requirements

Bachelor’s or Master’s degree with 10+ years of experience, or PhD with 7+ years of experience
Experience across end-to-end AI inference software stack architectures, with hands-on development across layers such as frameworks, inference engines, ML compilers, compiler infrastructure, hardware runtimes, or hardware abstraction layers
Experience debugging issues spanning applications, libraries, operating systems, kernels, networks, and hardware/software interfaces
Proficiency in C, C++, and Python, plus Linux-specific development experience including multithreading, IPC, ioctl, GDB, and working with /proc and /sys
Experience programming GPUs or other accelerators using CUDA, SYCL, OpenCL, or a comparable programming model
Ability to lead technical initiatives without supervisory authority
Familiarity with LLM serving internals, including KV-cache management, prefill and decode phases, KV-cache offloading, and disaggregated serving
Familiarity with inference engines and distributed serving frameworks such as SGLang, vLLM, NVIDIA Dynamo, or TensorRT-LLM
Strong understanding of heterogeneous compute architecture concepts: memory hierarchies, NUMA, caching, accelerator architectures, and interconnects including PCIe, CXL, NVLink, or UALink

Technologies

C, C++, Python, Rust
CUDA, SYCL, OpenCL
Linux, GDB, /proc, /sys
Multithreading, IPC, ioctl
KV-cache management, prefill and decode phases, KV-cache offloading, disaggregated serving
SGLang, vLLM, NVIDIA Dynamo, TensorRT-LLM
NUMA, PCIe, CXL, NVLink, UALink
Emulators, ML compilers

Preferred Qualifications

Technical leadership experience in a systems-focused pathfinding or research organization
Technical knowledge of HBM, DRAM, and emerging memory technologies
Technical knowledge of NVMe SSDs, NAND, and next-generation storage systems

Location: Richardson, TX (onsite)

Experience Level: 7+ years

View full job page →

Get Kv Cache Developer Jobs alerts

New Kv Cache Developer Jobs, daily.