DeveloperJobs.io
← Back to all jobs

Job Description

Lead the design, build, and scaling of the AI foundation behind intelligent customer experiences across Cisco.

Responsibilities

  • Design and build robust, multi-tenant AI services with scalable APIs, including versioning, backward compatibility, and quota management across cloud, on-premises, and air-gapped environments.
  • Select, evaluate, and integrate inference runtimes, orchestration frameworks, and serving technologies.
  • Convert emerging AI capabilities into production architectures optimized for scalability, latency, memory, and compute across GPU, CPU, and edge environments.
  • Design and implement cost optimization strategies across workloads through token efficiency improvements, model selection, inference performance, and compute utilization.
  • Define and maintain measurable engineering standards focused on cost efficiency.
  • Drive architecture and design reviews, resolve complex technical trade-offs, mentor senior engineers, and shape platform direction through hands-on technical leadership and execution.

Requirements

  • Education + experience alignment:
    • Bachelors + 14 years of related experience
    • Masters + 11 years of related experience
    • PhD + 7 years of related experience
  • Experience building production-grade, multi-tenant distributed systems in GO, Python, or similar, including high-throughput service APIs, asynchronous/event-driven architectures, and observability plus reliability engineering at scale.
  • Experience architecting and deploying production LLM and/or SLM inference runtimes with techniques such as continuous batching, KV-cache management, or speculative decoding.
  • Experience architecting and deploying production RAG systems and agentic workflows using vector databases.
  • Experience implementing automated evaluation frameworks for AI systems, including golden test datasets, LLM-as-a-judge methodologies, drift detection, and/or CI/CD regression gates.

Technologies

  • GO, Python
  • LLM, SLM
  • Vector databases
  • CI/CD
  • vLLM, SGLang, Triton
  • LangGraph

Preferred Qualifications

  • Core contributions to open-source inference, serving, or agent-orchestration projects: vLLM, SGLang, Triton, LangGraph.
  • Hands-on experience optimizing inference workloads for resource-constrained compute environments (CPU, NPU, small GPU footprints, and on-prem hardware appliances).

Benefits

  • Starting salary range posted for this position is $296,300.00 to $374,800.00 (projected salary range for new hires in U.S. and/or Canada locations; not including incentive compensation, equity, or benefits).
  • Medical, dental and vision insurance (U.S. employees, subject to plan eligibility rules).
  • 401(k) plan with a Cisco matching contribution.
  • Paid parental leave.
  • Short and long-term disability coverage.
  • Basic life insurance.
  • Grants of Cisco restricted stock units, which vest following continued employment with Cisco for defined periods of time.
  • 10 paid holidays per full calendar year.
  • 1 floating holiday for non-exempt employees.
  • 1 paid day off for employee’s birthday.
  • Paid year-end holiday shutdown.
  • 4 paid days off for personal wellness determined by Cisco.
  • Non-exempt employees receive 16 days of paid vacation time per full calendar year (accrued at 4.92 hours per pay period for full-time employees).
  • Exempt employees participate in Cisco’s flexible vacation time off program, with no defined limit (subject to availability and some business limitations).
  • 80 hours of sick time off provided on hire date and each January 1st thereafter; up to 80 hours of unused sick time carried forward.
  • Additional paid time away may be requested to deal with critical or emergency issues for family members.
  • Optional 10 paid days per full calendar year to volunteer.
  • For non-sales roles: eligible to earn annual bonuses subject to Cisco’s policies.
  • For sales plans: performance-based incentive pay on top of base salary, subject to the applicable Cisco plan.

Additional Information

  • Location: San Jose, CA (hybrid) with 3 days per week on site.
  • Base salary range listed: USD 263,700 - 431,000 per yearly.
  • Application window expected to close on: 12/17/2026.
  • Team focus: Cisco’s CX AI Incubation and Agents team owns the foundational AI platform behind intelligent customer experiences, spanning model and inference layers, evaluation systems for shipped AI behavior, agentic/orchestration infrastructure, foundational AI services and APIs, packaging and upgrade capabilities, plus delivery intelligence, network automation, and edge computing. Work spans classical ML, domain-specific language models, and agentic AI.
  • Applicable full salary ranges by location:
    • New York City Metro Area: $296,300.00 - $431,000.00
    • Non-Metro New York state & Washington state: $263,700.00 - $383,500.00
  • Quota-based sales note: ranges in this posting include base pay and sales target incentive compensation combined.

Similar Jobs