Machine Learning Engineer 5, Senior Manager - Intelligent Foundations and Experiences (IFX)

Information Technology Senior Management Forum is seeking a Senior Manager, Machine Learning Engineer 5 to help build and scale AI and ML capabilities, including production model training and serving within a multi-tenant ML platform. The role focuses on applying strong ML engineering fundamentals while partnering with Product and Data Science teams to deliver state-of-the-art big data and ML applications.

Based in McLean, VA onsite, this position will support end-to-end delivery, from data pipeline construction and CI/CD practices through ongoing model retraining, monitoring, and governance for Responsible and Explainable AI.

What you’ll do

• Design, build, and deliver ML models and components that address real business problems in collaboration with Product and Data Science teams
• Build and scale large multi-tenant platforms for high-scale ML model training and/or serving
• Use ML modeling expertise to guide infrastructure decisions, including model choice, data and feature selection, training approach, hyperparameter tuning, dimensionality, bias/variance considerations, and validation
• Solve complex engineering problems by writing and testing application code, developing and validating ML models, and automating tests and deployment
• Partner within a cross-functional Agile team to create and enhance software supporting big data and ML applications
• Retrain, maintain, and monitor models in production
• Leverage or build cloud-based architectures, technologies, and platforms to deliver optimized ML models at scale
• Construct optimized data pipelines to feed ML models
• Apply continuous integration and continuous deployment best practices, including test automation and monitoring, to support successful releases of ML models and application code
• Manage code to reduce vulnerabilities, ensure risk-aware model governance, and apply best practices for Responsible and Explainable AI
• Use programming languages such as Python, Scala, or Java

Required qualifications

• Bachelor’s degree or higher in Computer Science, Machine Learning, or a related quantitative field (Statistics, Economics, Operations Research, Analytics, Mathematics, Engineering)
• At least 6 years of experience programming with Python, Java, Golang, or C++
• At least 6 years of Machine Learning experience using industry frameworks PyTorch or Tensorflow and libraries such as Pandas, NumPy, Scikit-learn
• At least 6 years using and operating large-scale distributed systems such as Spark and Ray to prepare AI/ML data
• At least 4 years deploying and operating ML solutions in production and operating production services in the cloud (AWS, GCP, Azure), including Kubernetes for large-scale containerized ML systems

Technologies

• Python, Scala, Java, Golang, C++
• PyTorch, Tensorflow, Pandas, NumPy, Scikit-learn
• Spark, Ray, AWS, GCP, Azure, Kubernetes
• CI/CD

Preferred qualifications

• Master’s or Doctoral Degree in Computer Science, Electrical Engineering, Mathematics, or a related field
• +2 years building production RAG pipelines (chunking, embeddings, vector indexing, graph RAG, or fine-tuning)
• +2 years leveraging agentic patterns (Tool Calling, ReAct loops, Plan-and-Execute, Multi-Agent Collaboration)
• +2 years with LLM orchestration frameworks (LangChain, LangGraph, LlamaIndex, Semantic Kernel) and model APIs (OpenAI, Anthropic, Bedrock, Ollama, Hugging Face)
• +2 years building agentic AI systems and agentic workflows
• +2 years developing AI/ML algorithms or technologies (for example: LLM Inference, Similarity Search and VectorDBs, Guardrails, Memory) using Python, C++, C#, Java, CUDA, or Golang
• +2 years right-sizing models, instance counts, and hardware types based on requirements (for example: context length, token inputs, token outputs)
• 5+ years optimizing ML algorithms, configurations, and infrastructure
• 5+ years following software development best practices including source control, testing, code reviews, CI/CD, and related practices
• 5+ years building resilient software solutions including pre-production testing, advanced deployment techniques (one-box, blue/green, gradual dial-up), monitoring, alarms, and incident response plan preparation
• 5+ years working with Machine Learning techniques and model types (including supervised, semi-supervised, unsupervised, and reinforcement learning; and regression, classification, clustering)
• 5+ years working with model architectures (RNNs, CNNs, LSTMs, Transformers), training concepts (loss function, hyperparameters, regularization), and evaluating model accuracy to diagnose and address underfitting and overfitting
• 5+ years designing, implementing, and scaling production-ready data pipelines for training and evaluating ML models
• ML industry impact through conference presentations, papers, blog posts, open source contributions, or patents
• Ability to communicate complex technical and machine learning concepts to a variety of audiences

Compensation and benefits

Salary: USD 229,900 - 262,400 per year for McLean, VA (Machine Learning Engineer 5). Additional listed location: Richmond, VA: USD 209,000 - 238,500.

• This role is eligible for performance-based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI)
• Capital One offers comprehensive, competitive, and inclusive health, financial, and other benefits to support total well-being