Senior AI Software Engineer

The Senior AI Software Engineer will design, modernize, integrate, and sustain an on-premises, AI-powered enterprise chatbot platform supporting the U.S. Department of State. The role focuses on LLM and RAG capabilities, document processing, inference, and containerized deployment in highly controlled, classified, air-gapped environments.

Location and Work Setup

• Location: Sterling, VA (Onsite)
• U.S. Citizenship Required
• Travel: Less than 10%

Compensation

• Salary Range: USD 128,039 - 173,229 per year

Role Summary

Develop Python-based software and backend services for a secure enterprise chatbot platform. Responsibilities include integrating approved LLMs using OpenAI-compatible APIs and inference frameworks such as vLLM and Ollama, implementing RAG workflows for document search and question answering, and packaging the platform for deployment in disconnected and classified environments.

Responsibilities

• Design, develop, test, and maintain Python applications, microservices, and APIs for an on-prem chatbot platform.
• Extend, configure, and integrate OpenWebUI with enterprise AI services and backend systems.
• Develop custom OpenWebUI Functions, Tools, and integrations for mission-specific use cases.
• Integrate approved LLMs using OpenAI-compatible APIs and inference frameworks including vLLM and Ollama.
• Create reusable components for AI model access, prompt management, response processing, and application workflows.
• Collaborate with AI/ML engineers to improve inference performance, response quality, and application reliability.
• Support multimodal AI use cases, including document understanding, optical character recognition, and vision-language model integration.
• Build backend capabilities that enable secure enterprise AI workflows.
• Design, implement, and optimize RAG workflows for enterprise document search and question answering.
• Integrate document processing technologies including Docling, Tesseract, and vision-language models into automated ingestion pipelines.
• Develop document chunking, metadata extraction, embedding generation, and retrieval workflows.
• Integrate embedding and reranking models to improve retrieval accuracy and relevance.
• Use PostgreSQL and pgvector for vector search, semantic retrieval, and document indexing.
• Develop APIs and background services for document ingestion and knowledge base management.
• Coordinate with data engineers to ensure accurate storage, indexing, and retrieval of enterprise information.
• Implement testing and evaluation to measure RAG accuracy, retrieval relevance, and answer quality.
• Integrate and maintain AI inference services using vLLM and Ollama and compatible model-serving frameworks.
• Support deployment and configuration of approved open-weight LLMs, embedding models, rerankers, and vision-language models.
• Develop software integrations using OpenAI-compatible REST APIs.
• Assist with inference optimization, including context length, GPU memory utilization, concurrency, and model serving performance.
• Support transitions from existing inference frameworks to improved architectures as technologies mature.
• Collaborate on troubleshooting inference errors, model loading issues, and integration failures.
• Create benchmarking and performance-testing utilities for AI workloads.
• Support migration of DS Chat components from traditional deployments into containerized services.
• Develop, configure, and maintain container images using Podman or Docker, including Podman Compose configurations for services, networking, volumes, and environment variables.
• Integrate services including OpenWebUI, vLLM, Docling, PostgreSQL, pgvector, PgBouncer, and Redis.
• Write scripts and automation for repeatable installation, deployment, upgrades, and recovery.
• Implement health checks, readiness checks, logging, and service dependency management; troubleshoot container networking, authentication, storage, GPU access, and service connectivity.
• Collaborate with infrastructure and cybersecurity teams to improve deployment consistency and security.
• Research, prototype, and develop agentic AI capabilities for future DS Chat releases, including controlled tool-calling and multi-step AI workflows.
• Develop integrations enabling approved AI models to interact with authorized enterprise APIs and services.
• Explore emerging technologies involving AI orchestration, workflow automation, and Model Context Protocol (MCP).
• Design secure execution boundaries, permissions, audit logging, and approval mechanisms for AI-driven actions, including safeguards for human oversight and prevention of unauthorized actions.
• Follow software engineering best practices such as modular architecture, code reviews, documentation, and version control.
• Develop automated unit, integration, regression, and performance tests using pytest and build integration tests validating communication among OpenWebUI, inference services, document processing, and databases.
• Support CI/CD processes appropriate for disconnected and classified environments.
• Troubleshoot production issues using application logs, metrics, traces, and system diagnostics; integrate monitoring with tools such as Splunk.
• Assist with vulnerability remediation, dependency management, software supply chain reviews, and container security assessments.
• Produce technical documentation, deployment procedures, architecture diagrams, and operational runbooks; participate in Agile activities such as sprint planning, technical design reviews, and demonstrations.
• Ensure secure on-premises, air-gapped government environment readiness, including authentication integration, role-based access controls, API authorization, and data protection measures.
• Support offline deployment using preapproved dependencies, container images, and model artifacts.
• Collaborate with cybersecurity personnel to address security findings and support Risk Management Framework (RMF) activities, including evidence preparation for TRB/CCB reviews and Authority to Operate (ATO) processes.
• Follow approved procedures for handling sensitive and classified information.

Required Qualifications

• Clearance: Minimum Secret Clearance; no sponsorship
• Education: Bachelor’s degree in Computer Science, Software Engineering, Artificial Intelligence, Computer Engineering, or a related field (or equivalent practical experience)
• Experience: 5+ years of professional software engineering experience, including backend application development
• Strong proficiency in Python and experience developing production-quality software
• Experience developing RESTful APIs and integrating distributed application services
• Hands-on experience integrating LLMs or generative AI capabilities into software applications
• Understanding of transformer-based models, inference APIs, prompt engineering, and LLM application development
• Experience with RAG concepts, including document ingestion, embeddings, vector search, and retrieval pipelines
• Experience working with PostgreSQL
• Familiarity with container technologies such as Podman or Docker
• Experience with Linux-based development and application deployment
• Proficiency with Git, automated testing, debugging, and software development best practices
• Experience troubleshooting applications across multiple interconnected services
• Strong analytical, problem-solving, communication, and technical documentation skills
• Ability to work with cross-functional teams and government stakeholders
• Work requirement: 7+ years of related experience (may vary based on technical training, certification(s), or degree)

Key Technologies

• Python, OpenWebUI
• Large Language Models (LLMs), Retrieval-Augmented Generation (RAG)
• Vector databases, OpenAI-compatible APIs, OpenAI-compatible REST APIs
• vLLM, Ollama
• RESTful APIs
• Docling, Tesseract, vision-language models
• PostgreSQL, pgvector
• Podman, Docker, Podman Compose
• Redis, PgBouncer
• pytest, Splunk
• Model Context Protocol (MCP)
• Risk Management Framework (RMF)

Requisition Information

• REQ#: RQ230390
• Requisition Type: Pipeline

Identity Verification Process

• Identity verification uses advanced biometrics and artificial intelligence
• Expected to be on camera during virtual interviews
• May take picture to verify identity and prevent fraud
• Authorize collection, processing, and use of biometric data for identity verification and security purposes