AI Developer
Backend Developer
Ai Developer
APIs
Applied Ai Engineer
Artificial Intelligence
Azure
Azure Ai
Azure Ai Services
Azure Openai
Cloud
Cloud Data Engineering
Cloud Data Platform
Cloud Engineer
Cloud Platform
Cloud Platforms
Cloud Platforms Cloud Platforms
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Integration
Data Integration Tools
Data Pipeline
Data Platform
Data Processing
Database
Databases
DevOps
Engineering
ETL
Generative AI
Informatica
Information Technology (IT)
Microsoft Azure
Microsoft Azure Ai
Microsoft Azure Ai Foundry
Openai
Programming
Programming Language
Programming Languages
Software Development
Software Engineer
Software Engineering
SQL
Job Description
Cobalt Benefits Group LLC is seeking an AI Developer to design, build, test, and deploy governed AI solutions onsite in Boston.
Responsibilities
- Design, prototype, build, test, and deploy production AI applications and agents; choose between Copilot Studio, workflow, agent, custom-service, vendor, or non-AI options using Build-vs-Buy guidance.
- Build retrieval-augmented generation (RAG) systems, including document ingestion, chunking, indexing, metadata, retrieval tuning, and grounded responses with source citations.
- Integrate with internal and third-party systems via authenticated REST and GraphQL APIs; collaborate on data pipelines for documents and structured data.
- Apply least-privilege permissions, single sign-on, secrets management, and PHI/PII protections; implement output validation plus logging and retention controls with IT Security.
- Create evaluation harnesses using golden test sets and subject-matter expert rubrics; run production sampling and measure accuracy, citation support, completeness, latency, and transaction cost (including results by plan type).
- Implement telemetry for prompts, retrieval, tool calls, and outputs; maintain runbooks, hand-off documentation, cost controls, and operational monitoring.
- Advance work through the CBG AI Solution Delivery Playbook decision gates and required artifacts (risk assessments, requirements, evaluation plans, and transition documentation).
- Collaborate with the AI Risk Manager, Compliance, Data Protection Officer, Security, and the AI Ethics Committee as appropriate; share reusable components, templates, and approved prompt patterns.
Requirements
- Bachelor’s degree in Computer Science, Information Systems, related field, or equivalent practical experience; 3+ years of professional software development experience.
- Hands-on generative AI development and evaluation experience, including RAG, context engineering, tool/function calling, and at least one production or production-pilot solution.
- Strong Python and/or C#/.NET development skills; experience building production APIs, integrating with REST, using OAuth (or similar authorization), and securely managing secrets and configuration.
- Ability to review, test, debug, and maintain AI-generated code; working knowledge of SQL, data modeling, and data pipelines.
- Experience handling HIPAA-regulated information, PHI masking, and access controls.
- Experience with AI regression testing, monitoring, model version management, release controls, cost optimization, and guarded multi-step agents.
- Ability to document technical decisions, follow governance and change control, run business requirements sessions, and recommend delaying or stopping solutions that fail evaluation thresholds.
Technologies
- Copilot Studio, Power Automate, Power Platform, Azure OpenAI, Azure AI Search, Microsoft 365 Copilot, Power Apps
- Azure AI Foundry, Azure API Management, Entra ID, Key Vault
- Azure SQL, Cosmos DB, Azure Data Lake Storage, Blob Storage, Dataverse
- GitHub, CI/CD pipelines
- REST, GraphQL, OAuth, Python, C#, .NET, SQL
Benefits
- Fantastic medical, dental, and vision insurance
- Twice annual employer HSA contributions covering 50% of the HDHP plan’s annual deductible
- Company provided Basic Life and AD&D
- Company paid Short-Term and Long-Term Disability
- Flexible Spending Accounts
- 401(k) Retirement Plan with up to a 6% employer-match (100% fully vested after 3 years)
- 10+ paid holidays; fully paid half day Summer Fridays
- Generous paid vacation and sick time
- Annual Paid Volunteer Day
- Annual Tuition Reimbursement; Annual Health and Wellness Reimbursement
- Lots of fun company events
Preferred
- Experience with Microsoft Copilot Studio, Power Automate, Power Platform, Azure OpenAI, and Azure AI Search.
- Demonstrated improvement of retrieval or extraction systems using before-and-after evaluation results.
- Experience with AI coding agents and Model Context Protocol (MCP).
- Familiarity with healthcare and employee benefits domains including claims, eligibility, plan documents, or EDI.
Personal Attributes
- Collaborative communicator who translates business processes into technical requirements and partners across business, engineering, security, and compliance.
- Analytical, detail-oriented problem solver focused on measurable quality, documentation, and continuous improvement.
- Sound judgment and accountability in privacy, fairness, human oversight, and responsible AI delivery.
- Adaptable, hands-on builder comfortable selecting between low-code and custom engineering approaches.
Measures of Success
- First 90 days: complete AI governance onboarding, ship an enhancement through Production Readiness Review, establish a golden-test-set evaluation harness.
- First 6 months: release a new approved AI solution, publish reusable components adopted by another project, enable monitoring dashboards.
- First 12 months: meet approved business KPIs, demonstrate reusable delivery patterns, and maintain current inventory, risk classification, evaluation, and monitoring records.
Compliance and Responsible AI
- Solutions must meet CBG governance and technical security requirements, with privacy, security, and human oversight proportionate to risk.
- Automated claim denials without human review and use of PHI in public generative AI models are prohibited.
- Work aligns with HIPAA, HITRUST CSF, NIST AI Risk Management Framework, and applicable OWASP guidance.
Location and Salary
- Location: Boston, MA (onsite)
- Salary: USD 120,000 - 155,000 per year
- Minimum experience: 3 years