Senior AI Software Engineer

The Health Data Analytics Institute is expanding its Product Engineering team with a Senior AI Software Engineer focused on production AI and LLM backend systems. This role owns end-to-end execution from model artifacts and inference at scale to Amazon Bedrock pipelines that support AI summaries, PDD, and conversational experiences. The work connects Data Science and production software to deliver reliable, validated outputs for clinical use.

What you’ll own

• Own the production scoring system that evaluates patients against hundreds of predictive models, including 580+ models, and drive architectural improvements to address memory pressure and compute efficiency as the system scales.
• Partner with Data Science to take model artifacts such as coefficient files and R scoring logic, ensuring correct execution inside the production Python inference pipeline. Validate data handling and parity between the R development environment and production output, and build tooling to automate validation and reduce manual reimplementation.
• Build and maintain production LLM integrations on Amazon Bedrock (Claude) for AI summaries, PDD, and conversational interfaces.
• Manage the prompt lifecycle including design, versioning, testing, and evaluation, while handling token management, output validation, evidence citation, structured output parsing, and graceful degradation.
• Build and maintain production Python services on AWS using Lambda, DynamoDB, SQS, and S3, working across the existing stack including FastAPI and legacy services during migration to serverless.
• Create testing and evaluation frameworks for both model inference and LLM outputs. Define what “correct” means for AI-generated clinical content with Clinical and Data Science teams, write tests for critical paths, and monitor production output quality to catch regressions.
• Collaborate on model release planning and validation with Data Science, develop new prompt versions, and work with Platform Engineering on deployment infrastructure. Coordinate with the broader Product Engineering team on features spanning frontend and backend.

What you’ll bring

• 7+ years of experience as a software engineer, ML engineer, or AI engineer building production systems.
• Strong Python proficiency for production service development.
• Experience building and operating services on AWS with Lambda, API Gateway, DynamoDB, SQS, S3 (or an equivalent cloud platform).
• Experience in ML engineering (deploying model artifacts from Data Science to production, including understanding model scoring, feature engineering, and validating production implementation parity).
• Experience in LLM engineering (building production LLM applications, working with model APIs such as Bedrock or OpenAI, designing prompt architectures, and handling generative AI failure modes in production).
• AI-native engineering practice, using tools such as Claude Code or Cursor as part of the daily workflow.
• Comfort with ambiguity as requirements evolve while scaling to new health system partners.
• Clear written and verbal communication.

Technologies

• Python, AWS, Lambda, API Gateway, DynamoDB, SQS, S3, FastAPI
• Amazon Bedrock, Claude
• R
• Terraform (infrastructure-as-code)
• Claude Code, Cursor, OpenAI

Location and compensation

Dedham, MA (hybrid)

USD 160,367 - 185,457 per year

Benefits

• 401(k)
• 401(k) matching
• Dental insurance
• Health insurance
• Health savings account
• Paid time off
• Retirement plan
• Tuition reimbursement
• Vision insurance

Preferred qualifications

• Experience across both ML engineering and LLM engineering.
• Experience with Amazon Bedrock or similar managed LLM services.
• Experience with R or familiarity reading R code.
• Experience with model serving at scale (hundreds of models, multi-tenant environments).
• Healthcare or regulated industry experience.
• Experience building evaluation/testing frameworks for AI system outputs.
• Experience with FastAPI, event-driven architectures (SQS, SNS), or serverless patterns.
• Familiarity with Terraform and infrastructure-as-code.