Senior AI Software Engineer

City National Bank of Florida is seeking a hands-on Senior AI Software Engineer to architect, develop, and deploy secure, scalable, production-grade AI applications and agentic workflows. The role is onsite in Coral Gables, FL 33134, combining Python backend engineering with enterprise RAG on Azure, production operations, and governance-focused AI delivery.

What you’ll build

• Production-grade AI applications and backend services in Python using FastAPI and modern engineering practices.
• Stateful, multi-step agentic workflows using LangGraph and LangChain v1.
• Enterprise RAG systems using Azure AI Search, hybrid retrieval, reranking, metadata filtering, and source-grounded generation.
• Secure integrations across Azure OpenAI, enterprise APIs, document repositories, databases, and internal systems via strong, secure API contracts.
• MCP servers and tools with authentication, authorization, input validation, and access controls.

Key responsibilities

• Design and implement graph state management including checkpointing, conditional routing, retries, recovery, idempotency, and controlled execution.
• Build enterprise RAG pipelines: ingestion, chunking, embeddings, Azure AI Search, hybrid retrieval, reranking, metadata filtering, and grounded generation.
• Create reusable components such as application components, API interfaces, shared libraries, and common AI engineering patterns.
• Implement human-in-the-loop approval gates, workflow state persistence, audit trails, and prevention of unauthorized downstream actions.
• Establish automated quality gates covering unit testing, integration testing, retrieval evaluation, agent behavior, security, and regression testing.
• Engineer for reliability, scalability, latency, cost, observability, and maintainability.
• Apply secure cloud patterns with Azure identity, managed identities, Key Vault, network controls, and secrets management.
• Containerize and deploy services through CI/CD to Azure Container Apps (ACA), using Azure Kubernetes Service (AKS) when workload requirements justify it.
• Use Azure Monitor and telemetry for logs, metrics, tracing, incident investigation, and operational support.
• Review code, guide implementation decisions, troubleshoot complex production defects, and mentor other engineers while staying an active developer.
• Collaborate with AI leadership and stakeholders across architecture, cybersecurity, model risk, legal, and business teams to deliver governed banking AI solutions.

What you bring

• Approximately 8 years of professional software engineering experience, including substantial hands-on Python development.
• Strong understanding of software architecture, design patterns, SOLID principles, concurrency, asynchronous programming, API design, and distributed systems.
• Experience with FastAPI (or equivalent), Pydantic, SQL, Git, pytest, and automated code-quality practices.
• Demonstrated ability to design, implement, test, deploy, and troubleshoot production services.
• Hands-on experience delivering LLM-powered applications beyond proof-of-concept.
• Practical knowledge of LangGraph and LangChain v1 with direct LangGraph experience strongly preferred.
• Experience with stateful workflows including tool calling, conditional execution, structured outputs, retries, failure recovery, and human approval patterns.
• Strong end-to-end RAG knowledge: embeddings, vector and hybrid search, reranking, grounding, and retrieval evaluation.
• Ability to design repeatable LLM evaluation and regression-testing strategies.
• Capability to diagnose hallucinations, retrieval failures, prompt-injection risks, tool misuse, and inconsistent agent behavior.
• Hands-on experience with Azure OpenAI and preferably Azure AI Search.
• Experience with Azure identity and security services including managed identities and Key Vault.
• Experience with Docker, CI/CD, cloud deployments, automated testing, monitoring, and production troubleshooting.
• Experience building secure APIs and integrating enterprise systems.
• Strong understanding of authentication, authorization, least privilege, data protection, and auditability.

Education

Bachelor’s or master’s degree in Computer Science, Software Engineering, AI, or a related field, or equivalent practical experience.

Technologies

Python, FastAPI, LangGraph, LangChain v1, Azure OpenAI, Azure AI Search, MCP, Pydantic, SQL, Git, pytest, Docker, CI/CD, Azure Container Apps (ACA), Azure Kubernetes Service (AKS), Azure Monitor, Azure identity, managed identities, Key Vault, Azure Database for PostgreSQL, PostgreSQL, OpenTelemetry

Preferred qualifications

• Production experience implementing MCP servers and secure enterprise tools.
• Experience with Azure Container Apps and Azure Kubernetes Service.
• Experience with PostgreSQL or Azure Database for PostgreSQL, persistent workflow state, and object storage.
• Experience with enterprise document processing and financial data extraction.
• Experience with OpenTelemetry, distributed tracing, AI observability, latency optimization, and token-cost management.
• Banking or other regulated-industry experience, especially credit, lending, fraud, or risk workflows.
• Experience with model governance, explainability, human oversight, and responsible AI practices.

Interview guidelines

Applicants are not permitted to use AI-enabled devices during interviews, including AI glasses or similar wearable technology that can record, transmit, or generate real-time information.

Equal opportunity

City National Bank of Florida is an Equal Opportunity Employer and does not discriminate based on race, color, religion, sex, national origin, age, disability, genetic information, protected veteran status, or any status protected by federal, state, and Florida law. The bank complies with the ADA and applicable Florida laws.

Accommodation request

Applicants who require a reasonable accommodation to participate in the interview process should contact [email protected] in advance to submit an accommodation request.