Senior Data Engineer - AI & Analytics Platform - Hybrid

Allianz Life is seeking a Senior Data Engineer to help shape the AI analytics platform in a hybrid role based in Minneapolis. You will build and maintain data pipelines on the Azure and Databricks stack, enabling scalable AI-driven insights with a focus on ingestion, data architecture, security, and performance. The position offers a salary range of USD 110,000 to 151,000 per year and involves collaboration across the internal platform team, IT, data science, and analytics engineering groups.

Responsibilities

• Develop, deploy, and maintain platform components including data connections, data pipelines, Databricks Lakeflow jobs, related GitHub Actions, and Databricks Declarative Automation Bundle (DABs) pipelines, plus monitoring frameworks.
• Create and maintain templated data engineering project repositories for the internal platform team with a custom CLI; create and manage Databricks DAB Infrastructure as Code repositories as needed.
• Ensure high availability, performance, and observability of data pipelines and data quality cost monitoring through proactive monitoring and incident response.
• Maintain team documentation for the internal platform and for external platform users (developers).
• Mentor junior data engineers, review pull requests, and uphold standardized, rigorously reviewed code quality at the platform level.
• Collaborate with internal teams, IT, data science, and analytics engineering to support end-to-end solution delivery, including building and promoting data pipelines, provisioning the related Databricks DAB infrastructure for ingestion and transformation, and supporting DevOps pipelines (GitHub Actions) and processes.
• Provide support across the software development lifecycle, including CI/CD, release management, and environment promotion.
• Offer ad hoc deployment support for Databricks Apps space and Azure App Services; Kubernetes application experience is preferred.
• Evaluate emerging Azure and Databricks services and AI/ML capabilities to drive innovation and continuous improvement.
• Contribute to data engineering platform governance, including data ethics, compliance, and security best practices.
• Mentor technical teams and promote architectural standards and reusable patterns.
• Utilize artificial intelligence tools and resources, including generative AI, to enhance platform capabilities.

Requirements

• 6+ years of experience spanning data engineering, DevOps, and application development roles.
• Experience configuring and maintaining Databricks Workspaces, ACLs, and other workspace artifacts.
• Experience with Databricks Lakehouse architecture, Spark, Unity Catalog, and Delta tables.
• Experience with DevOps and SDLC practices in a regulated enterprise environment.
• Experience creating GitHub Actions workflows for code promotion across environments.
• Experience building data ingestion and data transformation pipelines in PySpark, preferably within Databricks.
• Legal authorization to work in the United States without requiring immigration sponsorship now or in the future (includes various visa categories such as H-1B, H4, L-1, L-2, TN, OPT, CPT).

Technologies

• Azure
• Databricks
• Databricks Lakeflow
• GitHub Actions
• Databricks Declarative Automation Bundle (DABs)
• Databricks DAB Infrastructure as Code
• Unity Catalog
• Delta tables
• Spark
• PySpark
• Databricks Workspaces
• ACLs
• Azure App Services
• Kubernetes

Benefits

• Comprehensive medical, dental, and vision plan options
• Flexible spending and health savings accounts
• Tuition reimbursement
• Student loan retirement program
• Generous annual paid leave
• Outstanding 401(k) company match
• Life insurance
• Onsite health center
• Child development center
• Fitness facility
• Convenience store
• Two cafeterias
• Employee Resource Groups