Senior Data Engineer (Foundations)
Senior
Apache Airflow
Aws Data Engineer
Azure Data Engineer
Big Data
Bigdata
Cloud
Cloud Data Engineering
Cloud Data Platform
Cloud Infrastructure
Cloud Platform
Cloud Platforms
Data Architecture
Data Engineer
Data Engineering
Data Integration
Data Pipeline
Data Pipelines
Data Platform
Data Processing
Data Transforms
Data Warehouse
Data Warehousing
Databricks
DevOps
Infrastructure As Code
Llm Assisted
Microsoft Fabric
Snowflake
Software Development
SQL
Workflow Orchestration
Job Description
Senior Data Engineer (Foundations) will be a hands-on senior contributor embedded in the Foundations squad, building the shared ingestion platform that supports company-wide data delivery.
Responsibilities
- Design and implement reusable ingestion components with dlt and dbt-core for both structured and unstructured sources
- Handle high-volume, append-heavy ingestion and schema-drifting patterns in production pipelines
- Own the Apache Airflow platform end-to-end: extend and maintain DAGs and shared operators, manage deployments and version upgrades, and provide hands-on support to consuming teams
- Ensure every pipeline produces incremental loads, data quality checks, and lineage metadata
- Find and remove redundant ingestion patterns across consuming teams and drive standardization onto shared Platform Foundation components
- Partner with Solution Architects to evolve platform architecture as new data sources and business needs emerge
- Support data product exposure by defining and implementing governed interfaces for reliable internal access
- Contribute to Terraform-managed infrastructure and participate in multi-cloud deployment patterns across AWS and Azure
- Use and evaluate AI-assisted development tools such as GitHub Copilot and Claude Code to accelerate delivery
- Promote AI tooling adoption within the squad, including best practices and guardrails for AI-generated code review
- Explore AI-powered capabilities for internal enablement, including RAG pipelines and LLM-assisted data cataloguing for platform documentation
- Maintain and improve CI/CD pipelines for Platform Foundation components using TeamCity and GitHub Actions
- Define and enforce observability standards such as DAG/Task-level alerting and SLA tracking
- Participate in an on-call rotation for critical ingestion pipelines and drive post-incident improvements
- Create and maintain Platform Foundation documentation, runbooks, and enablement materials for consuming squads
- Convert ambiguous or changing business requirements into concrete technical designs, including challenging scope when needed
- Mentor mid-level engineers and take part in hiring and technical assessments
Requirements
- Bachelor's degree in Business, Information Systems, Data/Analytics, Computer Science, or a related field
- Minimum 5 years in data engineering roles, including at least 2 years in a senior or platform-level position
- Proven experience building production ingestion and transformation pipelines at scale
- Experience contributing to shared platforms or internal developer tooling used by multiple teams
- Python: idiomatic, testable, production-grade code (not only scripting)
- dbt-core: advanced modeling (custom materializations), testing, documentation, packages
- Apache Airflow: DAG design patterns, custom operators, dynamic task mapping, SLA management
- Cloud data platform familiarity with one or more major warehouses: Snowflake, BigQuery, Databricks, Microsoft Fabric
- SQL: complex analytical queries, window functions, query profiling
- Git + CI/CD: trunk-based development, automated testing gates, pipeline-as-code
- Daily user of AI coding assistants such as GitHub Copilot or Claude Code
- Understands limits of AI-generated code and applies rigorous review rather than blind trust
- Interest in LLM-powered data tooling (including RAG pipelines, Cortex, and semantic layers) is a plus
Technologies
- dlt
- dbt-core
- Apache Airflow
- Terraform
- AWS
- Azure
- Python
- Git
- TeamCity
- GitHub Actions
- Snowflake
- BigQuery
- Databricks
- Microsoft Fabric
- SQL
- GitHub Copilot
- Claude Code
- RAG pipelines
- LLM-assisted data cataloguing
- Cortex
- semantic layers
AI and Modern Tooling
- Daily user of AI coding assistants (Copilot, Claude Code or equivalent)
- Applies rigorous review for AI-generated code and does not rely on blind trust
- Interest in LLM-powered data tooling (RAG pipelines, Cortex, semantic layers) is a plus
Location: Auburn Hills, MI (onsite)