DeveloperJobs.io
← Back to all jobs

Job Description

Senior Data Engineer (Foundations) will be a hands-on senior contributor embedded in the Foundations squad, building the shared ingestion platform that supports company-wide data delivery.

Responsibilities

  • Design and implement reusable ingestion components with dlt and dbt-core for both structured and unstructured sources
  • Handle high-volume, append-heavy ingestion and schema-drifting patterns in production pipelines
  • Own the Apache Airflow platform end-to-end: extend and maintain DAGs and shared operators, manage deployments and version upgrades, and provide hands-on support to consuming teams
  • Ensure every pipeline produces incremental loads, data quality checks, and lineage metadata
  • Find and remove redundant ingestion patterns across consuming teams and drive standardization onto shared Platform Foundation components
  • Partner with Solution Architects to evolve platform architecture as new data sources and business needs emerge
  • Support data product exposure by defining and implementing governed interfaces for reliable internal access
  • Contribute to Terraform-managed infrastructure and participate in multi-cloud deployment patterns across AWS and Azure
  • Use and evaluate AI-assisted development tools such as GitHub Copilot and Claude Code to accelerate delivery
  • Promote AI tooling adoption within the squad, including best practices and guardrails for AI-generated code review
  • Explore AI-powered capabilities for internal enablement, including RAG pipelines and LLM-assisted data cataloguing for platform documentation
  • Maintain and improve CI/CD pipelines for Platform Foundation components using TeamCity and GitHub Actions
  • Define and enforce observability standards such as DAG/Task-level alerting and SLA tracking
  • Participate in an on-call rotation for critical ingestion pipelines and drive post-incident improvements
  • Create and maintain Platform Foundation documentation, runbooks, and enablement materials for consuming squads
  • Convert ambiguous or changing business requirements into concrete technical designs, including challenging scope when needed
  • Mentor mid-level engineers and take part in hiring and technical assessments

Requirements

  • Bachelor's degree in Business, Information Systems, Data/Analytics, Computer Science, or a related field
  • Minimum 5 years in data engineering roles, including at least 2 years in a senior or platform-level position
  • Proven experience building production ingestion and transformation pipelines at scale
  • Experience contributing to shared platforms or internal developer tooling used by multiple teams
  • Python: idiomatic, testable, production-grade code (not only scripting)
  • dbt-core: advanced modeling (custom materializations), testing, documentation, packages
  • Apache Airflow: DAG design patterns, custom operators, dynamic task mapping, SLA management
  • Cloud data platform familiarity with one or more major warehouses: Snowflake, BigQuery, Databricks, Microsoft Fabric
  • SQL: complex analytical queries, window functions, query profiling
  • Git + CI/CD: trunk-based development, automated testing gates, pipeline-as-code
  • Daily user of AI coding assistants such as GitHub Copilot or Claude Code
  • Understands limits of AI-generated code and applies rigorous review rather than blind trust
  • Interest in LLM-powered data tooling (including RAG pipelines, Cortex, and semantic layers) is a plus

Technologies

  • dlt
  • dbt-core
  • Apache Airflow
  • Terraform
  • AWS
  • Azure
  • Python
  • Git
  • TeamCity
  • GitHub Actions
  • Snowflake
  • BigQuery
  • Databricks
  • Microsoft Fabric
  • SQL
  • GitHub Copilot
  • Claude Code
  • RAG pipelines
  • LLM-assisted data cataloguing
  • Cortex
  • semantic layers

AI and Modern Tooling

  • Daily user of AI coding assistants (Copilot, Claude Code or equivalent)
  • Applies rigorous review for AI-generated code and does not rely on blind trust
  • Interest in LLM-powered data tooling (RAG pipelines, Cortex, semantic layers) is a plus

Location: Auburn Hills, MI (onsite)

Similar Jobs