DeveloperJobs.io
← Back to all jobs

Job Description

Lead data engineering teams to build and support scalable, cloud-first data platforms using lakehouse and modern pipeline technologies.

Responsibilities

  • Direct Agile teams delivering scalable data engineering solutions across full-stack and cloud platforms
  • Guide a team of developers, data analysts, and data scientists with experience in machine learning, distributed microservices, lakehouse architecture, and full-stack systems
  • Stay hands-on by reviewing architecture designs, data models, and pipeline code across Python, Spark, Databricks, and Snowflake environments
  • Drive technical patterns and standards for code quality, maintainability, and reusability across data platforms and pipelines
  • Oversee the design and health of data platforms and pipelines, setting expectations for scalability, resilience, data quality, and operational efficiency
  • Coordinate with product managers and software engineers to deliver robust cloud-first data solutions
  • Communicate technical concepts and data outcomes clearly to internal and external stakeholders to maintain alignment
  • Serve as a force-multiplier through leading, mentoring, and growing a diverse team of data engineers
  • Lead end-to-end, large-scale transformative data initiatives, making critical architectural decisions and evaluating platform choices (for example, Snowflake vs Databricks) based on technical and business needs
  • Contribute to internal and external technology communities through experimentation, learning, and mentoring within the data community

Requirements

  • Bachelor’s degree or higher in Computer Science or a related quantitative field (Statistics, Economics, Operations Research, Analytics, Mathematics, Engineering)
  • 6+ years of experience in application development (internship experience does not apply)
  • 3+ years of people management experience
  • 2+ years driving technical delivery of roadmap features
  • 4+ years programming with at least one of: Python, Java, or Scala
  • 4+ years designing and developing data pipelines
  • 2+ years in data modeling and designing end-to-end data solutions using both relational and non-relational database systems

Technologies

  • Databricks, PySpark, Snowflake, Spark, Python, Java, Scala
  • Machine learning, distributed microservices, lakehouse architecture
  • AWS, Microsoft Azure, Google Cloud
  • EMR, Glue, Airflow, Dagster
  • Monte Carlo, Splunk
  • MongoDB, Cassandra, DynamoDB, Redshift
  • NoSQL, SQL

Preferred Qualifications

  • Master’s Degree in Computer Science or a related field
  • 9+ years of experience in application development with demonstrated proficiency in Python, SQL, Scala, or Java
  • 5+ years hands-on experience designing, deploying, and operating data workloads in at least one public cloud (AWS, Microsoft Azure, or Google Cloud)
  • 5+ years building or supporting distributed data or compute workloads using tools such as EMR, Spark, Glue, or Databricks
  • 5+ years designing, implementing, and operating real-time or streaming data pipelines
  • 3+ years data observability experience (e.g., Monte Carlo, Splunk) or data orchestration (e.g., Airflow, Dagster)
  • 5+ years working with unstructured or semistructured data using NoSQL databases (e.g., MongoDB, Cassandra, DynamoDB)
  • 5+ years designing and supporting data warehousing solutions (e.g., Snowflake, Redshift)
  • 3+ years working in an Agile development environment
  • 3+ years developing user-centric reusable data products

Benefits

  • Performance based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI)
  • Comprehensive, competitive, and inclusive set of health, financial and other benefits supporting total well-being

Compensation

  • Chicago, IL (Onsite): USD 209,000 - 238,500 per year

Experience: 6+ years • Education: Bachelor’s degree in Computer Science or related quantitative field

Similar Jobs