Senior Data Engineer

Dahl Consulting is offering a contract Senior Data Engineer opportunity for 6 to 12 months on a data services engineering team in a fraud and abuse technology environment. This hybrid role is based in Brooklyn Park, MN, focused on designing, modernizing, and operating scalable batch and streaming data pipelines across Hadoop and Google Cloud Platform (GCP).

In this role, you will help build trusted datasets that support analytics, fraud investigation, reporting, and machine learning use cases, while also improving reliability through strong data validation, monitoring, and operational practices.

Responsibilities

• Design, build, test, deploy, and support scalable batch and streaming data pipelines using Python, SQL, Apache Spark, Kafka, and related technologies.
• Modernize and migrate data pipelines and datasets from on-premises Hadoop environments to GCP, including BigQuery and other cloud-native services.
• Build and optimize ETL/ELT workflows for large-scale, high-volume datasets with a focus on performance, scalability, reliability, and cost efficiency.
• Develop trusted datasets supporting analytics, fraud investigation, reporting, and machine learning use cases.
• Implement data quality checks, testing, and validation processes.
• Troubleshoot production data issues, perform root-cause analysis, and implement durable solutions to prevent recurring failures.
• Implement and maintain monitoring, alerting, automation, and CI/CD practices for production data pipelines.
• Collaborate across both legacy Hadoop and modern GCP environments during platform modernization.
• Partner with engineers, analysts, data scientists, product teams, and other stakeholders to understand data requirements and deliver effective solutions.
• Identify technical risks, data quality concerns, and performance issues, and recommend improvements to platform reliability.
• Contribute to technical design, code reviews, engineering standards, documentation, and knowledge transfer.

Requirements

• 5+ years of hands-on data engineering or related software engineering experience building large-scale data systems.
• Strong proficiency in Python and SQL.
• Hands-on experience with Apache Spark and distributed data processing.
• Hands-on experience with the Hadoop ecosystem, including HDFS and Hive.
• Hands-on experience with Kafka or another event-streaming technology.
• Experience building and supporting ETL/ELT and data integration pipelines.
• Experience with GCP data services, particularly BigQuery.
• Experience optimizing data pipelines for performance, scalability, reliability, and cost.
• Experience supporting production data systems, including monitoring, logging, incident troubleshooting, and operational support.
• Working knowledge of Git, CI/CD, automated testing, and Agile engineering practices.
• Ability to independently own data engineering work from design through production deployment and ongoing support.
• Strong communication and collaboration skills across technical and business teams.

Technologies

Python, SQL, Apache Spark, Kafka, Hadoop, HDFS, Hive, Google Cloud Platform (GCP), BigQuery, ETL, ELT, Git, CI/CD, Agile

Preferred Qualifications

• Experience with additional GCP services such as Dataproc, Dataflow, Pub/Sub, or Cloud Storage.
• Experience migrating Hadoop workloads or enterprise data platforms to GCP or another public cloud environment.
• Experience with Spark Streaming, Flink, or similar stream-processing frameworks.
• Experience with data governance, lineage, security, privacy, and data quality frameworks.
• Experience with orchestration tools such as Airflow, Cloud Composer, Oozie, or similar technologies.
• Experience supporting machine learning data pipelines or GCP Vertex AI/MLOps workflows.
• Experience working with high-volume enterprise data in fraud, risk, security, retail, or e-commerce environments.

Compensation

$67 to $112 per hour (USD)

Benefits

Dahl Consulting is proud to offer a comprehensive benefits package to eligible employees that will allow you to choose the best coverage to meet your family’s needs. For details, please review the DAHL Benefits Summary: https://www.dahlconsulting.com/benefits-w2fta/.

How to Apply

To submit yourself for consideration, click the apply button and complete the mobile-friendly online application. After reviewing your details, a recruiter will reach out with next steps.

Equal Opportunity Statement: As an equal opportunity employer, Dahl Consulting welcomes candidates of all backgrounds and experiences to apply.