Senior Data Engineer

Vytwo is seeking a Senior Data Engineer to join its team in Minneapolis, MN, working onsite. In this role, you will design and maintain scalable data pipelines and help optimize Databricks and Spark workloads so teams can rely on analytics-ready data.

You will focus on building robust ingestion and transformation flows for large-scale structured and semi-structured datasets, with an emphasis on performance, reliability, and cost-aware operations within Databricks clusters.

What you will do

• Design, develop, and maintain scalable data pipelines using Apache Spark with PySpark and/or Scala
• Build and optimize data workflows on Databricks, including Delta Lake, notebooks, and scheduled jobs
• Ingest, transform, and curate large-scale structured and semi-structured datasets
• Perform performance tuning and cost optimization for Spark workloads and Databricks clusters
• Implement data quality checks, monitoring, and error handling
• Collaborate with analytics and business stakeholders to deliver well-modeled, analytics-ready data
• Support batch processing and, where applicable, streaming data pipelines
• Follow best practices for testing, documentation, security, and version control

What you bring

• 5+ years of experience in Data Engineering or a related role
• Strong hands-on experience with Apache Spark using PySpark or Scala
• Proven experience working in Databricks environments
• Strong SQL skills, including experience with relational and analytical databases
• Experience building and maintaining ETL/ELT pipelines at scale
• Familiarity with modern data lake architectures, with Delta Lake preferred
• Experience using Git-based version control
• Must be a U.S. Citizen

Tools you will work with

• Apache Spark, PySpark, Scala
• Databricks, Delta Lake
• Git, SQL

Location and work options

• Minneapolis, MN (onsite)
• Must be currently located in Minnesota (MN), or willing to relocate
• Flexible work from home options available