Senior Data Engineer – Spark / Scala

Astyra Corporation is hiring a Senior Data Engineer for a contract role in Richmond, VA. This position focuses on modernizing legacy ETL workloads and implementing Spark-based data pipelines, with emphasis on data accuracy, Spark job performance, and validation between legacy and target platforms.

Responsibilities

• Design, build, and maintain scalable data pipelines using Apache Spark and Scala
• Assist with the migration of legacy ETL workloads into a modern Spark-based data framework
• Develop and improve data transformation and processing solutions
• Conduct data comparison, reconciliation, and validation between legacy and target platforms
• Identify and troubleshoot data discrepancies during the migration lifecycle
• Implement data quality checks and controls to support completeness and accuracy
• Optimize Spark jobs and pipelines to improve performance, scalability, and reliability
• Collaborate with data engineers, architects, and other technical teams during data modernization
• Support testing, deployment, troubleshooting, and ongoing maintenance for production data pipelines
• Contribute to broader feature engineering and data modernization initiatives

Requirements

• 7+ years of experience in data engineering or a related technical discipline
• Strong hands-on experience designing and developing enterprise data pipelines
• Advanced experience with Apache Spark
• Strong development experience with Scala
• Experience with ETL development, modernization, and migration initiatives
• Strong understanding of data transformation, data quality, and reconciliation
• Experience comparing and validating data across legacy and modernized systems
• Experience developing and supporting production data-processing workloads
• Familiarity with modern CI/CD and software development practices
• Strong troubleshooting and problem-solving skills

Technologies

• Apache Spark
• Spark
• Scala
• Python
• ETL
• CI/CD

Benefits

• 401(k)
• Dental insurance
• Health insurance
• Vision insurance

Preferred Qualifications

• Experience migrating legacy ETL platforms to Spark-based architectures
• Experience supporting large-scale technology or data modernization initiatives
• Experience in financial services or other highly regulated environments
• Python experience
• Exposure to Generative AI or AI-assisted development and modernization tools
• Previous experience within a large enterprise engineering environment

Contract Details

• Job type: Contract
• Location: Richmond, VA 23238 (Required)
• Work model: Hybrid remote in Richmond, VA 23238
• Expected hours: 40.0 per week
• Compensation: USD 66.00 - 71.00 per hour