At The Hanover Insurance Group, this remote Data Engineer II position sits at the crossroads of cloud analytics and on-prem data operations within the Personal Lines team. The role centers on building and maintaining Azure Data Factory pipelines alongside an on‑prem SQL Server data environment, balancing production support with new development while advancing data quality, monitoring, and release discipline.
Responsibilities
- Provide production support and monitor the daily health of scheduled Azure Data Factory pipelines.
- Develop new pipelines and enhance existing ones to improve resiliency, maintainability, and scalability.
- Implement data validation checks, strengthen monitoring and alerts, and help establish SLAs for data freshness and availability.
- Establish foundational SDLC practices for data engineering, including Git workflows, Dev to Prod promotion practices, and a formal release process.
- Coordinate cross‑team dependencies where upstream ETL timelines affect downstream pipeline completion; design dependency‑aware orchestration and readiness checks.
- Contribute to future‑state Azure data strategy recommendations (Data Lake/Blob Storage, notebooks) and support long‑term migration planning from on‑prem SQL Server to cloud databases (timeline TBD).
- Engage in Azure/Fabric Data Factory engineering activities.
- Design and maintain curated SQL tables and views used for analytics and reporting; optimize for refresh performance and downstream usability.
- Develop and maintain ADF pipelines (Copy activities and Data Flows) with consistent logging, error handling, retries, and notifications.
- Implement parameterization and reusable components to reduce duplication and accelerate development.
- Apply incremental load and backfill strategies appropriate to data volume.
- Production Support & Reliability: monitor daily pipeline execution and triage incidents quickly to restore processing, perform root‑cause analyses, and implement preventative controls and standardized patterns; create and maintain runbooks for critical pipelines and common support scenarios.
- Data Quality, Validation & Trust: build automated validation routines and reconciliation checks (row counts, totals, null/duplicate thresholds, schema drift detection, anomaly flags); collaborate with analysts to define business rules and quality thresholds; document data definitions, transformations, and lineage.
- Stakeholder Collaboration & Dependency Management: work with Operations Data Analysts, business stakeholders, and partner analysts to deliver datasets for robust Power BI models and other analytical solutions; coordinate with upstream teams to align readiness signals and timelines; mentor less experienced team members.
Key measures of success
- Pipelines meeting SLA and on‑time delivery
- Pipeline success rate and reduced manual intervention
- Time to detect and resolve ETL failures (MTTD/MTTR)
- Improvements in query performance, runtime, and cost efficiency
- Reduction in recurring data quality defects; passing completeness and accuracy checks
- Documentation coverage, including runbooks for critical pipelines and data dictionary completeness
Requirements
- Bachelor’s Degree preferred in a related field.
- 6+ years of experience in data engineering, ETL/ELT, or related roles.
- Hands‑on experience building and supporting production pipelines in Azure Data Factory.
- Strong SQL/T‑SQL skills and experience with SQL Server environments supporting analytics and reporting workloads.
- Advanced data modeling capabilities.
- Experience integrating data from multiple sources including databases, flat files/SFTP, and APIs/vendor feeds.
- Familiarity with Azure data platform components (Data Lake/Blob Storage, notebooks) and/or cloud migration planning, or similar platforms.
- Strong troubleshooting, root‑cause analysis, and operational ownership mindset.
- Ability to work directly with stakeholders to gather requirements and translate them into technical solutions.
- Strong communication and documentation skills, with the ability to present findings and translate data into clear documentation.
Technologies
- Azure Data Factory
- SQL Server
- T‑SQL
- Data Lake/Blob Storage
- Notebooks
- Power BI
- Git
- CI/CD
- Python
- PowerShell
- SFTP
Benefits
- Medical, dental, vision, life, and disability insurance
- 401K with a company match
- Tuition reimbursement
- PTO
- Company paid holidays
- Flexible work arrangements
- Cultural Awareness Day in support of IDE
- On‑site medical/wellness center (Worcester only)
Career development
This role is viewed as a long‑term career opportunity. The company supports growth through on‑the‑job experiences, personalized coaching, and robust learning and development programs to help you advance at every level.
EEO statement
The Hanover values diversity in the workplace and among our customers. We provide equal opportunity for employment and promotion to all qualified employees and applicants on the basis of experience, training, education, and ability to do the available work, without regard to race, religion, color, age, sex/gender, sexual orientation, national origin, gender identity, disability, marital status, veteran status, genetic information, ancestry or any other status protected by law. We are committed to an equal opportunity workplace that is free of discrimination and harassment. Individuals with disabilities who wish to request a reasonable accommodation should contact HR Services.
Compensation
The target hiring range for this role is USD 75,000 to 100,000 per year. The posted range reflects our ability to hire at different position titles and levels. Additional compensation may include an annual bonus (which could take the form of a general bonus, sales incentive, or short‑term incentive), long‑term incentive, or spot recognition awards.
Privacy policy
To view our privacy policy and online privacy statement, click here.
Applicants who are California residents
To see the types of information we may collect from applicants and employees and how we use it, please click here.
