This Senior Azure Data Engineering role supports enterprise data modernization and analytics initiatives by building and optimizing scalable Azure and Databricks-based data pipelines, including Delta Lake and Lakehouse architectures. The position is based in Philadelphia, PA with a hybrid schedule.
Role Focus
Design, develop, and optimize cloud data solutions that enable reliable analytics and data modernization. The work centers on scalable ETL/ELT, ingestion and transformation frameworks, orchestration, and end-to-end pipeline observability across Azure-native services and Databricks.
Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines for enterprise data and analytics initiatives.
- Build ingestion, transformation, and orchestration frameworks using Azure-native technologies.
- Develop and optimize data processing solutions using Databricks and PySpark.
- Implement and maintain Delta Lake and Lakehouse architectures.
- Maintain data quality, accuracy, lineage, and reliability across data assets.
- Conduct root cause analysis and troubleshoot data-related issues.
- Support data migration, modernization, and cloud transformation efforts.
- Optimize performance, scalability, and cost efficiency for cloud-based data platforms.
- Build and maintain logical and physical data models.
- Create curated data layers for business intelligence, reporting, and advanced analytics workloads.
- Partner with business users and data consumers to translate requirements into scalable technical solutions.
- Implement CI/CD processes for data engineering solutions and support automated deployment and release management.
- Develop monitoring, logging, and observability capabilities for data pipelines.
- Follow enterprise data governance and security standards.
- Participate in Agile/Scrum ceremonies, including sprint planning, daily stand-ups, retrospectives, and backlog refinement.
- Collaborate with Product Owners, Architects, QA teams, and business stakeholders.
- Contribute to continuous improvement and engineering best practices.
Required Qualifications
- Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field.
- 7+ years of experience in Data Engineering.
- 5+ years of hands-on experience with Azure data technologies.
- 2+ years of experience in Healthcare, Managed Care, Health Plans, or Payer organizations.
- Strong expertise in: Azure Databricks, Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), PySpark, Python, and SQL.
- Extensive experience building enterprise-scale data pipelines and data integration solutions.
- Strong understanding of Data Warehousing, Data Lake, and Lakehouse concepts.
- Experience with large volumes of structured and unstructured data.
- Experience with Git, CI/CD, and DevOps practices.
- Strong analytical, troubleshooting, and problem-solving skills.
Technical Skills and Tools
- Must Have: Azure Databricks, Azure Data Lake (ADLS Gen2), Azure Data Factory (ADF), PySpark, Python, SQL, Delta Lake, Data Modeling, ETL/ELT Development, Azure Cloud Platform, Git, CI/CD, Agile/Scrum.
- Skills: Azure Databricks, Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), PySpark, Python, SQL.
Preferred Qualifications
- Experience working with: Claims Data, Membership Data, Provider Data, Care Management Data.
- Experience with Power BI or other reporting tools.
- Familiarity with FHIR, HL7, or healthcare interoperability standards.
- Experience with Azure DevOps and Infrastructure as Code (IaC).
- Exposure to AI/ML data pipelines and advanced analytics environments.
Location and Work Style
Philadelphia, PA (hybrid): onsite presence in the Philadelphia office at least three days per week.
About the Role
VeeRteq Solutions Inc. is seeking an experienced Data Engineer to lead design and optimization of scalable cloud-based data solutions leveraging Azure, Databricks, and modern Data Lake architectures. This role requires close collaboration with business stakeholders, data architects, analysts, and product teams to support data-driven decision-making.
