As a Senior Data Engineer, you will lead the creation and ongoing operation of a centralized data and analytics platform built on Microsoft Fabric, integrating healthcare and business systems into reliable, governed data products.
Role Focus
This position emphasizes data infrastructure, ingestion pipelines, and analytics-ready data modeling. Dashboard development and Power BI are not the primary responsibilities, though Power BI reporting support is part of enabling trusted datasets for downstream use.
Responsibilities
- Design and implement the Microsoft Fabric data architecture using OneLake, Lakehouse/Warehouse, Data Factory pipelines, notebooks, and related Fabric services.
- Build and maintain data ingestion pipelines from eClinicalWorks, HubSpot, REST APIs, APIs, databases, files, and future enterprise systems.
- Use REST APIs, FHIR APIs, webhooks, batch exports, and other integration methods to extract source-system data reliably.
- Establish raw, validated, and analytics-ready layers using a medallion-style architecture.
- Clean, standardize, deduplicate, and reconcile data across systems.
- Create data models for operational entities including patients, referrals, appointments, providers, procedures, cases, and outcomes.
- Develop record-matching methods for scenarios with inconsistent or missing identifiers.
- Produce reusable curated datasets that support Power BI reporting, operational analytics, automation, and future AI applications.
- Implement incremental loading, change detection, schema management, error handling, logging, alerting, and pipeline monitoring.
- Define data quality checks and processes to identify incomplete, inconsistent, or unexpected source data.
- Apply appropriate security, access controls, and governance practices for healthcare data.
- Use Git and CI/CD practices for version-controlled, documented, and maintainable pipelines and transformations.
- Monitor Fabric usage, performance, capacity, and cost as the platform scales.
- Document architecture, data definitions, pipelines, dependencies, and operating procedures.
- Maintain and improve the platform after initial delivery instead of treating it as a one-time migration.
- Assess existing systems, data sources, reporting requirements, and the Microsoft environment to shape the initial Fabric setup.
- Design and configure the initial Fabric architecture and establish secure ingestion from HubSpot and eClinicalWorks.
- Create a raw historical layer that preserves source-system data, plus cleaned and standardized datasets that reconcile cross-system information.
- Build an initial analytics-ready model to support high-priority operational reporting, with automated monitoring, validation, documentation, and recovery.
- Establish engineering standards that enable adding new data sources and use cases without rebuilding the platform.
Required Qualifications
- 4+ years of professional experience in data engineering, data platform work, or data integration.
- Hands-on experience delivering production solutions with Microsoft Fabric or a closely related Azure data stack.
- Strong SQL skills.
- Strong Python and/or PySpark skills.
- Experience designing ETL/ELT pipelines and data ingestion frameworks.
- Experience with Microsoft Fabric Lakehouse, OneLake, Data Factory / Pipelines, notebooks, Delta tables, or equivalent technologies.
- Experience integrating third-party systems via REST APIs.
- Experience implementing incremental data loads rather than relying only on full refreshes.
- Strong understanding of dimensional modeling, relational modeling, lakehouse architecture, and analytics-oriented data modeling.
- Experience troubleshooting API authentication, pagination, rate limits, schema changes, malformed records, and pipeline failures.
- Experience with Git and CI/CD practices.
- Experience implementing monitoring, logging, data-quality validation, and alerting for production pipelines.
- Ability to work independently and make practical architectural decisions while the data platform is still being built.
- Healthcare experience is strongly preferred.
Technology Stack
Microsoft Fabric, OneLake, Lakehouse, Warehouse, Data Factory, Pipelines, Notebooks, Delta tables, REST APIs, FHIR APIs, Webhooks, Python, PySpark, Git, CI/CD, Power BI, eClinicalWorks, HubSpot, FHIR, HL7, Microsoft Entra ID, Azure security, Microsoft Purview, Master Patient Index
Healthcare / Integration Experience (Preferred)
- Healthcare experience is strongly preferred.
- Particularly valuable experience includes healthcare EHR/EMR data, eClinicalWorks, FHIR / HL7, healthcare operational or clinical data models, and HIPAA-regulated data environments.
- Experience with patient identity matching or Master Patient Index concepts.
- HubSpot APIs and CRM data experience.
- Microsoft Entra ID and Azure security experience.
- Microsoft Purview or other data governance tools.
- Power BI semantic models.
- Direct eClinicalWorks experience is a major plus, but strong healthcare API/FHIR integration experience is acceptable if you can learn unfamiliar source systems quickly.
Compensation and Location
- Location: Irving, TX (onsite). Listing also notes the Dallas–Fort Worth, TX area.
- Employment Type: Full-Time
- Work Arrangement: In-Office
- Compensation: USD 125,000 - 150,000 per year (depending on experience)
- Pay: $125,000.00 - $150,000.00 per year
- Work Location: In person
Benefits
- 401(k)
- 401(k) matching
- Dental insurance
- Health insurance
- Life insurance
- Paid time off
- Vision insurance
