Senior Data Engineer
Job Description
SCP Health is seeking a Senior Data Engineer for an onsite role in Lafayette, LA. This position leads the design and operationalization of a data platform spanning ingestion, transformation, orchestration, and serving, with ownership of critical data products, governance, reliability, and incident response.
Role Overview
The Senior Data Engineer defines and evolves reference architectures and standards for core platform layers, ensuring solutions are scalable, secure, observable, and maintainable. The role owns end-to-end delivery for high-impact datasets and pipelines used in revenue cycle, physician scheduling, billing, and performance monitoring, and serves as an escalation point for major incidents across critical pipelines and domains.
Key Responsibilities
- Define and evolve reference architectures and standards for ingestion, transformation, orchestration, and serving layers, including Snowflake + dbt and adjacent tooling; lead design reviews and set guardrails for reliable and observable delivery.
- Own end-to-end delivery and operational excellence for critical datasets and pipelines; define SLAs/SLOs, establish support models, and manage reliability through error budgets where applicable.
- Lead strategy and implementation for complex, multi-facility integrations, including EHR extracts, HL7, FHIR, APIs, and flat files; execute backfills and reconciliation and establish maintainable patterns.
- Own and enforce platform-wide conventions across Bronze/Silver/Gold layers, including naming, modeling, testing, documentation, lineage, and certification to reduce ambiguity across domains.
- Serve as an escalation point for major incidents impacting critical pipelines or multiple domains; coordinate triage, resolution, blameless post-incident reviews, and systemic fixes.
- Lead Snowflake performance and cost optimization initiatives, including warehouse/workload design, scheduling, resource monitors, and query tuning; communicate tradeoffs for reliability, latency, and consumption.
- Architect and implement HIPAA-aligned controls for PHI/PII, including least-privilege RBAC, masking, row/column policies, auditing, and secure sharing; partner with Security/Compliance on reviews and remediation.
- Define and operationalize platform-level data quality strategy using tests, reconciliation, anomaly detection, and measurable freshness/completeness targets, and promote consistent adoption across teams.
- Lead technical discovery to clarify ambiguous requirements, propose implementation options, and align stakeholders on acceptance criteria, change impacts, and rollout plans.
- Mentor Data Engineers I–III through coaching, pairing, and feedback; raise engineering standards via reviews, reusable libraries/macros, documentation standards, and practical tradeoff guidance.
- Improve data CI/CD and release practices including testing gates, automated checks, environment promotion, release strategy, and rollback planning to improve delivery predictability.
- Partner with the Data Engineering Manager and stakeholders to shape the platform roadmap, identify systemic risks and dependencies, and recommend investments to improve scalability, reliability, governance, and velocity.
Required Qualifications
- Expert-level ability to design, optimize, and troubleshoot complex SQL workloads, including diagnosis at the query and system level and scalable access/transformation patterns.
- Deep expertise with Snowflake architecture and operations, including workload strategy, governance controls, performance tuning, cost optimization, and operational monitoring.
- Advanced proficiency with dbt, including project organization, macros, testing and documentation strategy, incremental modeling, and environment promotion.
- Ability to define canonical models and data products across domains and manage change over time (including SCDs), driving alignment on definitions.
- Ability to define SLAs/SLOs, design monitoring/alerting and runbooks, lead incident response and postmortems, and drive systemic improvements through problem management.
- Advanced knowledge of batch and near-real-time patterns (watermarking, CDC, replay/backfill strategy, idempotency), plus experience with APIs, authentication, and semi-structured/columnar formats such as JSON and Parquet.
- HIPAA-aligned design capability for PHI/PII including least-privilege access models, masking, row/column policies, auditability, and secure sharing, with partnership in remediation efforts.
- Ability to conduct technical discovery and design reviews, make pragmatic tradeoffs, and influence without formal authority through enablement and consistent standards.
- Experience identifying systemic issues and investment opportunities, articulating multi-quarter plans, and evaluating build vs. buy tradeoffs with risk-aware sequencing.
- Strong communication skills with technical and non-technical partners, including transparent updates on status, risk, and tradeoffs.
- Bachelor’s degree (BA/BS) in MIS, Computer Science, or a related field; or an equivalent combination of education and experience.
- 8+ years of progressive professional experience in Data Engineering, Analytics Engineering, or a related technical role, including ownership of production data pipelines and cross-domain data products and leadership of complex initiatives.
- 4+ years of hands-on experience building and operating solutions within the Snowflake ecosystem, including performance tuning and practical cost management.
- Healthcare, Revenue Cycle (RCM), and/or integrating clinical data feeds is strongly preferred, and experience working with PHI/PII and regulated data is expected.
- Agile/Scrum experience including technical leadership across design reviews, estimation, delivery coordination, and mentoring.
- HIPAA Security/Privacy training is required and/or related healthcare compliance certifications are preferred.
- Relevant certification or license in Data, Analytics, Cloud, or Security is a plus.
Technologies
- Snowflake
- dbt
- SQL
- HL7
- FHIR
- APIs
- EHR extracts
- Flat files
- HIPAA
- RBAC
- JSON
- Parquet
- CDC
Compensation
$86,539.00 - $128,101.00 USD per year.
Work Location
- Primary location: Lafayette, LA (Corporate HQ)
- Work arrangement: Onsite
- Secondary locations: Atlanta, GA; Dallas, TX
Benefits
- Comprehensive benefits package
- Competitive compensation
- Medical, dental, vision insurance
- 401(k) plan with a company match
- Paid time off and holidays
- Professional development support
- Employee wellness resources
- Work-life balance
Certificates and Licenses
- Relevant certification or license in Data, Analytics, Cloud, or Security is a plus.
- SnowPro Advanced: Data Engineer (preferred) or SnowPro Core Certification.
- dbt Analytics Engineering Certification (preferred), with demonstrated experience operating dbt in production.
- HIPAA Security/Privacy training (required) and/or related healthcare compliance certifications (preferred).
Work Environment and Physical Demands
- Professional setting.
- Continuous sitting.
- Continuous oral and written communication and listening skills.
- Continuous computer use.
- Occasional bending, kneeling, lifting, pulling, and pushing up to 10 pounds.
- Minimum travel, when applicable.
- Job requires a high level of mental awareness.
Why SCP Health
- Strong track record of providing excellent work/life balance.
- Comprehensive benefits package and competitive compensation.
- Commitment to an inclusive culture of belonging and empowerment through core values: collaboration, courage, agility, and respect.