Principal Data Engineer
Big Data
Bigdata
Clinical Data
Cloud
Cloud Data Engineering
Cloud Data Platform
Cloud Data Warehouse
Cloud Data Warehouse
Cloud Operations
Cloud Platform
Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Engineering Lead
Data Integration
Data Integration Tools
Data Lakehouse
Data Pipeline
Data Platform
Data Processing
Data Security
Data Warehouse
Data Warehousing
Database
Databases
Databricks
Engineer
ETL
Informatica
Information Technology (IT)
Integration
Lead Data Engineering
Principal Data Analyst
Programming
Programming Language
Programming Languages
Project Management
Snowflake
Software Development
Spark
SQL
Job Description
Abbott Laboratories is hiring a Principal Data Engineer in Madison, WI (onsite).
Responsibilities
- Own the technical outcome of assigned cross-domain initiatives from initial ambiguity through production and lifecycle support.
- Translate enterprise needs into integrated architecture and executable technical plans, breaking complex work into deliverable increments across domains.
- Coordinate technical execution across domain Staff Engineers and partner teams, identifying dependencies and architectural risks.
- Escalate decisions requiring business or delivery authority.
- Stay hands-on across prototypes, reference implementations, critical-path development, technical validation, and production problem solving.
- Define, steward, and evolve enterprise data engineering standards, patterns, and reference architectures.
- Drive convergence where inconsistent or duplicative implementations create enterprise cost, risk, or operational burden.
- Lead enterprise architecture for assigned capabilities, including semantic and metrics layers, canonical data models, and governed data products for analytics, machine learning, and AI.
- Lead enterprise design and code reviews for high-complexity or cross-domain work.
- Coach and mentor Staff and Senior Engineers across domains without formal people-management authority.
- Establish enterprise data-contract standards, including schemas, service expectations, compatibility, breaking-change policy, and producer-consumer responsibilities.
- Partner with Platform Engineering to convert recurring cross-domain needs into shared capabilities.
- Communicate architecture and technical tradeoffs clearly to engineering, business, and executive stakeholders, and produce reusable guidance to improve consistency across Enterprise Data.
- Evaluate architecture tradeoffs across reliability, scalability, performance, security, privacy, operability, adoption, and technical cost.
- Drive cost-efficient use of compute, storage, streaming, and orchestration.
- Serve as enterprise technical escalation for incidents involving multiple domains or shared architecture patterns; lead root-cause analysis and preventive changes for recurring issues.
- Design and review architectures handling protected health information to meet applicable security and privacy requirements, including lineage, audit, Quality Management System, HIPAA, and CLIA.
- Provide technical direction and due diligence for vendor and external-partner solutions.
- Advance responsible engineering practices, including approved AI-assisted development capabilities.
- Work nights and/or weekends as needed.
- Uphold company mission and values through accountability, innovation, integrity, quality, and teamwork.
- Support and comply with the company Quality Management System policies and procedures.
- Maintain regular and reliable attendance.
- Act with an inclusion mindset and model these behaviors for the organization.
Requirements
- Bachelor’s Degree in Data Science, Computer Science, Information Systems, Mathematics, or Engineering; or High School Diploma/General Education Degree with 12 years of relevant experience in lieu of the degree.
- Expert-level experience with software development design and development plus relevant domain-specific skills.
- Spark on Databricks or comparable platforms.
- Experience with Python, Scala, SQL, and Snowflake.
- ETL and ELT data pipeline experience, including batch and event-driven patterns.
- Experience designing and implementing data modeling solutions using relational, dimensional, and/or NoSQL databases.
- Database architecture testing methodology, including execution of test plans, debugging, and testing scripts/tools.
- Experience with open data file/table formats: Parquet, Avro, Delta Lake.
- Experience with cloud infrastructure and delivery services: AWS, S3, SQS, and GitLab CI/CD.
- REST API development experience; familiarity with BI concepts and Tableau performance considerations.
- Agile development tool experience including JIRA and Confluence repository.
- Demonstrated ability to lead through influence across multiple teams and communicate complex technical decisions to senior engineering, business, and executive stakeholders.
- Ability to perform the essential duties of the position with or without accommodation.
Preferred Qualifications
- Enterprise-scale experience with Databricks, Apache Spark, Delta Lake, and Unity Catalog.
- Experience with Kafka, change data capture, and production event-streaming architectures.
- Experience designing semantic or metrics layers, canonical data models, and governed data products.
- Cloud data architecture experience in AWS, Azure, or Google Cloud Platform.
- Experience in life sciences, diagnostics, or clinical laboratory environments involving protected health information and requirements such as HIPAA, CLIA, FDA, or Quality Management System.
- Experience providing technical assessment and architecture direction for vendor and external-partner platforms.
Technologies
- Databricks, Unity Catalog, Spark
- Python, Scala, SQL, Snowflake
- ETL, ELT, Parquet, Avro, Delta Lake
- AWS, S3, SQS, GitLab CI/CD
- REST API, Tableau
- JIRA, Confluence
- HIPAA, CLIA
Base Pay Range
USD 129,300 - 258,700 per year. In specific locations, pay range may vary from the range posted.