Staff Data Engineer, Customer & Identity Resolution (Austin, TX On-Site)
Job Description
About the Role
Since H-E-B Digital Technology’s inception, we have been investing heavily in our customers’ digital experience, reinventing how people find inspiration from food, make food decisions, and ultimately get food into their homes. This is an exciting time to join H-E-B Digital, using the best available technologies to deliver modern, engaging, reliable, and scalable experiences for a growing audience.
As a Staff Data Engineer, you will lead, coach, and mentor engineers and teams while providing technical direction and support. You will partner with Product, Data Science, Application, and Analytics teams to clarify data and data infrastructure needs, resolve data-related technical issues, and ensure optimal data design and efficiency.
Once eligible, you will become an Owner in the company. We are looking for commitment, hard work, and a focus on quality and Customer service. “Partner-owned” means our most important resources, People, drive the innovation, growth, and success that make H-E-B the Greatest Omnichannel Retailing Company.
Core Responsibilities
- Lead teams designing and developing data integrations that support application engineering and system integration.
- Lead design, development, and maintenance of large-scale data pipelines; assist in diagnosing and solving complex production support issues.
- Provide operational and technical expertise to design algorithm patterns that support large and complex datasets for analytics, implement calculations, cleanse data, ensure data standardization, and map and link data from multiple sources.
- Lead engineers across one or more squads to deliver initiatives.
- Mentor and support junior data engineers; coach engineers in engineering techniques, processes, and new technologies.
- Build and support complex data pipelines, APIs, data integrations, data streaming solutions, and predictive model implementations.
- Identify complex data from upstream sources to enable new capabilities.
- Design and build large-scale batch and real-time data pipelines using big data processing frameworks.
- Architect monitoring capabilities based on business SLA and data quality.
- Maintain and streamline existing data pipelines end to end.
- Perform the full SDLC process, including planning, design, development, certification, implementation, and support on more complex projects.
- Establish team operational plans and develop and implement new processes, standards, and operational plans that impact results, including technical roadmaps.
- Make recommendations for overall data platforms, workflows, design, architecture, security, scalability, reliability, and performance.
- Recommend process and tooling changes at the group or department level based on industry standards, patterns, and practices.
- Test technical solutions to ensure data integrity and system functionality, with consideration for broader system and E2E impacts.
- Design scalable and efficient data models to support data integration, storage, and retrieval across complex systems.
- Develop and implement data quality frameworks for data accuracy, consistency, and completeness across workflows.
- Work with Product, Business, and Analyst stakeholders to confirm data quality, discuss requirements, and support data testing.
- Create team documentation and training related to technology stacks and standards.
- Diagnose and troubleshoot extremely complex issues independently.
- Perform data validation and quality assurance on your work and on junior engineers’ work.
- Collaborate with external technical teams to ensure timely, high-quality solutions.
- Engage with shared services teams and vendors as needed.
- Influence others in technical decision-making and technology adoption in the assigned domain.
- Maintain knowledge in machine learning concepts.
Qualifications & Key Requirements
Work Experience
- 7+ years of experience related to data engineering.
- Experience in team leadership.
- Experience working in large-scale infrastructure, large data sets, and mission-critical SLAs.
Knowledge / Skills / Abilities
- Customer Data and Identity Resolution: Hands-on experience architecting customer data platforms, Identity Resolution frameworks, and Customer 360 solutions. Proficient in designing deterministic and probabilistic matching strategies to resolve fragmented identifiers into unified customer profiles while maintaining identity graph stability.
- Cloud: Proven experience working with Google Cloud Platform (GCP) and managed services, including Cloud Dataflow (Apache Beam), Cloud Dataproc (Spark), and orchestration via Managed Service for Apache Airflow (Cloud Composer).
- Programming & Data Processing: Strong proficiency in Python, PySpark, and SQL for large-scale data manipulation and pipeline development.
- Data Architecture & Streaming: Deep understanding of modern Data Lakehouse architectures (Databricks, BigQuery), dimensional data modeling, scalable ETL/ELT pipelines, and real-time streaming using Kafka or Google Cloud Pub/Sub.
- Graph Database Engineering: Experience designing, modeling, and querying relationship analytics in graph databases. Proficient in Google Cloud Spanner Graph (operational traversals) and BigQuery Graph (analytical clustering).
- ML & Entity Resolution: Experience building and operationalizing machine learning pipelines for record linkage and entity resolution. Experience training pairwise classical and probabilistic classification models to score match probabilities using vector embeddings (semantic similarity) and applying graph-based clustering algorithms.
- Data Modeling: Practical expertise in dimensional modeling and designing optimized, high-performance schemas.
- CI/CD & Engineering Best Practices: Experience with streamlined deployment workflows using Git and establishing automated testing and CI/CD pipelines.
- Cross-Functional Collaboration: Track record partnering across Product, Data Science, and Engineering teams to turn business objectives into resilient technical solutions.
Nice-to-Have Qualifications
- Domain Experience: Prior background in retail, e-commerce, or high-volume digital commerce environments.
- Probabilistic Matching Frameworks: Hands-on experience with Splink (Fellegi-Sunter methodology) for probabilistic record linkage and rule-based matching at scale.
- Graph Data Science & Algorithms: Experience applying graph algorithms (for example, Connected Components for grouping matched records, Community Detection) to cluster, analyze, and partition massive identity graphs.
- Graph Ecosystems: Working knowledge of Neo4j and the Cypher / ISO GQL query language.
- Downstream Activation & CDPs: Familiarity with piping resolved customer profiles into CDPs or marketing platforms.
Education
A related degree or comparable formal training, certification, or work experience.
Physical Demands & Working Conditions
- Function in a fast-paced environment.
- Work extended hours.
- Sit for extended periods.
- The work environment characteristics described here are representative of those.
Duties and responsibilities described above represent the general nature and level of work assigned to this position. This is not an exhaustive list of all duties, responsibilities, and skills required. Duties and responsibilities may be modified at any time based on business needs. Employees may be required to perform other job-related tasks as requested by a supervisor, subject to reasonable accommodations.