Senior Data Engineer, Customer & Identity Resolution
Senior
Artificial Intelligence
Big Data
Cloud
Cloud Native
Cloud Platforms
Customer Data Platform
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Integration
Data Integration Tools
Data Pipeline
Data Platform
Data Processing
Database
Databases
Engineer
ETL
Google Cloud
Identity Resolution
Informatica
Information Technology (IT)
Integration
Reporting and Analytics
Software Development
SQL
Job Description
Senior Data Engineer role focused on building data patterns, pipelines, integrations, and APIs for analytics plus customer and identity resolution.
Responsibilities
- Designs data patterns to enable analytics dataset creation, including implementing calculations, cleansing data, standardizing data, and mapping/linking across multiple source systems
- Performs data validation and quality assurance on junior engineers’ work; creates automated tests
- Maintains and streamlines existing data pipelines end to end
- Builds and supports advanced pipelines, application programming interfaces (APIs), integrations, streaming solutions, and predictive model implementations (quality, velocity, etc.)
- Finds opportunities in upstream data to enable new capabilities
- Tests technical solutions to ensure data integrity and system functionality, including end-to-end validation
- Builds large-scale batch and real-time data pipelines using big data processing frameworks
- Designs and develops data integrations to support application engineering and system integration
- Designs, develops, and maintains large data pipelines; diagnoses and resolves production support issues
- Creates documentation and provides training on assigned team technology stacks and standards
- Designs and implements monitoring capabilities aligned to business SLAs and data quality targets
- Contributes to refinement of Digital Engineering tools, standards, and training
- Collaborates with external technical teams to deliver timely, high-quality solutions
- Works with shared services teams and vendors as needed
- Partners with Product, Data Science, Application, and Analytics teams to clarify data and data infrastructure needs; helps resolve data-related technical issues; improves data design efficiency
- Owns the full SDLC cycle including planning, design, development, certification, implementation, and support
- Engages with Product, Business, and Analyst stakeholders to confirm data quality, align on requirements, and support data testing
- Conducts peer code reviews and adapts based on review feedback
- Contributes to design and architecture covering security, scalability, reliability, and performance
- Mentors and supports junior Data Engineers
- Builds more complex data models to support insightful analytics while maintaining high data integrity standards
- Applies knowledge of machine learning concepts
Requirements
- 5+ years of experience in data engineering
- A related degree or comparable formal training, certification, or work experience
- Experience with Lean Startup or Agile development methodologies
- Experience supporting large-scale infrastructure, large data sets, and mission-critical SLAs
- Advanced knowledge of Lean Startup and Agile methods
- High proficiency in Python, PySpark, and SQL for large-scale data manipulation and pipeline development
- Proven experience with Google Cloud Platform (GCP) and managed services, including Cloud Dataproc (Spark), Cloud Dataflow, Apache Airflow (Cloud Composer), Google Cloud Storage, and Cloud Pub/Sub
- Knowledge of modern Data Lakehouse architectures (Databricks, BigQuery) and scalable ETL/ELT pipelines
- Knowledge of business intelligence, analytics and reporting, and application integration
- Strong understanding of data architecture and data modeling best practices across data and analytics platforms
- Experience developing and deploying internal data-serving REST APIs (FastAPI), including integration with downstream applications and microservices
- Strong adherence to software engineering best practices: unit testing, integration testing, code reviews, Git branching strategies, and CI/CD automation
- Strong prioritization skills
- Strong verbal and written communication, including data presentation
- Ability and willingness to learn new technologies as they emerge
- Ability to remain calm under pressure
- Ability to work a flexible schedule as needed
- Ability to collaborate across multiple work locations
- Team-oriented mindset with willingness to take feedback from peers and mentors
Technologies
- Python, PySpark, SQL
- Google Cloud Platform (GCP)
- Cloud Dataproc (Spark), Cloud Dataflow
- Apache Airflow (Cloud Composer)
- Google Cloud Storage, Cloud Pub/Sub
- Databricks, BigQuery
- FastAPI, REST APIs
- ETL/ELT
- Machine learning concepts
- CI/CD, Git
Values
- Heart for People: willing to facilitate solutions with multiple engineers, provide upward communication, and mentor others
- Head for Business: consistent adherence to standards for coding, infrastructure, and process
- Passion for Results: capable of high-velocity contributions across multiple technical domains
Nice-to-Have Qualifications
- Retail concepts background, including POS/transactions and customer/identity resolution
- Graph ecosystem knowledge: Neo4j and Cypher / ISO GQL
Location & Work Requirements
- Austin, TX 78702 (onsite)
- Ability to function in a fast-paced environment with ability to work from office
- Ability to work extended hours and sit for extended periods