Home Depot’s Online Data Engineering team builds the infrastructure that translates business requirements into reliable customer data capture and actionable analytics. In this Senior Data Engineer role, you will design, implement, and maintain enterprise data pipeline architectures, data validation methods, and in-house solutions.
Role Focus
You will build and maintain database pipeline architectures, create new data validation methods, and develop data analysis tools. The role also includes developing APIs to retrieve data and supporting in-house enterprise solutions through monitoring, continual testing, and validation to improve reliability and confidence.
Using big-data techniques, you will cleanse, organize, and transform datasets while maintaining, defending, and updating data structures and integrity on an automated basis. You will also establish design standards and assurance processes for data connections, flows, and storage to ensure compatibility and operability across systems and applications.
Additionally, you will review internal and external business and product requirements for data operations and activities, and recommend system or storage changes and upgrades to accommodate ongoing needs.
Responsibilities
- Apply extensive business knowledge to drive solution approaches and develop trust with internal customers and cross-functional teams.
- Provide education on advanced analytics to technical and non-technical partners.
- Demonstrate a deep understanding of IT needs to help the team address business problems.
- Identify new business opportunities where data science can create competitive advantage.
- Partner with project teams and business stakeholders to define project goals, guide prioritization, and ensure quality of outcomes.
- Provide mentoring and coaching to support technical competencies for more junior roles.
- Collaborate with managers and the team to distribute workload and resources, and support recruiting and hiring efforts.
- Execute assigned work with high efficiency and quality using appropriate advanced analytical methodologies.
- Communicate insights and recommendations to technical and non-technical leaders, including clear articulation of recommendation impacts to drive alignment and implementation.
- Prepare reports, updates, and presentations on project progress and solution status.
- Lead requirements gathering, design, and development for complex applications and programs.
- Continue learning key developments in advanced analytics, technical skill sets, and additional data sources.
- Contribute to continuous improvement of predictive and prescriptive analytics by developing replicable solutions.
Core Technologies
- Python, JavaScript, React
- ETL and ETL pipelines, Batch pipelines, Streaming pipelines
- BigQuery, PySpark, AirFlow, DataProc
- SQL
- CI/CD
Location and Work Requirements
- Location: Atlanta, GA (onsite)
- Travel: Typically requires overnight travel 5% to 20% of the time
- Reporting: Typically reports to Manager or Above
- Direct reports: 0
Minimum Qualifications
- Minimum 4 years of experience
- Must be 18 years of age or older
- Must be legally permitted to work in the United States
- Education: bachelor’s degree program or equivalent degree in a field related to the job
Preferred Qualifications
- Bachelors or Masters in a quantitative field (Analytics, Computer Science, Math, Physics, Statistics, etc.) or relevant work experience
- 5+ years of relevant work experience
- Demonstrated experience in predictive modeling, data mining, and data analysis
- Experience developing and testing ETL jobs/pipelines, configuring orchestration, automated CI/CD, writing automation scripts, and supporting pipelines in production
- Experience defining and capturing metadata and rules associated with ETL processes
- Experience building Batch and Streaming pipelines
- Prior direct experience writing analytical SQL queries and performance-tuning queries
- Ability to stitch and maintain data from multiple sources
- Experience using JavaScript and front-end development frameworks (React, Nucleus), and QA apps (Retina, KPI Shield, Alert Goose)
- Ability to produce tags for site data
- Ability to code in Python and Google BigQuery to stitch and enrich raw data from multiple sources
- Proven ability to use PySpark, AirFlow, and DataProc to engineer and automate data flow pipelines
- Ability to optimize pipeline runtime and reduce cost on slots and storage consumption
- Ability to prioritize requests and manage a product roadmap
- Coaching junior engineers to improve code, best practices, and understanding of data engineering principles
- Strong verbal and written communication skills at all levels
Working Conditions and Physical Requirements
- Most of the time is spent sitting in a comfortable position, with frequent opportunity to move about.
- On rare occasions, there may be a need to move or lift light articles.
- Work is performed in a comfortable indoor area; any unpleasant conditions would be infrequent and not objectionable.
Additional Leadership and Certifications
- Minimum leadership experience: No previous leadership experience
- Preferred leadership experience: No previous leadership experience
- Certifications: None
Competencies
- Action Oriented
- Collaborates
- Drives Engagement
- Communicates Effectively
- Customer Focus
- Drives Results
- Manages Conflict