DeveloperJobs.io
← Back to all jobs

Job Description

Senior Data Engineer role focused on building data patterns, pipelines, integrations, and APIs for analytics plus customer and identity resolution.

Responsibilities

  • Designs data patterns to enable analytics dataset creation, including implementing calculations, cleansing data, standardizing data, and mapping/linking across multiple source systems
  • Performs data validation and quality assurance on junior engineers’ work; creates automated tests
  • Maintains and streamlines existing data pipelines end to end
  • Builds and supports advanced pipelines, application programming interfaces (APIs), integrations, streaming solutions, and predictive model implementations (quality, velocity, etc.)
  • Finds opportunities in upstream data to enable new capabilities
  • Tests technical solutions to ensure data integrity and system functionality, including end-to-end validation
  • Builds large-scale batch and real-time data pipelines using big data processing frameworks
  • Designs and develops data integrations to support application engineering and system integration
  • Designs, develops, and maintains large data pipelines; diagnoses and resolves production support issues
  • Creates documentation and provides training on assigned team technology stacks and standards
  • Designs and implements monitoring capabilities aligned to business SLAs and data quality targets
  • Contributes to refinement of Digital Engineering tools, standards, and training
  • Collaborates with external technical teams to deliver timely, high-quality solutions
  • Works with shared services teams and vendors as needed
  • Partners with Product, Data Science, Application, and Analytics teams to clarify data and data infrastructure needs; helps resolve data-related technical issues; improves data design efficiency
  • Owns the full SDLC cycle including planning, design, development, certification, implementation, and support
  • Engages with Product, Business, and Analyst stakeholders to confirm data quality, align on requirements, and support data testing
  • Conducts peer code reviews and adapts based on review feedback
  • Contributes to design and architecture covering security, scalability, reliability, and performance
  • Mentors and supports junior Data Engineers
  • Builds more complex data models to support insightful analytics while maintaining high data integrity standards
  • Applies knowledge of machine learning concepts

Requirements

  • 5+ years of experience in data engineering
  • A related degree or comparable formal training, certification, or work experience
  • Experience with Lean Startup or Agile development methodologies
  • Experience supporting large-scale infrastructure, large data sets, and mission-critical SLAs
  • Advanced knowledge of Lean Startup and Agile methods
  • High proficiency in Python, PySpark, and SQL for large-scale data manipulation and pipeline development
  • Proven experience with Google Cloud Platform (GCP) and managed services, including Cloud Dataproc (Spark), Cloud Dataflow, Apache Airflow (Cloud Composer), Google Cloud Storage, and Cloud Pub/Sub
  • Knowledge of modern Data Lakehouse architectures (Databricks, BigQuery) and scalable ETL/ELT pipelines
  • Knowledge of business intelligence, analytics and reporting, and application integration
  • Strong understanding of data architecture and data modeling best practices across data and analytics platforms
  • Experience developing and deploying internal data-serving REST APIs (FastAPI), including integration with downstream applications and microservices
  • Strong adherence to software engineering best practices: unit testing, integration testing, code reviews, Git branching strategies, and CI/CD automation
  • Strong prioritization skills
  • Strong verbal and written communication, including data presentation
  • Ability and willingness to learn new technologies as they emerge
  • Ability to remain calm under pressure
  • Ability to work a flexible schedule as needed
  • Ability to collaborate across multiple work locations
  • Team-oriented mindset with willingness to take feedback from peers and mentors

Technologies

  • Python, PySpark, SQL
  • Google Cloud Platform (GCP)
  • Cloud Dataproc (Spark), Cloud Dataflow
  • Apache Airflow (Cloud Composer)
  • Google Cloud Storage, Cloud Pub/Sub
  • Databricks, BigQuery
  • FastAPI, REST APIs
  • ETL/ELT
  • Machine learning concepts
  • CI/CD, Git

Values

  • Heart for People: willing to facilitate solutions with multiple engineers, provide upward communication, and mentor others
  • Head for Business: consistent adherence to standards for coding, infrastructure, and process
  • Passion for Results: capable of high-velocity contributions across multiple technical domains

Nice-to-Have Qualifications

  • Retail concepts background, including POS/transactions and customer/identity resolution
  • Graph ecosystem knowledge: Neo4j and Cypher / ISO GQL

Location & Work Requirements

  • Austin, TX 78702 (onsite)
  • Ability to function in a fast-paced environment with ability to work from office
  • Ability to work extended hours and sit for extended periods

Similar Jobs