DeveloperJobs.io
← Back to all jobs

Job Description

Work onsite in Phoenix, AZ to build and optimize large-scale data solutions using GCP and Big Data tooling.

Responsibilities

  • Design, develop, and optimize data pipelines for large-scale data processing, transformation, and analysis using Big Data and GCP technologies
  • Develop and tune SQL queries and work across database systems
  • Design and optimize data models with a focus on performance and scalability
  • Build event processing pipelines using Kafka or GCP PubSub
  • Apply knowledge of distributed, multi-tiered systems, algorithms, and relational databases

Requirements

  • 6 to 8 years of Data Engineering experience in a Big Data environment
  • Hands-on software development experience with Big Data and Analytics solutions
  • Experience with Hadoop Hive, Spark, Python, and shell scripting
  • Working knowledge of GCP Cloud services including BigQuery, Airflow, DataProc, and PubSub
  • Proficiency in SQL and database systems
  • Experience designing and optimizing data models for performance and scalability
  • Experience building event processing pipelines with Kafka or GCP PubSub
  • Experience with CI/CD pipelines, automated test frameworks, and source code management tools including XLR, Jenkins, and Git
  • Good knowledge and experience with configuration management tools like GitHub

Technologies

  • Hadoop Hive
  • Spark
  • Python
  • Shell scripting
  • GCP Cloud: BigQuery
  • Airflow
  • DataProc
  • PubSub
  • SQL
  • Kafka
  • CI/CD pipelines
  • Automated test frameworks
  • XLR
  • Jenkins
  • Git
  • GitHub

Location

  • Phoenix, AZ (in person)

Salary

  • USD 85,700 - 104,000 per year

Similar Jobs