Senior GCP Data Engineer
Job Description
Persistent is hiring a Senior GCP Data Engineer (Contract) in Charlotte, NC (onsite) with 7 to 12 years of experience to build and optimize GCP data pipelines and data virtualization solutions.
Responsibilities
- Design, develop, and deploy AI agents using Python, LangGraph, Microsoft Agent Framework, and Azure OpenAI
- Build and optimize Survey Programming Agents including:
- Questionnaire parsing
- Routing logic
- JSON generation
- Dimensions setup
- STG generation
- Translation workflows
- DMS script creation
- Link testing
- Survey quality control
- Change impact analysis
- Design, develop, and deploy scalable ETL/ELT data pipelines using Python and GCP-native services
- Build and maintain enterprise-grade data integration across multiple data sources and platforms
- Integrate MongoDB data into Google Cloud Storage and BigQuery environments
- Design and optimize ingestion, transformation, and analytical processing workflows
- Develop and support data virtualization and federated query solutions using Starburst and/or Dremio
- Optimize BigQuery workloads through partitioning, clustering, query tuning, and performance engineering
- Monitor and troubleshoot production data pipelines; resolve data quality issues
- Perform debugging, root cause analysis, and issue resolution for data processing failures
- Optimize Dataflow jobs and MongoDB indexing to improve performance and reduce operational costs
- Implement monitoring, alerting, and operational support processes for data platforms
- Collaborate with Data Scientists, Software Engineers, Architects, and business stakeholders to deliver reliable data solutions
- Develop and maintain technical documentation, operational runbooks, and deployment procedures
- Implement CI/CD for data engineering solutions and cloud deployments
- Ensure adherence to data governance, security, compliance, and operational best practices
Requirements
- 7-12 years of experience in Data Engineering, Cloud Data Platforms, or Big Data environments
- Strong hands-on experience with GCP
- Expertise in BigQuery, Cloud Dataflow, Cloud Composer (Apache Airflow), and Cloud Storage
- Mandatory hands-on experience with Starburst and/or Dremio
- Strong experience implementing ETL/ELT pipelines and enterprise data integration solutions
- Advanced Python programming and scripting expertise
- Strong SQL development skills and query optimization experience
- Experience working with MongoDB and NoSQL database technologies
- Solid understanding of data warehousing, dimensional modeling, and analytical data platforms
- Hands-on experience monitoring and supporting production-grade data environments
- Experience with Cloud Logging, Cloud Monitoring, and incident troubleshooting
- Knowledge of CI/CD implementation and DevOps practices for data engineering deployments
- Strong analytical, troubleshooting, and problem-solving skills
- Experience working in Agile and cross-functional teams
- Excellent communication and stakeholder management abilities
Technologies
- Google Cloud Platform (GCP), BigQuery, Dataflow, Cloud Composer (Apache Airflow), Cloud Storage
- Starburst, Dremio
- Python, LangGraph, Microsoft Agent Framework, Azure OpenAI
- MongoDB, SQL
- Cloud Logging, Cloud Monitoring
Benefits
- Competitive salary and benefits package
- Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
- Opportunity to work with cutting-edge technologies
- Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
- Annual health check-ups
- Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents
Values-driven, People-centric & Inclusive Work Environment
- Hybrid work and flexible hours to fit diverse lifestyles
- Accessibility-friendly office with ergonomic setups and assistive technologies to support employees with physical disabilities
- If you are a person with disabilities and have specific requirements, inform us during the application process or at any time during your employment