Senior Databricks Data Engineer

CGI Group, Inc. offers a Senior Databricks Data Engineer role in Miami, FL (hybrid) with competitive compensation and a benefits package designed to support your health, learning, and family needs. You will help build scalable data integration solutions end to end, owning production ETL/ELT pipelines and the engineering lifecycle from design through deployment and support.

Compensation: USD 91,800 to 203,800 per year (U.S. estimate). CGI anticipates accepting applications through November 13, 2026.

What you’ll do

Design, develop, and implement scalable enterprise data integration solutions using Databricks and Apache Spark. This position includes ownership of the end-to-end engineering lifecycle for production ETL/ELT pipelines, from requirements analysis to deployment readiness.

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, Apache Spark, Python/PySpark, and SQL
• Build batch and incremental data integration pipelines from files, databases, and enterprise data sources
• Develop reusable, maintainable data processing solutions and complex SQL transformations for cleansing, enrichment, and aggregation
• Optimize Spark workloads for performance, scalability, reliability, and cost efficiency
• Implement data quality controls, reconciliation, monitoring, and error handling
• Design and maintain curated datasets, dimensional models, and reporting views to support enterprise analytics and downstream consumers
• Implement and maintain pipeline orchestration using Airflow, Databricks Workflows, or comparable tools
• Apply CI/CD practices for version control, automated deployment, and release management
• Troubleshoot pipeline failures and production performance issues
• Collaborate with data architects, integration teams, engineering teams, and business stakeholders
• Prepare and present technical solution designs for architecture and engineering review, incorporate feedback, and document designs, decisions, and operational procedures

What you bring

• Minimum 5 years of relevant experience in data engineering, data integration, or related disciplines
• Strong, recent hands-on development experience with Databricks
• Proven experience designing, building, and implementing production ETL/ELT pipelines
• Advanced proficiency in Apache Spark, PySpark, and Python with demonstrated hands-on engineering experience
• Strong SQL skills, including complex joins, window functions, transformations, and performance tuning
• Experience with at least one major cloud platform: AWS, Azure, or GCP
• Experience with workflow orchestration tools such as Apache Airflow, Databricks Workflows, or equivalent platforms
• Experience with CI/CD, source control, and automated deployment of data pipelines
• Ability to independently own the full development lifecycle, including requirements evaluation, solution design, technical review, implementation, and production support
• Strong problem-solving skills with accountability, independence, and execution in fast-paced environments
• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field

Skills and tools

• Databricks, Apache Spark, Python, PySpark, SQL
• Airflow, Databricks Workflows
• CI/CD, AWS, Azure, Google Cloud Platform (GCP)
• Delta Lake, Unity Catalog, REST API

Benefits

• Comprehensive insurance options
• Matching contributions through the 401(k) plan and the share purchase plan
• Paid time off for vacation, holidays, and sick time
• Paid parental leave
• Learning opportunities and tuition assistance
• Wellness and Well being programs
• Competitive compensation

Nice to have

• Calling external APIs from Databricks notebooks, including REST API integration, authentication, request handling, and ingestion of API responses (candidates who consume/call APIs are eligible; building/exposing APIs is not required)
• Streaming and event-driven data processing
• Delta Lake and medallion architecture
• Unity Catalog, governance, and lineage
• Metadata-driven and reusable pipeline frameworks
• Enterprise scale Databricks implementations and complex enterprise data integration environments
• Additional strengths in data quality, reconciliation, monitoring/observability, production troubleshooting, technical documentation, design review participation, and Agile collaboration