Senior Data Engineer
Job Description
CoStar Group is hiring a Senior Data Engineer to design, build, and operate reliable, scalable data pipelines and data products on a modern cloud data platform. This is a hands-on position centered on pipeline development, performance optimization, production reliability, and ongoing improvement.
You will translate business needs into maintainable technical designs, implement production-grade data quality and monitoring, and collaborate across Product Management, Application Engineering, and stakeholders to keep data solutions observable, governed, and well supported.
What you’ll do
- Design, build, test, and maintain reliable, scalable data pipelines and data products.
- Develop and optimize data transformations and orchestrations using SQL, Python, dbt, Airflow, Glue, PostgreSQL, and Snowflake.
- Convert business requirements into production-ready technical designs and solutions.
- Implement automated testing, data-quality checks, monitoring, and failure handling with recovery processes.
- Optimize large-scale data workloads for performance, scalability, and cost.
- Troubleshoot production issues and perform root-cause analysis across pipelines and databases.
- Use Git-based development with code review, CI/CD, and controlled release practices.
- Ensure data products are documented, observable, governed, and supported with appropriate metadata and lineage.
- Reduce technical debt and recurring operational issues through automation and improved design.
- Collaborate with Product Management, Application Engineering, and business stakeholders.
Minimum qualifications
- 6+ years of professional experience in data engineering or a closely related discipline.
- Strong Python skills for data pipelines, automation, integrations, and supporting data applications.
- Advanced SQL skills with experience working on large-scale datasets.
- Strong understanding of data modeling, database design, and data transformation patterns.
- Experience with dbt, including model development, dependency management, testing, incremental processing, and production deployment.
- Experience with Airflow, including DAG development, orchestration, dependencies, retries, failure handling, scheduling, and production operations.
- Hands-on PostgreSQL experience, including complex SQL development, indexing, query optimization, and performance tuning.
- Hands-on Snowflake experience in production data workloads context.
- Experience with AWS data services including S3, RDS/Aurora, Glue, MWAA, IAM, and CloudWatch.
- Experience implementing automated data validation and quality controls.
- Experience with Git, pull requests, code review, conflict resolution, and CI/CD workflows.
- Demonstrated ability to troubleshoot complex production pipeline and data issues.
- Understanding of monitoring, failure recovery, and production support practices.
Technologies
- SQL, Python, dbt, Airflow, Glue, PostgreSQL, Snowflake
- AWS S3, RDS/Aurora, MWAA, IAM, CloudWatch
- Git, CI/CD
Compensation
- Base compensation: $128,000 to $190,000 per year
Benefits
- Comprehensive healthcare coverage: Medical, Vision, Dental, and Prescription Drug
- Life, legal, and supplementary insurance
- Virtual and in-person mental health counseling services for individuals and family
- Commuter and parking benefits
- 401(K) retirement plan with matching contributions
- Employee stock purchase plan
- Paid time off
- Tuition reimbursement
- On-site fitness center and/or reimbursed fitness center membership costs (location dependent)
- Access to CoStar Group’s Employee Resource Groups
- Complimentary gourmet coffee, tea, hot chocolate, fresh fruit, and other healthy snacks
Additional information
- Location: Irvine, CA (onsite)
- Visa sponsorship: CoStar Group is not able to provide visa sponsorship for this position.
- Equal Employment Opportunity: CoStar Group is an Equal Employment Opportunity Employer; maintains a drug-free workplace and performs pre-employment substance abuse testing.