Senior, Data Engineer

Epsilon is seeking a Senior Data Engineer (Employer: Epsilon Data Management LLC) to build and optimize AWS and Greenplum data pipelines that support internal data products. The role focuses on end-to-end delivery, from SQL and Python development through testing, monitoring, and data governance, with an emphasis on secure, scalable processing at scale.

This position is based in Chicago, IL (hybrid), with required in-office attendance certain days per week and telecommuting for the rest of the week.

What you’ll do

• Develop and execute software tests to surface software problems and identify root causes.
• Write and optimize complex SQL scripts and build Python scripts to run large-scale transformations on Greenplum.
• Create test plans, run unit and integration tests, and deliver reproducible, well-documented results.
• Design and maintain AI-enabled PySpark pipelines on AWS, primarily using EMR, Glue, Athena, DynamoDB, and S3 to process large datasets.
• Harden and tune jobs for performance, cost, and compliance, including managing data partitioning, cataloging, and lineage.
• Automate deployments and infrastructure using CloudFormation and parameterized configurations, and schedule batch workloads for timely, reliable pipelines.
• Build internal dashboards and data products (including Amplify apps) and implement monitoring and alerting to detect issues early.
• Handle client tickets by translating ambiguous requests into clear technical tasks.

What you bring

• A Master’s degree in Computer Science, Information Technology, Engineering, or a related field.
• 2+ years of experience in the job offered, or 2 years in any occupation where the required experience was gained (employer will accept the specified combination of education and experience).
• Experience building PySpark ETL on AWS using EMR/Glue with S3, including partitioning, predicate pushdown, and broadcast joins to produce analytics-ready fact/dimension tables on schedule.
• Ability to author and tune complex SQL on Greenplum/Postgres, including window functions and ROW_NUMBER dedupe, plus physical design and maintenance approaches such as distribution keys, AO/AOCO compression, and VACUUM/ANALYZE.
• Experience managing AWS Glue Data Catalog and curated layers with Athena, designing S3 layouts (prefix/partition strategy and lifecycle policies) to reduce query time and storage cost.
• Hands-on CI/CD using CodeCommit, CodeBuild, and CodePipeline, plus parameterized environments and CloudFormation-based infrastructure as code for repeatable, auditable rollbacks.
• Observability engineering with CloudWatch logs/metrics/alarms, job health checks, SLA dashboards, and alerting to detect failures early and reduce MTTR.
• Data quality enforcement through SQL/Python unit tests, reconciliation (row counts, sums, duplicates), referential integrity checks, and documented UAT evidence for change control.
• Security and governance experience, including normalizing and encrypting sensitive identifiers, using KMS-managed keys, and applying IAM least privilege with audit logging.
• Experience delivering GenAI-assisted features by integrating LLMs (using AWS Bedrock) with PySpark/SQL to suggest transformations, generate test cases, and summarize validation results.
• Experience building lightweight internal data apps using AWS Amplify with DynamoDB/Athena to manage lookup keys/brand lists and enable stakeholder self-service with controlled access.

Tools and technologies

• SQL, Python, Greenplum, PySpark
• AWS EMR, AWS Glue, AWS Athena, Amazon DynamoDB, Amazon S3
• CloudFormation, AWS CodeCommit, AWS CodeBuild, AWS CodePipeline
• Amazon CloudWatch, AWS KMS, AWS IAM
• AWS Bedrock, AWS Amplify, Postgres

Compensation

USD $145,870 - $182,000 per year.

Benefits

• Flexible time off (FTO), 14 paid holidays, paid sick time
• Parental/new child leave, childcare & elder care assistance, adoption assistance
• Comprehensive health coverage
• 401(k)
• Tuition assistance
• Commuter benefits
• Professional development
• Employee recognition
• Charitable donation matching
• Health coaching and counseling

Location

• 35 West Wacker Drive, Chicago, IL 60601

Required to work from the company office in Chicago, IL certain days per week. Telecommuting is allowed for the balance of the week.

About Epsilon

Epsilon is a global leader in data and identity, with deep expertise in digital media, clean rooms, customer data platforms, loyalty and marketing services. Epsilon operates in 30+ countries across EMEA, the Americas, and APAC, with more than 8,000 employees.

Epsilon’s values include Act with integrity, Work together to win together, Innovate with purpose, Respect all voices, and Empower with accountability.

Equal opportunity and accommodations

• Epsilon is an equal opportunity employer and does not discriminate based on protected categories.
• Accommodations are available for applicants who need help completing the application process. Contact [email protected] to request an accommodation.
• In the San Francisco Bay and Los Angeles areas, Epsilon will consider qualified applicants with criminal histories in a manner consistent with applicable fair chance ordinances.