Associate - Data Engineer

The Associate Data Engineer role supports New York Life’s Data Engineering team by building clean, well-structured, and dependable enterprise data foundations used for analytics and AI initiatives. The position emphasizes scalable pipeline development, data standards, and close collaboration with data scientists, technology teams, and business stakeholders.

Key Responsibilities

• Design, build, and maintain scalable data models and pipelines using dbt, PySpark, AWS Glue, Databricks, and streaming technologies to enable clean and reliable data for analytics, data science, and AI use cases.
• Apply software engineering best practices across data development, including version control, testing, integration, code reviews, and pull request approvals to help ensure quality and consistency.
• Partner with data scientists, business stakeholders, and data stewards to source strategic enterprise data, address data quality and availability needs, and align delivery with data governance standards.
• Convert business and technical requirements into actionable development plans and Jira stories, define milestones, and manage delivery through completion.
• Collaborate with Technology teams on data strategy, tooling, system integrations, and deployment processes, while communicating technical concepts and recommendations to varied audiences.

Required Qualifications

• 2–5+ years of relevant data engineering or related experience, including hands-on work building and maintaining dbt data models and transformation pipelines.
• Proficiency with AWS Glue, including ETL development, job orchestration, and catalog integration, plus experience using PySpark for data processing.
• Strong working knowledge of Databricks, including SQL, notebooks, and distributed data processing.
• Experience with Kafka or comparable streaming technologies for real-time ingestion and processing.
• Strong analytical and problem-solving skills, including the ability to troubleshoot complex data and engineering issues and develop practical solutions.
• Effective communication and collaboration skills across technical and business teams, including explaining technical concepts to diverse audiences.

Technologies

• dbt
• PySpark
• AWS Glue
• Databricks
• Kafka (streaming technologies)
• SQL
• Notebooks
• Jira

Preferred Skills

• Experience with Google Cloud Platform (GCP) or related cloud data technologies.
• Experience working in the insurance, financial services, or another regulated industry.
• Experience partnering with data science teams to support analytics, machine learning, or AI use cases.

Compensation and Incentives

• Salary Range: USD 81,000 to 115,500 per year
• Discretionary Bonus: Annual discretionary bonus
• Incentive Program Eligibility: In addition to base salary, eligibility to participate in an incentive program
• Overtime Status: Overtime eligible: Exempt
• Pay Transparency: Salary Range $81,000-$115,500; Sales bonus eligible: No

Benefits

• Leave programs
• Adoption assistance
• Student loan repayment programs
• Annual discretionary bonus

Work Location and Role Details

• Location: Lebanon, NJ (Hybrid)
• Hybrid Expectation: 3 days per week
• Department: Tech Data AI Ventures
• Job Function: Tech Data AI Ventures
• Job Level: LEVELPF2
• Job Requisition ID: 95121

Associate - Data Engineer: Basic Requirements Summary

Minimum experience of 2 years with a focus on building and maintaining dbt transformation models and data pipelines, along with hands-on experience across AWS Glue, PySpark, Databricks, and streaming technologies.