Senior Data Engineer

Senior Data Engineer based in Newark, NJ, with a hybrid on-site requirement (3 days per week) at PGIM, designing, building, and maintaining scalable data and AI solutions on modern cloud platforms.

Responsibilities

• Design and maintain scalable ETL and ELT data pipelines
• Develop data solutions using Microsoft Fabric, Azure Data Factory, or equivalent tools
• Implement data quality checks, transformations, and data modeling
• Work with large-scale datasets using PySpark and SQL
• Deploy and integrate machine learning models into data pipelines
• Develop AI powered features such as semantic search, embeddings, and retrieval augmented generation (RAG)
• Build and expose data and AI services via APIs
• Contribute to CI/CD pipelines using GitHub Actions or Azure DevOps
• Apply best practices for data security, governance, and access control
• Monitor and troubleshoot data pipelines and AI systems
• Collaborate with analysts, scientists, and stakeholders
• Contribute to documentation and reusable components

Requirements

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field
• 3–6 years of experience in data or AI engineering
• Strong Python and SQL skills
• Experience with data pipeline tools such as Azure Data Factory, Microsoft Fabric, AWS Glue, or equivalent
• Experience with Spark or PySpark
• Familiarity with data lakes and data warehouses
• Experience deploying machine learning models
• Basic understanding of MLOps
• Exposure to Generative AI concepts (embeddings, RAG, LLM APIs)
• Experience with REST APIs and microservices
• Familiarity with CI/CD and Git
• Understanding of data governance and security
• Strong problem-solving and communication skills

Technologies

• Microsoft Fabric
• Azure Data Factory
• AWS Glue
• PySpark
• Spark
• Python
• SQL
• REST APIs
• GitHub Actions
• Azure DevOps
• LLM APIs

Benefits

• Market-competitive base salary with annual bonus potential
• Medical, dental, vision, life and disability insurance; PTO and parental/military leave
• 401(k) with company match up to 4%
• Company-funded pension plan
• Wellness programs with up to $1,600 per year reimbursement for wellbeing needs
• Work/Life resources
• Education benefit for degree or certificate programs
• Employee Stock Purchase Plan (ESPP) with 85% purchase price after one year

Compensation note

Salary range: The annual base salary for this role is $115,000 to $155,000. Specific pricing may vary within this range based on geographic location, candidate experience, and skills, and is reported in compliance with applicable state laws.