Data Engineer
Azure
Azure Data Factory
Azure Data Lake
Azure Functions
Azure Synapse Analytics
Data Architecture
Data Engineer
Data Engineering
Data Factory
Data Factory Azure
Data Integration
Data Lakehouse
Data Pipeline
Data Pipelines
Data Platform
Data Processing
Databricks
ETL
Microsoft Azure
Pyspark
Spark
SQL
Job Description
Freshfields is hiring a Data Engineer (3–5 years experience) to build and operate modern cloud data pipelines and data products on Microsoft Azure.
Responsibilities
- Design, develop, and maintain batch and near real-time ETL/ELT pipelines using Azure Data Factory, Databricks, PySpark, and SQL.
- Ingest, transform, and deliver data from multiple enterprise source systems.
- Create reusable, scalable data processing components aligned with engineering standards.
- Build and maintain data lake and lakehouse solutions using Azure Data Lake, Databricks, and Delta Lake.
- Support data models that enable analytics, reporting, and self-service data consumption.
- Help develop reusable data products and datasets.
- Implement data quality controls, validation frameworks, and monitoring processes within pipelines.
- Support metadata management, data cataloguing, and lineage activities using tools such as Collibra and Purview.
- Improve data quality, consistency, and trust across enterprise datasets.
- Monitor pipeline and processing workloads to ensure reliability and operational stability.
- Investigate and resolve data and platform issues in a timely manner.
- Identify opportunities to improve performance, scalability, and cloud cost efficiency.
- Collaborate with Data Architects, Lead Data Engineers, Data Analysts, and business stakeholders to deliver solutions aligned to business requirements.
- Participate in agile delivery ceremonies including sprint planning, stand-ups, and retrospectives.
- Support testing, deployment, and release activities for data products and platforms.
- Contribute to source control, automated deployment, and testing processes using Git, Azure DevOps, and related tooling.
- Follow established engineering standards, coding practices, and documentation requirements.
- Participate in peer reviews and continuous improvement initiatives.
- Maintain technical documentation, solution designs, and operational procedures.
- Support knowledge sharing across the Data Office through collaboration and learning initiatives.
- Contribute to the development of best practices and engineering standards.
Requirements
- 3–5 years of experience in data engineering or a related technical discipline.
- Experience developing ETL/ELT data pipelines in a cloud environment, preferably Microsoft Azure.
- Hands-on experience with Databricks, SQL, and Python/PySpark.
- Experience with enterprise data platforms, data warehouses, data lakes, or lakehouse architectures.
- Familiarity with data governance, metadata management, and data quality concepts.
- Experience using Git and contributing to CI/CD delivery processes.
- Strong stakeholder engagement and collaboration skills.
- Bachelor’s degree in computer science, Engineering, Information Systems, or a related field, or equivalent practical experience.
- Experience with Azure Data Lake Storage Gen2.
- Experience with Azure Data Factory.
- Experience with Azure Synapse Analytics.
- Azure Functions (desirable).
- Experience developing solutions in Databricks.
- Strong SQL skills.
- Good working knowledge of Python and PySpark.
- Understanding of Delta Lake concepts and modern lakehouse architectures.
- Familiarity with dbt (data build tool).
- Experience using Git for source control.
- Exposure to CI/CD practices using Azure DevOps or GitHub.
- Basic understanding of orchestration tools such as Apache Airflow.
- Awareness of data governance principles, metadata management, and data lineage.
- Experience with Collibra, Purview, or similar data governance platforms is advantageous.
- Understanding of data quality frameworks and controls.
- Strong analytical and problem-solving skills.
- Effective verbal and written communication skills.
- Ability to work collaboratively across technical and business teams.
- Comfortable working within Agile delivery environments.
- Curious mindset with a willingness to learn new technologies and capabilities.
- Strong attention to detail and commitment to delivering high-quality solutions.
Technologies
- Microsoft Azure
- Azure Data Factory
- Databricks
- Delta Lake
- PySpark
- SQL
- Python
- Azure Data Lake
- Azure Data Lake Storage Gen2
- Azure Synapse Analytics
- Azure Functions
- Collibra
- Purview
- Git
- Azure DevOps
- GitHub
- dbt (data build tool)
- Apache Airflow
- ETL, ELT
Role overview
- Design, build, and maintain scalable cloud-based data pipelines and data products on Microsoft Azure using Databricks and Delta Lake.
- Support data governance and data quality initiatives to help deliver trusted data for reporting, analytics, AI, and machine learning.
- Work within the Global Data Office alongside the Data Engineering Lead, Data Architects, Analysts, Data Governance teams, and business stakeholders.
- Operate in a fast-paced, evolving environment with collaboration across geographically dispersed teams and multiple office locations.
Location
- Raleigh, NC (onsite)
Experience and education
- Minimum experience: 3 years (Data Engineer or related technical discipline; listed range: 3–5 years)
- Bachelor’s degree in computer science, Engineering, Information Systems, or a related field (or equivalent practical experience)