Build and modernize forecasting data at scale. National Grid is seeking a Senior Data Engineer to help evolve a forecasting platform with scalable, end-to-end data pipelines that bring together large datasets from multiple sources. This role sits within the Information Technology and Digital organization and operates under a hybrid working model with options based in Waltham, MA, Brooklyn, NY, or Hicksville, NY.
In this position, you’ll focus on data quality, governance, and modernising data engineering capabilities, ensuring teams can rely on secure, consistent, and accessible data for analytics, reporting, and forecasting.
What you’ll do
- Design, build, and support scalable data pipelines that are robust, well-tested, and able to meet growing business needs.
- Build and evolve data integrations and APIs to handle increasing volumes and complexity.
- Champion data quality and governance so data remains secure, consistent, reliable, and accessible across the organisation.
- Develop scalable data solutions aligned with architectural roadmaps and standards across integration, storage, reporting, and analytics.
- Partner with analytics and business teams to improve data models, increase accessibility, and enable better data-driven decision making.
- Design and enhance data integration and quality frameworks, backed by strong unit, integration, and functional testing.
- Build and automate distributed data solutions that collect and process streaming events from multiple sources.
- Investigate and resolve complex data issues using data analysis to identify root causes and deliver fixes.
- Guide and mentor junior engineers through best practices in coding, engineering standards, and optimisation.
What you bring
- Strong data engineering experience (data engineering, analytics, or business intelligence), ideally around 5 years relevant experience.
- Strong programming skills, especially Python, with hands-on experience in PySpark and SQL.
- Experience working with Databricks and building scalable data solutions.
- Experience designing and developing integrations across multiple systems and APIs.
- Hands-on experience with Azure data technologies including Azure Data Lake, Azure Data Factory, Azure DevOps, and Azure Functions.
- Strong SQL expertise, including optimising queries across large-scale, complex datasets.
- Experience with data warehousing and developing ETL and/or ELT pipelines.
- Understanding of Agile software development methodologies and modern engineering practices.
- Excellent problem-solving and troubleshooting with a structured, process-driven approach.
- Strong documentation skills and a focus on maintainable, well-engineered solutions.
- Degree or equivalent experience in Computer Science, Mathematics, Statistics, or another technical discipline.
Key technologies
Python, PySpark, SQL, Databricks, Azure Data Lake, Azure Data Factory, Azure DevOps, Azure Functions, ETL, ELT
Additional “nice to have”
- Experience designing data schemas and working with SQL and NoSQL databases.
- Knowledge of Kafka, Flink, Fivetran, or Matillion.
- Exposure to Data Science and Machine Learning.
- A background in software engineering.
- Experience working with Snowflake.
Location and salary
- Hicksville, NY (hybrid)
- Salary: USD 129,000 - 180,000 per year
Notices
- National Grid will not provide visa sponsorship for this position now or in the future. You must be able to work without current or future visa sponsorship.
- National Grid uses an AI or statistically based assessment to evaluate job qualifications/characteristics.
- There is a career path with advancement opportunities within and across bands as you develop and evolve in the role.
- National Grid is committed to equal employment opportunity and does not discriminate based on protected class.