Data Engineer, Microsoft Fabric and AI Director

GS1 Global Office is hiring a senior, hands-on Data Engineer Director (individual contributor) to build secure, scalable enterprise analytics and AI-enabled data pipelines.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Microsoft Fabric and Azure data services.
• Build and support Lakehouse, Data Warehouse, and semantic model solutions using reusable design patterns.
• Create reliable data integration processes using Microsoft Fabric Data Pipelines, Azure Data Factory, APIs, and other approved integration methods.
• Support migration and modernisation initiatives involving Microsoft Fabric and Azure analytics services.
• Optimise data processing for performance, scalability, maintainability, and cost efficiency.
• Influence enterprise data architecture decisions and help evolve shared data models and analytics standards.
• Design and build retrieval-augmented generation (RAG) pipelines that securely connect approved enterprise data to Claude AI and other approved large language model platforms.
• Develop AI-enabled search and analytics building blocks, including data preparation, chunking, embeddings, indexing, and retrieval.
• Support responsible adoption of approved enterprise AI and development tools, including Claude AI and Claude Code, with secure data-handling and access patterns.
• Evaluate and prototype AI-enabled analytics use cases with senior leaders and technical stakeholders, including assessment of business value, architecture, risk, and guardrails.
• Monitor and improve AI pipeline reliability, quality, performance, and cost in line with GS1 governance standards.
• Maintain documentation, traceability, and human oversight for AI-enabled solutions.
• Develop and maintain Power BI dashboards, reports, datasets, and semantic models to deliver actionable insights.
• Support operational reporting, data-quality reporting, and prioritized ad hoc analytics requests.
• Work with business stakeholders to understand requirements, define acceptance criteria, and translate needs into sustainable reporting solutions.
• Promote consistent definitions, measures, and reporting practices across the organisation.
• Implement data validation, observability, monitoring, and quality controls across data pipelines and analytics solutions.
• Support metadata management, data lineage, documentation, retention, and governance requirements.
• Design solutions aligned with GS1 information security, data privacy, access-control, and responsible AI requirements.
• Implement role-based access controls, data classification, and auditability for data sensitivity and intended use.
• Identify and escalate concerns related to data quality, security, privacy, model risk, and governance.
• Develop automated testing and validation for data pipelines, semantic models, reports, and AI-enabled solutions.
• Use Git, source control, CI/CD, and environment-management practices for reliable deployments across development, test, and production.
• Monitor critical solutions, resolve incidents, and troubleshoot failures including refresh errors, reporting issues, RAG errors, and performance bottlenecks.
• Perform root-cause analysis and implement preventative improvements.
• Maintain technical documentation, operational runbooks, and recovery procedures; support release validation and business-continuity activities.
• Provide technical leadership on data architecture, solution design, engineering standards, and responsible AI implementation.
• Review code and solution designs, share knowledge, and coach team members in data engineering and analytics practices.
• Partner with Product Owners, Data Engineers, QA, Software Engineering, and business stakeholders across a globally distributed organisation.
• Communicate technical options, dependencies, risks, and costs to technical and non-technical audiences.
• Assess trade-offs and make recommendations balancing business value, usability, security, scalability, cost, and maintainability.
• Contribute to planning, architecture discussions, continuous improvement, and workload priorities across the BIDA team.

Requirements

• Bachelor’s degree in computer science, data engineering, information systems, or a related field (or equivalent relevant professional experience).
• At least 5 years of relevant experience in data engineering, business intelligence, or analytics, including responsibility for production solutions.
• Strong practical experience with Microsoft Fabric, or significant experience with Azure Synapse, Databricks, or comparable modern cloud analytics platforms.
• Strong experience developing Power BI reports, semantic models, and datasets.
• Advanced SQL skills and experience with Azure SQL or comparable relational database services.
• Practical experience building ETL or ELT solutions using Azure Data Factory, Microsoft Fabric Data Pipelines, or comparable orchestration tools.
• Experience with dimensional modelling, data warehousing, and enterprise semantic models, including DAX.
• Proficiency with Python and/or PySpark for data processing and automation.
• Practical experience building/supporting RAG pipelines and working with large language model APIs such as Claude, OpenAI, or comparable platforms.
• Experience with embeddings, vector search, or vector databases, plus prompt-based retrieval patterns and structured/unstructured data integration.
• Experience with source control, automated testing, CI/CD, and production monitoring.
• Experience implementing data security, role-based access controls, and privacy requirements in cloud data and analytics environments.

Technologies

• Microsoft Fabric
• Azure data services
• Power BI
• Azure Data Factory
• Microsoft Fabric Data Pipelines
• APIs
• Lakehouse
• Data Warehouse
• Semantic models
• Azure Synapse
• Databricks
• DAX
• SQL
• Azure SQL
• Git
• CI/CD
• Python
• PySpark
• Retrieval-augmented generation (RAG)
• Claude AI
• Claude Code
• OpenAI
• Embeddings
• Vector search
• Vector databases
• Prompt-based retrieval patterns
• Role-based access controls

Preferred Experience

• Microsoft Fabric or related Microsoft data-platform certification.
• Experience with Microsoft Purview, OneLake, Azure DevOps, or GitHub.
• Experience with Power BI administration, tenant governance, or capacity management.
• Experience with large-scale analytical datasets and cloud cost optimisation.
• Experience with Claude AI, Claude Code, or comparable enterprise AI coding and knowledge-work tools.
• Knowledge of AI governance, responsible AI, model evaluation, and enterprise data-privacy practices.
• Experience working in a global, federated, or matrixed organisation.

Travel Requirements

• Occasional international travel and regular collaboration across European and United States time zones.

Location: Ewing, NJ (onsite) • Salary: USD 140,000 - 160,000 per year • Experience: 5+ years