Data Engineer / Lead Data Engineer
Manager
Analytics
Azure
Azure Data Engineer
Azure Data Factory
Azure Data Platform
Big Data
Business Intelligence
Cloud
Cloud Data Engineering
Cloud Data Platform
Cloud Engineer
Cloud Platform
Cloud Platforms
Cloud Platforms Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Factory
Data Factory Azure
Data Integration
Data Integration Tools
Data Pipeline
Data Platform
Data Processing
Data Visualization
Data Warehouse
Database
Databases
Digital Marketing
ETL
Informatica
Microsoft Azure
Reporting and Analytics
SQL
Job Description
MSR Cosmos is looking for a Data Engineer / Lead Data Engineer to help shape data foundations and deliver end-to-end analytics in a complex energy trading setting. This hybrid role is based in Houston, with a focus on building scalable ingestion, transformation, modeling, reporting, and data quality controls on Microsoft Azure.
What you’ll do
- Lead the design and development of scalable data ingestion and transformation pipelines using standard data lake and cloud engineering frameworks.
- Build solutions that handle structured and unstructured data in high-volume, high-velocity, high-variety environments.
- Create transformation notebooks, physical and tabular data models, data endpoints, semantic layers, and visualizations.
- Develop reusable data products that increase accessibility, consistency, and business value.
- Lead ETL/ELT work across source systems, data platforms, and downstream business intelligence solutions.
- Publish dashboards and reports to deliver stakeholders accessible, actionable information.
- Conduct trend and correlation analysis, detect data discrepancies, and implement controls to protect data quality across platforms.
- Define and enforce engineering standards, reusable patterns, documentation practices, and development approaches.
- Build automated continuous integration and release pipelines, applying disciplined source-control practices.
- Collect functional and non-functional requirements, translate features and Product Backlog Items into technical designs, and set delivery direction.
- Collaborate with cross-functional teams in a dynamic business environment and mentor engineers when needed.
Requirements
- Demonstrable data engineering experience within energy trading, commodities trading or an ETRM/CTRM environment. This requirement is mandatory.
- Strong hands-on experience designing production-grade ingestion, storage, transformation, modeling, and serving layers on Microsoft Azure.
- Advanced SQL skills, including developing and optimizing complex queries for transformation and analysis.
- Hands-on experience with Azure Data Factory, Azure SQL, Azure Databricks, Azure Blob Storage, and Azure Data Lake Storage.
- Strong ETL/ELT and data warehousing experience, including delivery of business intelligence solutions.
- Experience with dimensional, physical, and tabular data modeling, including performance optimization.
- Experience with GitHub and/or Azure DevOps, including automated build, test, and release pipelines.
- Ability to gather requirements, shape technical solutions, and communicate clearly with both technical and business stakeholders.
- Proven ability to lead workstreams, manage priorities, and deliver effectively in a collaborative team environment.
Technologies
- SQL
- Microsoft Azure
- Azure Data Factory
- Azure SQL
- Azure Databricks
- Azure Blob Storage
- Azure Data Lake Storage
- GitHub
- Azure DevOps
Preferred experience
- Experience with dbt, Azure Analysis Services, Azure Table Storage, and Azure Logic Apps.
- Experience with Power BI and building decision-oriented dashboards and reports.
- Exposure to Power Apps and broader Microsoft Power Platform capabilities.
- Familiarity with IBM DataStage or comparable enterprise integration tools.
- Basic understanding of AI engineering, including how governed data products support analytics and AI use cases.
- Knowledge of data governance, lineage, metadata management, observability, and automated data-quality controls.
Location: Houston, TX (hybrid)
Compensation: USD 65 - 70 per hourly