Data Engineer II

The Data Engineer II role at Lewis Energy Group supports the design, build, and ongoing maintenance of the organization’s data platform on Microsoft Fabric. This position focuses on delivery of reliable pipelines, well-structured data models, and Power BI semantic models that support reporting and business decision-making.

Responsibilities

• Design, build, and maintain data pipelines in Microsoft Fabric using Data Factory pipelines, Dataflows Gen2, and notebooks to move data from source systems into Lakehouses and Warehouses.
• Develop data transformation and processing logic in Python and PySpark within Fabric notebooks, including work with Delta tables and efficient handling of large or complex datasets.
• Develop and optimize T-SQL code, including stored procedures, views, and transformation logic to support data processing and reporting needs.
• Apply data modeling best practices such as medallion architecture and star schema design to organize data in OneLake for performance and usability.
• Monitor pipeline executions, troubleshoot failures, and implement alerting and error handling to ensure timely and accurate data delivery.
• Evaluate existing on-premises SQL Server databases, SSIS packages, and scheduled jobs to plan migration to Microsoft Fabric.
• Rebuild or refactor legacy ETL processes as Fabric-native pipelines and PySpark notebooks to improve performance and maintainability.
• Coordinate with team members and stakeholders on migration timelines to minimize disruption to reporting and operations.
• Build and maintain semantic models used as the foundation for Power BI reports, including relationships, measures, ontology, and Direct Lake or Import configurations.
• Partner with report developers and business users to translate reporting requirements into well-structured datasets.
• Tune data models and queries to improve report performance and refresh reliability.
• Communicate technical concepts, data limitations, and project status clearly to non-technical stakeholders.
• Implement data quality checks and validation rules to prevent issues from reaching end users.
• Manage workspace organization, access permissions, and security settings in Fabric according to company data governance standards.
• Use source control with Azure DevOps and Git, along with deployment pipelines, to manage changes across development, test, and production environments.
• Create and maintain documentation covering data sources, pipelines, models, dictionaries, and business logic.

Requirements

• Demonstrated ability to design relational and dimensional data models, including fact and dimension tables, keys, relationships, referential integrity, and historical changes.
• Strong T-SQL skills for complex joins, transformations, query tuning, and troubleshooting.
• Hands-on experience building and supporting integrations across databases, APIs, or file-based sources, including scheduling, incremental processing, and recovery.
• Ability to validate data completeness and accuracy, reconcile source and target results, and investigate duplicates and broken relationships.
• Experience with Python or comparable scripting for data processing or automation, plus version control for maintainable development.
• Ability to translate business requirements into documented models and clearly explain technical decisions; willingness to learn unfamiliar platforms.

Technologies

• Microsoft Fabric, Data Factory, Dataflows Gen2
• PySpark, Python, T-SQL
• SQL Server, SSIS, SSAS
• OneLake, Lakehouses, Warehouses, Delta
• Power BI, Power Query, DAX
• Azure Data Lake Storage, Azure SQL, Synapse Analytics, Databricks
• Azure DevOps, Azure Git

Preferred Experience

• Power BI semantic models, Power Query, DAX, or comparable business intelligence tools.
• Microsoft SQL Server, SSIS, SSAS, or comparable database and integration technologies.
• Microsoft Fabric, Azure Data Factory, Azure Data Lake Storage, Azure SQL, Synapse Analytics, Databricks, or Azure DevOps.
• Oil and gas production data, SCADA, telemetry, time-series data, IoT or Streaming data.

Location and Work Schedule

• San Antonio, TX (onsite)
• Candidates must be able to work full-time from the San Antonio office.

Minimum Qualifications

• Minimum experience: 2 years
• Education: Bachelor’s degree from a four-year college or university

Benefits

• Insurance: Medical/Dental/Vision, Basic Life, AD&D, and other benefit options to meet personal needs
• 401K with immediate matching contribution
• Annual Discretionary Bonus
• Generous Paid Time Off and Paid Holidays
• Employee Assistance Program (EAP) with free, confidential support for employees and dependents
• Wellness and company events throughout the year