Lead Data Engineer

Kaizen Dynamics is seeking a Lead Data Engineer to support the OCTO Enterprise Data team and the Cloud Data Exchange. This onsite contract role is focused on building and maintaining enterprise-grade data pipelines using native Azure capabilities for ETL, data loading, and transformation, with workflows aligned to enterprise governance expectations.

The position involves hands-on technical leadership across complex data engineering work, including designing solutions that can interoperate across cloud and on-prem environments, implementing reliable processing pipelines, and coordinating with stakeholders across agencies and technical teams.

What you will do

• Analyze, design, develop, implement, replicate, and support complex enterprise data projects.
• Interface with other agencies and consult with user departments on system requirements, environment constraints, and operating difficulties in the current state.
• Advise on and resolve issues through cloud-based solutions and help develop and replicate future enhancements to District data systems.
• Apply strong ETL knowledge using frameworks such as Azure Data Factory, Synapse, Databricks, and Informatica by gathering requirements from stakeholders or analyzing existing code, then delivering enhancements or new development.
• Establish cloud and on-prem connectivity across systems including ADLS, ADF, Synapse, and Databricks.
• Deliver hands-on work in Azure services such as Azure Data Factory or Azure Synapse, MSSQL DB, Azure SQL DB, Azure Data Lake Storage Gen2, Blob Storage, and Python.
• Create end-to-end pipelines that read from multiple source systems and load to a landing layer or SQL tables.
• Use data integration and pipeline tools including Informatica, Synapse, Apache NiFi, and Apache Airflow.
• Work with data formats including database formats (Oracle, SQL Server, DB2, Quickbase), text formats (CSV, XML), and binary formats (Parquet, AVRO).
• Develop, standardize, and optimize data workflow and pipelines while adhering to best practices.
• Support enterprise data governance standards by ensuring accuracy, consistency, security, and reliability.
• Automate, monitor, alert, and manage data pipelines and workflows.
• Analyze and evaluate system changes, including feasibility determination, alternative solutions, and backup and rollback procedures.
• Contribute to development of new systems, upgrades, and enhancements to existing systems while ensuring alignment with approved standards.
• Develop complex programs and reports using database query language.
• Support or design industry-specific applications that require integration with statewide systems or applications.
• Convey technical and functional concepts for a specific technical specialty, including working with executive-level business users or technical experts.
• Identify improvements to project standards aimed at higher quality services and products.
• Coordinate with application development teams when programming activities need alignment, and work with business and technical personnel to resolve intent, inaccuracy, or feasibility concerns.
• Collaborate with interested personnel to determine whether modifications or enhancements are necessary.
• Leverage written and verbal communication skills to develop new business process and programming solutions as directed by business and technical stakeholders.

Required qualifications

• Bachelor’s degree in Information Technology or a related field, or equivalent experience.
• SQL: 10 years (required).
• Extract-Transform-Load (ETL) processes: 10 years (required).

Technologies you will work with

• Azure Data Factory, Azure Synapse, Databricks
• Informatica
• ADLS, MSSQL DB, Azure SQL DB
• Azure Data Lake Storage Gen2, Blob Storage
• Python
• Apache NiFi, Apache Airflow
• Oracle, SQL Server, DB2, Quickbase
• CSV, XML, Parquet, AVRO

Location: Washington, DC (onsite). Compensation: USD 70 to 80 per hour. Experience: 10+ years. Job type: Contract.