Data Engineer 3 (Mid-Level)
Job Description
UNCOMN is seeking a mid-level Data Engineer 3 to support the IGC team with enterprise ETL and data ingestion across integrated transportation data systems.
Responsibilities
- Lead design, development, testing, deployment, and maintenance of enterprise ETL and data-integration solutions using Informatica and approved technologies.
- Build and optimize Informatica mappings, transformations, workflows, sessions, parameter files, reusable components, scheduling configurations, and restart/recovery logic.
- Modernize legacy ETL by refactoring mappings, SQL, transformation logic, interfaces, and operational processes for approved database and cloud environments.
- Develop and optimize complex SQL (queries, views, stored procedures, functions, indexes) and transformation logic for high-volume data-processing workloads.
- Conduct design and code reviews; establish reusable ETL patterns, development practices, and technical approaches for complex integration work.
- Design ingestion patterns for relational databases, APIs, web services, message-based interfaces, structured and unstructured files, and other approved enterprise sources.
- Implement batch, micro-batch, incremental, event-driven, and change-data-capture processing approaches aligned to mission, interface, and performance requirements.
- Analyze source structures, dependencies, formats, and business rules; produce source-to-target mappings, transformation specifications, data dictionaries, lineage documentation, and interface-control artifacts.
- Create analytics-ready datasets, views, and data services supporting downstream applications, reporting, and approved BI platforms.
- Update ETL mappings and transformation logic to support migration to approved cloud-managed database platforms, including Amazon Aurora where implemented.
- Coordinate interface changes with application, database, BI, testing, cybersecurity, and production-support teams to protect end-to-end data-flow accuracy and availability.
- Implement data profiling, validation, cleansing, standardization, reconciliation, deduplication, exception handling, and automated controls for completeness, accuracy, timeliness, integrity, and consistency.
- Develop validation queries, reconciliation reports, automated tests, and technical evidence to confirm transformation and end-to-end processing behavior.
- Investigate and resolve failed workflows, rejected records, data discrepancies, interface defects, bottlenecks, and recurring data-quality issues.
- Tune Informatica workflows, SQL, database access, transformation logic, and scheduling to improve throughput, scalability, processing time, and resource efficiency.
- Implement pipeline logging, metrics, alerting, audit trails, restart controls, and recovery procedures for rapid diagnosis and restoration.
- Design and optimize ingestion and transformation to meet IGC data-availability and latency objectives while minimizing backlogs and repeated failures.
- Maintain ETL code and supporting artifacts in approved Git repositories using established branching, review, and configuration-management practices.
- Develop ETL build, validation, test, packaging, and deployment automation within GitLab CI/CD; coordinate platform and Runner requirements with DevSecOps personnel.
- Develop automation scripts using Python, Bash, PowerShell, SQL, or comparable approved technologies.
- Apply secure data-handling practices (least privilege, encryption, secrets protection, secure transfer, logging, auditability) and support remediation of security findings impacting ETL and ingestion components.
- Provide technical evidence and remediation support for RMF, STIG, secure-configuration, testing, and release activities for assigned components.
- Prepare and maintain technical designs, data-flow diagrams, source-to-target mappings, interface specs, data dictionaries, lineage documentation, deployment procedures, troubleshooting guides, and operational runbooks.
- Lead technical design discussions for complex ingestion, transformation, and interface requirements; identify technical risks, dependencies, and mitigations.
- Mentor ETL developers and data engineers on Informatica development, SQL, data quality, performance tuning, testing, and maintainable integration design.
- Collaborate with software developers, database engineers, BI developers, cloud/DevSecOps, cybersecurity, testers, functional analysts, Product Owners, operations, and Government stakeholders.
- Participate in Agile planning, technical reviews, sprint demos, release activities, defect resolution, and production-support as assigned.
Requirements
- 8+ years of professional experience in ETL development, data engineering, data integration, database development, or related technical work.
- Bachelor’s degree in computer science, Cybersecurity, Data Science, Software Engineering, Information Technology, Information Systems, or a related technical field; OR an active/current certification combination:
- CCSP, CFR, or FITSP-M
- GSEC, CompTIA Security+, or SSCP
- Must be eligible to obtain a security clearance granted by the U.S. Government, requiring U.S. citizenship, with eligibility determined using 13 adjudicative guidelines.
- Demonstrated experience designing and supporting complex enterprise data-ingestion and transformation solutions.
- Advanced hands-on experience with Informatica PowerCenter and/or another Informatica enterprise data-integration platform (mappings, transformations, workflows, sessions, reusable components, parameterization, scheduling).
- Advanced SQL skills: complex queries, joins, views, stored procedures, functions, indexing, query optimization, and troubleshooting.
- Experience with Oracle, PostgreSQL, MySQL, Amazon Aurora/RDS, SQL Server, or comparable relational platforms.
- Experience integrating data from APIs, web services, relational databases, message-based interfaces, and file-based sources.
- Experience designing batch, incremental, event-driven, micro-batch, or CDC ingestion processes.
- Experience developing source-to-target mappings, transformation rules, metadata, lineage, and interface documentation.
- Experience implementing data-quality validation, reconciliation, error-handling, restart, and recovery controls.
- Experience troubleshooting and tuning high-volume or mission-critical ETL and data-processing workloads.
- Experience using Python, Bash, PowerShell, SQL, or another applicable scripting/programming language for data processing and automation.
- Experience using Git and automated software-delivery processes for ETL, SQL, or data-engineering components.
- Familiarity with cloud-native data services and managed database platforms supporting enterprise ingestion and transformation workloads.
- Strong technical leadership, analytical ability, troubleshooting skills, documentation, mentoring, and stakeholder communication.
Technologies
- Informatica, Informatica PowerCenter
- SQL
- Amazon Aurora, Amazon Aurora/RDS, Amazon RDS
- Oracle, PostgreSQL, MySQL, SQL Server
- Python, Bash, PowerShell
- Git, GitLab CI/CD
- APIs, web services, message-based interfaces
- RMF, STIG
- CCSP, CFR, FITSP-M, GSEC, CompTIA Security+, SSCP
Benefits
- Instant Flexible PTO
- Generous Holidays
- Immediate Health Coverage
- 401K Safe Harbor Match
- Growth Opportunities
- Free Employee Assistance Program (EAP)
Preferred
- Experience modernizing legacy Informatica workflows or migrating data-integration capabilities to AWS-based environments.
- Experience with Informatica Intelligent Cloud Services (IICS), Informatica Cloud Data Integration, or comparable Informatica cloud capabilities.
- Experience processing large-volume, high-velocity, or mission-critical transportation and logistics data.
- Experience with Amazon Aurora PostgreSQL or Aurora MySQL, Amazon RDS, S3, Glue, Lambda, Kinesis, Database Migration Service, CloudWatch, or comparable approved cloud data services.
- Experience preparing datasets, views, or data services for Apache Superset or another enterprise business-intelligence platform.
- Familiarity with data lakes, event streaming, data catalogs, metadata management, and data-governance practices.
- Familiarity with containerized environments, infrastructure-as-code processes, and automated cloud deployment practices used by cloud/DevSecOps teams.
- Experience supporting USTRANSCOM, Department of Defense logistics, or another mission-critical federal data environment.
Location: Scott AFB, IL (onsite)
Compensation: USD 95,000 - 108,000 per year