Data Engineer Ab initio
Ab Initio
Automation
Big Data
Cloud Infrastructure
Cloud Native
Cloud Native Technologies
Cloud Platform
Cloud Platforms
Cloud Platforms Cloud Platforms
Cloud Technology
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Integration
Data Integration Tools
Data Pipeline
Data Pipelines
Data Platform
Data Processing
Data Warehouse
Database
Databases
DevOps
Devops Tools
DevSecOps
Docker
Engineer
Engineering
Engineering Software
ETL
Event Driven Architecture
Informatica
Information Technology (IT)
Infrastructure
Infrastructure As Code
Integration
Kafka
Kubernetes
Platform Engineering
Programming
Programming Language
Programming Languages
Security Devops
Software Development
SQL
Stream Processing
Streaming Data
Unix Linux Shell
Job Description
The Data Engineer Ab initio role at Robotics Technologies is based in Houston, TX onsite and focuses on building high-performance, event-driven Ab Initio data pipelines. The position emphasizes low-latency processing, high availability, and operational reliability through continuous flows and strong production monitoring.
Responsibilities
- Create complex Ab Initio continuous flow graphs, including stateful joins, sliding time windows, and aggregations.
- Implement event-driven pipelines using Kafka, MQ, and file streams.
- Ensure resilience of continuous flows through checkpointing and recovery to support exactly-once processing semantics.
- Use advanced Ab Initio components such as Reformat, Rollup, Join, and Partition to maintain low-latency performance.
- Monitor live production streams and troubleshoot data issues to maintain 24/7 reliability.
- Develop ETL pipelines for both batch and real-time ingestion and transformation.
- Implement and enforce data validation, data security, data integrity, and compliance across big data platforms.
- Monitor and troubleshoot performance challenges in large-scale clusters.
- Collaborate with data scientists, analysts, and application teams to deliver high-quality data solutions.
- Automate workflows and improve operational efficiency using scripting and orchestration tools.
Requirements
- Deep understanding of credit card process system.
- Deep knowledge of GDE (Graphical Development Environment), EME (Enterprise Meta>Environment), Conduct>It, and Continuous Flows.
- Understanding of Kafka, message queues, and real-time stateful services.
- Proficiency in Unix/Linux shell scripting, SQL, and database technologies such as Oracle and Teradata.
- Experience with Java, Scala, Python, or Kafka is a plus.
- Familiarity with Linux/Unix environments and shell scripting.
- Understanding of data security, governance, and compliance standards.
- Experience with cloud-based big data platforms.
- Exposure to containerization technologies such as Docker and Kubernetes for big data workloads.
- Knowledge of CI/CD pipelines for data engineering projects.
Technology Focus
- Ab Initio, GDE, EME, Conduct>It, Continuous Flows
- Kafka, MQ, Reformat, Rollup, Join, Partition, ETL
- Unix/Linux shell scripting, SQL, Oracle, Teradata
- Java, Scala, Python
- Linux/Unix, Docker, Kubernetes
- CI/CD, checkpointing, recovery
Job Details
- Location: Houston, TX (onsite)
- Salary: USD 78 - 78 per hourly
- Job ID: J54188
- Duration: 8 Months + Extension
- Domain Exposure: Material
- Client: To Be Discussed Later
- Employment Type: W-2, 1099, C2C
Work Authorization
- US Citizen
- Green Card
- OPT-EAD
- CPT
- H-1B
- H4-EAD
- Other