Data Engineer II
Job Description
Data & Analytics team role focused on building and modernizing enterprise data integration pipelines.
Responsibilities
- Collaborate in the Warehousing & Pipelines function with DBAs, Data Architects, BI Developers, Data Scientists, Software Engineers, and business stakeholders.
- Design, develop, test, deploy, and maintain batch and streaming data pipelines for payment processing systems with strict availability and performance requirements.
- Build data integration solutions using SQL, C#, Python, PowerShell, and related automation technologies.
- Develop and support event-driven architectures using Change Data Capture (CDC), Kafka, messaging platforms, and stream-processing technologies.
- Maintain and modernize legacy data integration systems, including SQL Server Integration Services (SSIS), transitioning toward code-first and version-controlled development practices.
- Create and maintain CI/CD pipelines for data integration, analytics, and data platform solutions.
- Implement monitoring, logging, alerting, and observability capabilities for data platforms and pipelines.
- Partner with Data Architects and enterprise data governance teams to populate and maintain enterprise data warehouse, lakehouse, and analytical data platform solutions.
- Tune SQL queries and database interactions to reduce impact on source systems and improve application performance.
- Optimize pipelines for throughput, scalability, resiliency, and cost efficiency.
- Produce and maintain technical documentation including architectural diagrams, operational runbooks, and deployment procedures.
- Use AI-assisted development tools responsibly to improve productivity and code quality, while independently validating, reviewing, and testing generated code.
- Participate in on-call rotations and provide after-hours support as required.
- Perform other job duties as assigned.
Requirements
- Bachelor’s degree in Computer Science, Information Technology, or Engineering.
- Minimum 3 years of experience in data engineering, software engineering, or database development.
- Experience developing solutions on relational database platforms such as SQL Server and/or MySQL.
- Experience developing applications or services using C# or similar object-oriented programming languages.
- Experience developing and supporting ETL/ELT and data integration solutions.
- Experience working with source control systems such as Git and participating in code reviews and pull request workflows.
- Experience working with containerization technologies such as Docker.
Technologies
- SQL
- C#
- Python
- PowerShell
- Change Data Capture (CDC)
- Kafka
- SQL Server Integration Services (SSIS)
- CI/CD
- Git
- Docker
- MySQL
- ETL/ELT
Nice to Have
- Experience modernizing legacy analytics platforms, OLAP solutions, or analytical workloads.
- Experience with cloud services and data platforms in GCP, AWS, and/or Azure.
- Experience with Infrastructure as Code (IaC) using Terraform, Pulumi, Bicep, Ansible, or similar technologies.
- Experience building and supporting Kubernetes-based applications and services.
- Experience developing data lakes, lakehouses, and open table format solutions using technologies such as Apache Iceberg, Delta Lake, Hudi, Parquet, or similar platforms.
- Experience operating under PCI DSS, SOC, financial services, healthcare, or similar regulatory frameworks.
Location / Onsite Interview Requirement
- This is a remote position.
- Candidates must live within approximately a 70-mile radius of office locations in Allen, TX; Lenexa, KS; Springfield, MO; or Monett, MO; Louisville, KY; Troy, MI; Birmingham, AL.
- All positions, regardless of location, may require an onsite interview or in-person onboarding to verify identity.