Pandera Developer Jobs

1 job found

Today

Data Engineer

Infinitive Inc McLean, VA

McLean, VA

Today

$90k - $154k/yr Full time Apache Kafka Apache Kafka Connect API

Apply Now

Job Description

Infinitive Inc is hiring a Data Engineer to help design and scale next-generation, event-driven data platforms. The work centers on building reliable streaming pipelines with Apache Kafka and orchestrating long-running distributed workflows with Temporal, alongside batch and near-real-time ETL/ELT.

This onsite role is based in McLean, VA, and you will collaborate across engineering teams to standardize data contracts, keep schemas validated, and deliver production-grade data model performance across modern cloud data platforms.

Responsibilities

Architect, deploy, and maintain high-volume distributed data streams using Apache Kafka, including producers, consumers, Kafka Connect, and Schema Registry.
Define and enforce schema design standards, versioning strategies, and automated schema validation using formats such as Apache Avro, Protocol Buffers, and JSON Schema to maintain strict data contracts across microservices, streaming consumers, and lakehouse storage.
Design and implement durable execution workflows with Temporal to coordinate long-running distributed pipelines, apply compensation via the Saga pattern, and manage cross-system ETL tasks.
Build end-to-end batch and near-real-time pipelines using Python, SQL, and Apache Spark / PySpark.
Design and optimize analytical data models, including dimensional/star schema patterns, in cloud data warehouses and lakehouses such as Snowflake, BigQuery, Databricks, and Redshift.
Implement automated testing, continuous schema validation, data drift detection, and observability across both streaming and batch workflows.
Partner with software engineers, machine learning engineers, and analysts to define standard schema definitions, data contracts, and production-grade CI/CD release patterns.

Requirements

4+ years of professional experience in data engineering, backend distributed systems, or software engineering.
Hands-on experience with Temporal (or Cadence), including durable workflows, activities, retries, signals, queries, and long-running distributed orchestration.
Deep expertise with Apache Kafka, including message partitioning, consumer groups, offset management, and topic design.
Demonstrated proficiency with schema definition frameworks such as Apache Avro, Protocol Buffers/gRPC, or JSON Schema.
Practical experience managing schema evolution with compatibility modes such as backward, forward, and full, using schema registries like Confluent Schema Registry or AWS Glue Schema Registry.
Experience enforcing data validation rules, contract testing, and data quality checks using tools such as Great Expectations, Pandera, Pydantic, and dbt tests.
Strong programming proficiency in Python (with Go or Java as a plus), including clean code, design patterns, and unit/integration testing standards.
Hands-on development with Apache Spark via PySpark and Spark SQL for large-scale datasets.
Strong experience with relational databases, dimensional data modeling, and query performance tuning.

Technologies

Apache Kafka, Kafka Connect, Schema Registry
Temporal, Cadence
Apache Avro, Protocol Buffers, gRPC, JSON Schema
Confluent Schema Registry, AWS Glue Schema Registry
Great Expectations, Pandera, Pydantic, dbt
Python, SQL
Apache Spark, PySpark, Spark SQL
Snowflake, BigQuery, Databricks, Redshift

Compensation: A reasonable estimate of the current range for this role in the U.S. is $90,000 - $154,00.00 per year.

View full job page →

Get Pandera Developer Jobs alerts

New Pandera Developer Jobs, daily.