Data Engineer

What you’ll get

Join a hybrid Data Engineer opportunity with Horizontal Talent, where you’ll help build and improve modern big data solutions across ingestion, integration, and transformation. The role supports cloud-based data platform services and works on reliable, efficient pipelines at scale.

Based in Knoxville, TN (hybrid), you can collaborate with project teams and attend onsite interviews or equipment pickup when required.

Responsibilities

• Design and implement data engineering components aligned to client requirements and technical solution goals.
• Build and maintain batch and real-time ingestion, integration, and transformation workflows.
• Work with large-scale data processing and storage technologies to support data pipelines.
• Contribute to analytics, search, and aggregation capabilities across data platforms.
• Participate in technical design discussions to help ensure solution health and performance.
• Support cloud-based data platform services and optimize workflows for performance and scalability.
• Collaborate in a hybrid environment, including coordination for onsite needs such as equipment pickup.

Requirements

• 4+ years of hands-on experience with big data technologies.
• Experience with Hadoop ecosystem tools such as HDFS, Kafka, Spark, Spark Streaming, Hive, NiFi, Airflow, Flink, Storm, Oozie, or similar platforms.
• Strong programming skills in Java, Python, or Scala, with Java preferred.
• Working knowledge of NoSQL and MPP data platforms such as HBase, MongoDB, Cassandra, AWS Redshift, Azure SQLDW, or GCP BigQuery.
• Experience with AWS data platform services.
• Ability to translate business needs into technical design and implementation.
• Strong problem-solving skills and the ability to work independently on design discussions and solution delivery.

Technologies you may work with

• HDFS, Kafka, Spark, Spark Streaming, Hive, NiFi, Airflow, Flink, Storm, Oozie
• Java, Python, Scala
• HBase, MongoDB, Cassandra
• AWS Redshift, Azure SQLDW, GCP BigQuery
• AWS messaging: ActiveMQ, RabbitMQ, Solace
• Data tools and databases: Informatica, Talend, Oracle, MySQL, SQL Server, Postgres, Collibra, Alation

Preferred skills

• Experience with traditional ETL tools such as Informatica or Talend.
• Familiarity with database technologies such as Oracle, MySQL, SQL Server, or Postgres.
• Knowledge of data governance practices, including security, lineage, and catalog tools such as Collibra or Alation.
• Exposure to distributed messaging tools such as ActiveMQ, RabbitMQ, or Solace.
• Experience with search/indexing and microservices architectures.
• Background in performance tuning and data pipeline optimization.
• Cloud data certifications or other relevant big data credentials.

Additional information

Horizontal Talent may contact you about your application using automated technology, including phone calls, SMS/text messages, or email, which may be delivered by their virtual AI recruiter, Alex.

Horizontal is committed to building an inclusive workplace where diverse perspectives are valued, respected, and encouraged.