Data Engineer

Agile Defense is building trusted solutions in complex domains, and this role supports mission-critical data systems through scalable engineering, real-time processing, and strong data governance. You will work onsite in Omaha, NE within a SCIF in STRATCOM, contributing to reliable pipelines and analytics-ready data that teams can act on with confidence.

Responsibilities

• Design, build, and maintain highly scalable, reliable, and efficient ETL/ELT pipelines that move data from multiple sources into data warehouses, data lakes, and other storage systems.
• Collaborate with data scientists, analysts, and other stakeholders to understand data needs and ensure data flows efficiently across platforms and systems.
• Develop, deploy, and maintain real-time streaming solutions with low latency using tools such as Apache Kafka, Flink, or Spark Streaming.
• Optimize and automate data workflows so pipelines remain efficient and dependable as data volumes grow.
• Oversee data model and schema design to support accurate, accessible, and high-performance storage for analytics and reporting.
• Ensure data quality, integrity, and consistency through validation, monitoring, and error-handling mechanisms.
• Monitor systems to identify and troubleshoot performance bottlenecks affecting data processing and data access.
• Stay current with emerging trends and best practices in data engineering, recommending tools, technologies, and methods to improve infrastructure.
• Mentor and guide junior data engineers, reinforcing best practices and supporting a collaborative, high-performance team environment.

Requirements

• Strong understanding of data validation, model testing, and performance evaluation techniques.
• Experience with data visualization and storytelling using tools such as Tableau, Plotly, or Matplotlib.
• Excellent technical communication skills, including the ability to explain complex concepts to non-technical audiences.
• 5+ years of experience in data engineering or software engineering, including experience designing and managing complex data pipelines.

Preferred

• 4+ years of experience in applied data science, machine learning engineering, or data pipeline development.
• Proficiency in Python, SQL, and distributed data frameworks such as Spark, Databricks, and PySpark.
• Experience developing ML models from training through deployment using tools and libraries such as scikit-learn, TensorFlow, XGBoost, and MLflow.
• Familiarity with MLOps, API development, and secure cloud environments (e.g., AWS, Azure, Palantir Foundry).

Technologies

Apache Kafka, Flink, Spark Streaming, Tableau, Plotly, Matplotlib, Python, SQL, Spark, Databricks, PySpark, scikit-learn, TensorFlow, XGBoost, MLflow, AWS, Azure, Palantir Foundry

Education

Bachelor’s degree in Computer Science, Information Systems, Data Engineering, or a related field (master’s degree preferred). Equivalent relevant work experience may substitute for education, with each year of experience substituting for each year of required education.

Core Values

• Happy - Be Infectious.
• Helpful - Be Supportive.
• Honest - Be Trustworthy.
• Humble - Be Grounded.
• Hungry - Be Eager.
• Hustle - Be Driven.