Agile Defense is building trusted solutions in complex domains, and this role supports mission-critical data systems through scalable engineering, real-time processing, and strong data governance. You will work onsite in Omaha, NE within a SCIF in STRATCOM, contributing to reliable pipelines and analytics-ready data that teams can act on with confidence.
Responsibilities
- Design, build, and maintain highly scalable, reliable, and efficient ETL/ELT pipelines that move data from multiple sources into data warehouses, data lakes, and other storage systems.
- Collaborate with data scientists, analysts, and other stakeholders to understand data needs and ensure data flows efficiently across platforms and systems.
- Develop, deploy, and maintain real-time streaming solutions with low latency using tools such as Apache Kafka, Flink, or Spark Streaming.
- Optimize and automate data workflows so pipelines remain efficient and dependable as data volumes grow.
- Oversee data model and schema design to support accurate, accessible, and high-performance storage for analytics and reporting.
- Ensure data quality, integrity, and consistency through validation, monitoring, and error-handling mechanisms.
- Monitor systems to identify and troubleshoot performance bottlenecks affecting data processing and data access.
- Stay current with emerging trends and best practices in data engineering, recommending tools, technologies, and methods to improve infrastructure.
- Mentor and guide junior data engineers, reinforcing best practices and supporting a collaborative, high-performance team environment.
Requirements
- Strong understanding of data validation, model testing, and performance evaluation techniques.
- Experience with data visualization and storytelling using tools such as Tableau, Plotly, or Matplotlib.
- Excellent technical communication skills, including the ability to explain complex concepts to non-technical audiences.
- 5+ years of experience in data engineering or software engineering, including experience designing and managing complex data pipelines.
Preferred
- 4+ years of experience in applied data science, machine learning engineering, or data pipeline development.
- Proficiency in Python, SQL, and distributed data frameworks such as Spark, Databricks, and PySpark.
- Experience developing ML models from training through deployment using tools and libraries such as scikit-learn, TensorFlow, XGBoost, and MLflow.
- Familiarity with MLOps, API development, and secure cloud environments (e.g., AWS, Azure, Palantir Foundry).
Technologies
Apache Kafka, Flink, Spark Streaming, Tableau, Plotly, Matplotlib, Python, SQL, Spark, Databricks, PySpark, scikit-learn, TensorFlow, XGBoost, MLflow, AWS, Azure, Palantir Foundry
Education
Bachelor’s degree in Computer Science, Information Systems, Data Engineering, or a related field (master’s degree preferred). Equivalent relevant work experience may substitute for education, with each year of experience substituting for each year of required education.
Core Values
- Happy - Be Infectious.
- Helpful - Be Supportive.
- Honest - Be Trustworthy.
- Humble - Be Grounded.
- Hungry - Be Eager.
- Hustle - Be Driven.