DeveloperJobs.io
← Back to all jobs

Job Description

Guidehouse is seeking a Cleared Data Engineer to build and support scalable batch and streaming data pipelines in Databricks. This role supports cybersecurity metrics, risk analysis, and reporting, and requires an Active Secret clearance along with experience across data quality, orchestration, optimization, governance, and stakeholder collaboration.

Location

Arlington, VA (onsite)

Compensation

USD 85,000 - 141,000 per year

Key Responsibilities

  • Design, develop, test, deploy, and maintain scalable batch and streaming data pipelines in Databricks using Python, PySpark, SQL, and reusable engineering patterns.
  • Transform raw source data into curated, analysis-ready datasets and dimensional models that support cybersecurity metrics, risk analysis, dashboards, and executive reporting.
  • Build automated data-quality controls for ETL jobs, including checks for schema, completeness, validity, consistency, uniqueness, timeliness, referential integrity, and source-to-target reconciliation.
  • Define and implement handling for failed quality checks, including quarantining invalid records, stopping critical loads, generating alerts, documenting exceptions, and coordinating corrective actions with data owners.
  • Schedule and orchestrate production workflows, monitor job status and pipeline performance metrics (freshness, throughput, latency, resource utilization), troubleshoot root causes, and restore processing within agreed service levels.
  • Improve pipeline reliability, performance, and cost through efficient transformations, workload configuration, incremental processing, partitioning, and compute-resource management.
  • Implement and maintain data governance and quality assurance standards, including access controls, metadata, data classification, ownership, lineage, retention, auditability, and protection of sensitive cybersecurity information.
  • Maintain source-to-target mappings, data dictionaries, transformation rules, validation evidence, standard operating procedures, runbooks, and other technical documentation for operations, knowledge transfer, and audit readiness.
  • Collaborate with cybersecurity analysts, data owners, system administrators, dashboard developers, and business stakeholders to define data requirements, metrics, thresholds, and acceptance criteria.
  • Maintain effective client and internal relationships, communicate delivery risks and dependencies, anticipate stakeholder needs, and address issues in a timely and professional manner.
  • Organize and lead client meetings by coordinating schedules, creating and distributing agendas and pre-read materials, facilitating discussions, recording minutes, maintaining required documentation, and tracking action items through closure.
  • Participate in peer reviews, testing, release activities, and continuous-improvement initiatives to strengthen engineering practices, reduce recurring defects, and improve data product reliability and usability.

Required Qualifications

  • ACTIVE and CURRENT SECRET federal security clearance
  • Bachelor’s degree
  • SIX (6) years total experience in a data or programming discipline, including THREE (3) years of data engineering experience
  • Proficiency in data engineering-related programming languages such as Python, PySpark, and/or SQL
  • Experience with scheduling, monitoring, troubleshooting, and optimizing production ETL/ELT pipelines
  • Experience with data engineering principles and best practices
  • Excellent problem-solving skills and attention to detail
  • Selected Candidate must be able to commute to client and Guidehouse offices as needed

Preferred (Nice to Have)

  • Master’s degree in computer science or related field
  • Experience working with system and application logs
  • Familiarity with the Databricks platform
  • Strong communication and collaboration skills for effective teamwork

Technology Stack

  • Databricks
  • Python
  • PySpark
  • SQL

Benefits

  • Medical, Rx, Dental & Vision Insurance
  • Personal and Family Sick Time & Company Paid Holidays
  • Position may be eligible for a discretionary variable incentive bonus
  • Parental Leave and Adoption Assistance
  • 401(k) Retirement Plan
  • Basic Life & Supplemental Life
  • Health Savings Account, Dental/Vision & Dependent Care Flexible Spending Accounts
  • Short-Term & Long-Term Disability
  • Student Loan PayDown
  • Tuition Reimbursement, Personal Development & Learning Opportunities
  • Skills Development & Certifications
  • Employee Referral Program
  • Corporate Sponsored Events & Community Outreach
  • Emergency Back-Up Childcare Program
  • Mobility Stipend

Travel Requirements

Up to 50%

Clearance Required

Active Secret

Minimum Experience

3 years (data engineering)

Similar Jobs