DeveloperJobs.io
← Back to all jobs

Job Description

Arlo Technologies, Inc. is hiring a Senior Staff Data Engineer to design and scale its data platform on Databricks.

Responsibilities

  • Define and evolve data architecture on the Databricks Lakehouse using medallion patterns (Bronze/Silver/Gold) for batch and streaming workloads.
  • Design, build, and maintain scalable ETL/ELT pipelines using Python, PySpark, and SQL.
  • Implement and manage Unity Catalog, including catalogs and schemas, access controls, data lineage, auditing, and secure data sharing.
  • Define and enforce data governance practices across data quality, metadata management, data classification, retention, privacy, and regulatory compliance.
  • Build and maintain CI/CD pipelines for data workflows using Git and GitHub Actions, Azure DevOps, or Jenkins, including Databricks Asset Bundles with automated testing and deployment.
  • Implement data quality checks, plus monitoring and alerting to support reliable, trusted data.
  • Monitor and optimize Databricks jobs for performance, reliability, and cost.
  • Mentor engineers, lead design reviews, and contribute to the data roadmap.

Requirements

  • 5–6 years of experience in data engineering, building production data pipelines and architectures.
  • Hands-on expertise in Databricks (including Spark, Delta Lake, and Workflows).
  • Hands-on experience with Unity Catalog (access control, lineage, catalog and schema design).
  • Strong understanding of data governance concepts: data quality, cataloguing, lineage, privacy, and compliance (including GDPR and CCPA).
  • Advanced skills in Python (PySpark) and SQL.
  • Experience with CI/CD and DataOps practices.
  • Experience with data modelling and a major cloud platform: AWS, Azure, or GCP.
  • Strong communication and cross-functional collaboration.
  • Bachelor’s degree in Computer Science, Engineering, or a related field.

Technologies

  • Databricks
  • Spark
  • Delta Lake
  • Workflows
  • Unity Catalog
  • Python
  • PySpark
  • SQL
  • Git
  • GitHub Actions
  • Azure DevOps
  • Jenkins
  • Databricks Asset Bundles
  • AWS
  • Azure
  • GCP

Location

  • Milpitas, CA (onsite)

Compensation

  • USD 140,000 - 160,000 per year
  • The total compensation package may also include other elements, including bonus, equity, and a full range of benefits.

Similar Jobs