DeveloperJobs.io
← Back to all jobs

Job Description

Principal Mineral Holdings is hiring a Senior Data Engineer in Camden, SC (onsite) to build a production-grade Databricks lakehouse and the pipelines that support analytics, automation, and ML/AI workflows.

Responsibilities

  • Architect, build, and maintain batch and near-real-time ELT/ETL pipelines that are scalable, reliable, and maintainable.
  • Ingest and curate data from multiple sources, including internal ERP systems and external market, trade, and commodity API feeds.
  • Design and maintain central data models in dbt (or similar) to support analytics, data science, optimization, and agentic AI workflows.
  • Work with business owners and subject matter experts to define and standardize operational KPIs (yield, inventory, margins, component SKUs) in a unified semantic layer.
  • Run QA checks comparing source systems to business reports; reconcile discrepancies with data owners.
  • Implement data quality validation, monitoring, and alerting so downstream users can trust delivered data.
  • Help establish best practices for data modeling, naming, version control, and documentation.
  • Optimize lakehouse architecture and pipeline performance while balancing cost and scalability.
  • Collaborate with ML and platform engineers on model training, scoring, and deployment needs.
  • Support security, privacy, lineage, and governance standards, including requirements tied to PMC U.S. government contracts.
  • Implement and review role-based access control and security-as-code alongside the DevOps/MLOps Engineer.

Requirements

  • 4+ years of professional data engineering experience building production pipelines.
  • Strong SQL and Python; experience with Spark or a comparable distributed engine.
  • Hands-on experience with a lakehouse or cloud warehouse (Databricks preferred; Snowflake or BigQuery considered).
  • Experience with dbt or a comparable transformation and testing framework.
  • Experience with Git-based workflows, CI, and orchestration (Databricks Workflows, Airflow, or similar).
  • Comfort working directly with non-technical business owners to define metrics and resolve data issues.
  • Ability to work as a U.S. person and support government contract data requirements.

Preferred

  • Experience extracting data from SAP (ECC or S/4HANA) or other ERP systems.
  • Experience in manufacturing, supply chain, or industrial data.
  • Experience with Unity Catalog, Delta Lake, and data quality tooling.
  • Experience supporting ML feature pipelines.

Technologies

  • Databricks, Databricks lakehouse, dbt, SQL, Python, Spark
  • Snowflake, BigQuery
  • Git, CI, Databricks Workflows, Airflow
  • SAP, SAP ECC, SAP S/4HANA
  • Unity Catalog, Delta Lake

Similar Jobs