Senior Data Engineer
Senior
Apache Airflow
Artificial Intelligence
Big Data
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Integration
Data Lake
Data Lakehouse
Data Modeling
Data Pipeline
Data Platform
Data Processing
Data Warehouse
Data Warehousing
Databricks
Databricks Workflows
Dbt
ETL
Google Cloud Bigquery
Lakehouse
Reporting and Analytics
Snowflake
Spark
SQL
Workflow Orchestration
Job Description
Principal Mineral Holdings is hiring a Senior Data Engineer in Camden, SC (onsite) to build a production-grade Databricks lakehouse and the pipelines that support analytics, automation, and ML/AI workflows.
Responsibilities
- Architect, build, and maintain batch and near-real-time ELT/ETL pipelines that are scalable, reliable, and maintainable.
- Ingest and curate data from multiple sources, including internal ERP systems and external market, trade, and commodity API feeds.
- Design and maintain central data models in dbt (or similar) to support analytics, data science, optimization, and agentic AI workflows.
- Work with business owners and subject matter experts to define and standardize operational KPIs (yield, inventory, margins, component SKUs) in a unified semantic layer.
- Run QA checks comparing source systems to business reports; reconcile discrepancies with data owners.
- Implement data quality validation, monitoring, and alerting so downstream users can trust delivered data.
- Help establish best practices for data modeling, naming, version control, and documentation.
- Optimize lakehouse architecture and pipeline performance while balancing cost and scalability.
- Collaborate with ML and platform engineers on model training, scoring, and deployment needs.
- Support security, privacy, lineage, and governance standards, including requirements tied to PMC U.S. government contracts.
- Implement and review role-based access control and security-as-code alongside the DevOps/MLOps Engineer.
Requirements
- 4+ years of professional data engineering experience building production pipelines.
- Strong SQL and Python; experience with Spark or a comparable distributed engine.
- Hands-on experience with a lakehouse or cloud warehouse (Databricks preferred; Snowflake or BigQuery considered).
- Experience with dbt or a comparable transformation and testing framework.
- Experience with Git-based workflows, CI, and orchestration (Databricks Workflows, Airflow, or similar).
- Comfort working directly with non-technical business owners to define metrics and resolve data issues.
- Ability to work as a U.S. person and support government contract data requirements.
Preferred
- Experience extracting data from SAP (ECC or S/4HANA) or other ERP systems.
- Experience in manufacturing, supply chain, or industrial data.
- Experience with Unity Catalog, Delta Lake, and data quality tooling.
- Experience supporting ML feature pipelines.
Technologies
- Databricks, Databricks lakehouse, dbt, SQL, Python, Spark
- Snowflake, BigQuery
- Git, CI, Databricks Workflows, Airflow
- SAP, SAP ECC, SAP S/4HANA
- Unity Catalog, Delta Lake