DeveloperJobs.io
← Back to all jobs
RADcube

Data Engineer – Databricks (IBU Consumer)

RADcube
Indianapolis, IN Full time Posted 5d ago
Big Data Bigdata Cloud Data Engineering Cloud Data Platform Cloud Platform Cloud Platforms Data Data Analysis Data Analytics Data Architecture Data Engineer Data Engineering Data Governance Data Integration Data Integration Tools Data Lake Data Lakehouse Data Management Data Pipeline Data Pipelines Data Platform Data Processing Data Security Data Warehouse Database Databases Databricks Databricks Lakehouse Databricks Pyspark Databricks Workflows ETL Informatica Information Technology (IT) Integration Programming Programming Languages Software Development Spark SQL Unity Catalog
Apply Now

Job Description

Build and maintain production-grade data pipelines on Databricks to support consumer-facing IBU data products.

Responsibilities

  • Develop and maintain scalable data pipelines on Databricks for consumer-facing IBU data products
  • Create ETL/ELT pipelines using PySpark, Spark SQL, and Delta Lake
  • Implement a medallion architecture (bronze/silver/gold) using Unity Catalog and Workflows
  • Optimize cluster performance and cost for production workloads
  • Monitor and troubleshoot Databricks jobs in production

Requirements

  • 4+ years of data engineering experience
  • Experience with Databricks
  • Strong PySpark and SQL skills
  • Experience with Git and CI-CD
  • Nice to have: experience with pharma and/or consumer data
  • Nice to have: Databricks certification

Technology Stack

  • Databricks
  • PySpark
  • Spark SQL
  • Delta Lake
  • Unity Catalog
  • Workflows
  • Git
  • CI-CD

Job Details

  • Job Type: Full time
  • Location: Indianapolis, Indiana, United States (onsite)
  • Experience Range: 4-8 years
  • Date Opened: 10/29/2026
  • Industry: Technology
  • Certifications: Databricks
  • Postal Code: 46204

Similar Jobs