Databricks Platform Data Engineer
Apache Spark
Azure Databricks
Big Data
Bigdata
CI/CD
Cloud Data Engineering
Cloud Data Platform
Cloud Platform
Cloud Platforms
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Engineering
Data Governance
Data Integration
Data Integration Tools
Data Lake
Data Lakehouse
Data Management
Data Pipeline
Data Platform
Data Processing
Data Security
Data Warehouse
Database
Databases
Databricks
Databricks Dlt
Databricks Genie
Databricks Lakeflow
Databricks Lakehouse
Databricks Lakehouse Platform
Databricks Pyspark
Databricks Spark
Delta Lake
DevOps
Devops Tools
Engineer
ETL
Informatica
Information Technology (IT)
Programming
Programming Language
Programming Languages
Reporting and Analytics
Software Development
Software Engineering
Spark
SQL
Job Description
NTT DATA (Databricks Center of Excellence, CoE) is building enterprise-grade Lakehouse platforms and client-ready data solutions. This Databricks Platform Data Engineer role supports that mission by designing, deploying, governing, and scaling Databricks Lakehouse environments across major cloud platforms. You’ll also help drive adoption of modern Databricks capabilities, enabling teams to deliver production workloads, AI/BI dashboards, and advanced analytics use cases with a strong focus on quality and reliability.
What you’ll do
- Design and deploy enterprise-scale Databricks Lakehouse platforms across AWS and Azure
- Establish secure, governed environments using Unity Catalog, role-based access controls, and data lineage
- Define platform standards, reusable patterns, and guardrails to accelerate scalable adoption
- Optimize performance, cost, and reliability for production workloads
- Implement CI/CD, environment promotion, and DevOps automation for Databricks
- Lead adoption of modern capabilities such as Unity Catalog, Dataflow / declarative pipelines, and Genie / AI-assisted development
- Enable use cases for AI/BI dashboards, model serving, and advanced analytics
- Build and optimize batch and streaming pipelines using Apache Spark and Delta Lake
- Develop data products and domain-oriented pipelines aligned to enterprise data strategies
- Own end-to-end delivery of client use cases from requirements through production deployment
- Drive data quality, observability, and pipeline reliability across platforms and workloads
- Serve as a trusted advisor to client stakeholders across architecture, data, risk, and business teams
- Translate business requirements into technical architecture and delivery roadmaps
- Lead workshops, solution design sessions, and platform adoption strategies
- Support proposals, solutioning, and client innovations within regulated industries
- Contribute to NTT DATA accelerators, frameworks, and reusable assets
- Define best practices, reference architectures, and playbooks for enterprise Databricks adoption
- Mentor junior engineers and help build capability across the CoE
What you bring
- 12+ years of experience in data engineering, platform engineering, or data architecture
- 5+ years hands-on experience with Databricks in large enterprise environments
- Deep expertise in Apache Spark using Scala/Python
- Deep expertise in Delta Lake and Lakehouse architectures
- Deep expertise in Databricks workspace setup, cluster policies, and job orchestration
- Strong cloud platform experience: AWS, Azure, or GCP
- Experience implementing data governance, security, and compliance controls
- Proven ability to design and deliver scalable, production-grade data platforms
- Strong client-facing and communication skills
Technologies you’ll work with
- Databricks, Unity Catalog, Dataflow, Genie
- Apache Spark, Scala, Python, Delta Lake, Lakehouse
- AWS, Azure, GCP
- CI/CD, DevOps automation
- Kafka, Spark and Delta Lake, Structured Streaming
- MLOps, Model Serving, AI/BI
- AI-assisted development workflows and agent-based tooling
Locations
- Plano, TX (onsite)
- Minneapolis, MN
- Chicago, IL
- Atlanta, GA
Experience level: 12+ years overall; 5+ years with Databricks (enterprise-scale).
Preferred qualifications
- Experience in financial services or other regulated industries
- Familiarity with data governance frameworks, regulatory reporting, and risk data environments
- Databricks certifications (e.g., Data Engineer, Solutions Architect)
- Experience with real-time streaming (Kafka, Structured Streaming)
- Experience with MLOps / Model Serving
- Experience with data marketplace / data product architectures
- Exposure to AI-assisted development workflows and agent-based tooling