Databricks

Databricks Developer Jobs

95 jobs found

Today

Lead Software Engineer - Python, AWS, Data, Agentic AI

JPMorganChase Houston, TX

Houston, TX

Today

Full time Backend Developer Python Amazon Emr

Apply Now

Job Description

Lead Software Engineer at JPMorganChase in Houston, TX, focused on delivering secure, scalable technology across multiple business functions with emphasis on Python, AWS, data-intensive workflows, and agentic AI.

Responsibilities

Architect and implement creative software solutions, handling development and debugging to address non-standard problems and decompose complex technical challenges.
Write secure, production-grade code and perform code reviews and debugging for teammates.
Proactively identify recurring issues and automate remediation to boost stability across applications and systems.
Lead evaluation sessions with external vendors, startups, and internal teams to assess architectures, credentials, and fit within the existing information architecture.
Foster communities of practice among software engineers to promote awareness and adoption of cutting-edge technologies.
Contribute to an inclusive, diverse, and respectful team culture.

Requirements

Formal training or certification in software engineering and application development with at least seven years of hands-on experience.
Hands-on expertise in Python and/or Java with a track record of delivering secure, production-quality code.
Extensive experience building cloud-native solutions on AWS covering compute, storage, security/IAM, networking, and observability.
Experience with LLMs and agentic systems, including orchestration patterns, tool calls, retrieval, structured outputs, evaluations, and safety/guardrails.
Background building data-intensive applications and workflows with Databricks (Spark, Delta, jobs/workflows).
Proficiency in JavaScript/TypeScript and React for modern web applications.
Experience with automation and continuous delivery practices; strong SDLC discipline including CI/CD, testing, code quality, and release controls.
Advanced understanding of agile delivery, application resiliency, and secure engineering practices.
Strong system design skills with the ability to make pragmatic trade-offs and align stakeholders.

Technologies

Python
Java
JavaScript
TypeScript
React
AWS
EC2
EKS
RDS
S3
Lambda
EMR
Glue
Databricks
Spark
Delta Lake
PySpark
Hadoop
Snowflake
Airflow

Benefits

Comprehensive health care coverage
On-site health and wellness centers
A retirement savings plan
Backup childcare
Tuition reimbursement
Mental health support
Financial coaching

View full job page →

Today

Software Engineer III-Databricks

JPMorganChase Wilmington, DE

Wilmington, DE

Today

Full time Big Data Cloud Operations Data Analysis

Apply Now

Job Description

A Software Engineer III specializing in Databricks, onsite in Wilmington, DE, within JPMorgan Chase's Corporate Sector Enterprise Technology team, delivering secure, scalable solutions across multiple domains to support the firm's objectives.

Responsibilities

Provide technical leadership across design, development, and troubleshooting for complex, multi-domain solutions; establish engineering standards and best practices for the team.
Write secure, high-quality code in Python and/or Java; lead code reviews and mentor engineers to improve quality and maintainability.
Build data pipelines using Databricks ETL.
Develop and productionize cloud-based ML pipelines; drive model deployment and operationalization in collaboration with Data Science and SRE/Platform teams.
Own MLOps workflows; coordinate infrastructure and production changes with SRE; ensure resiliency, observability, and security across the ML lifecycle.
Apply SDLC tooling and automation to improve delivery velocity and reliability; champion CI/CD and cloud-native best practices.
Partner with Product Owners and business stakeholders to translate requirements into scalable solutions aligned to CCB Finance objectives.
Foster a team culture of diversity, opportunity, inclusion, and respect; model proactive learning in AI/ML and emerging technologies.
Leverage enterprise-authorized AI coding assist tools to improve code quality, delivery speed, and productivity across complex deliverables; ensure outputs are peer reviewed, automated tests are used, and security standards are upheld; share learnings and reusable patterns to enhance team effectiveness.
Apply knowledge of the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to maximize the value of automation.

Requirements

Formal training or certification in software engineering with 3+ years of applied experience.
Hands-on experience across software engineering, system design, application development, testing, and operational stability.
Proficiency in Python; strong grounding in secure data practices.
Hands-on Databricks experience across Delta Lake, Unity Catalog, Workflows, Repos/notebooks, and SQL Warehouses, including cluster configuration and optimization.
Cloud engineering experience building ML pipelines and deploying models to production with AWS services such as ECS, EMR, Lambda, EC2, SageMaker.
Experience with PySpark, Kafka, Terraform, and Kubernetes for data processing, streaming, IaC, and container orchestration.
Database experience with Oracle and/or Cassandra.
Familiarity with CI/CD, application resiliency, security best practices, Agile/Scrum methodologies, and SDLC automation tools.
Hands-on experience using enterprise-authorized AI-assisted software development tools within the work environment, with demonstrated ability to critically evaluate, validate, and refine AI-generated outputs for correctness, performance, and security.
Understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; ability to guide peers on safe and effective usage within team practices.

Technologies

Python
Java
Databricks
Delta Lake
Unity Catalog
Workflows
Repos/notebooks
SQL Warehouses
AWS ECS
AWS EMR
AWS Lambda
AWS EC2
AWS SageMaker
PySpark
Kafka
Terraform
Kubernetes
Oracle
Cassandra

Benefits

Base salary determined by role, experience, skills, and location
Commission-based pay and discretionary incentive compensation, paid in cash and/or forfeitable equity
Comprehensive health care coverage
On-site health and wellness centers
Retirement savings plan
Backup childcare
Tuition reimbursement
Mental health support
Financial coaching

View full job page →

Today

Senior Software Engineer II

RWE Americas, LLC Austin, TX

Austin, TX

Today

Full time Senior Automation Azure

Apply Now

Job Description

The Senior Software Engineer II will advance the Smart Data Pipeline platform on Microsoft Azure, combining hands-on development with architectural leadership to automate ingestion, cleaning, analysis, forecasting, alerting, and equipment lifecycle management for wind assets, with potential expansion to solar and energy storage systems in the future. This role blends technical delivery with cross-functional ownership and mentoring to shape technical strategy across the team.

Responsibilities

Architect and own scalable, cloud-native data pipeline solutions that integrate data systems across the SDP platform.
Define system architecture, data flows, and service boundaries for ingestion, pre-analysis, analysis, alerting, and monitoring pipelines.
Create and maintain architecture diagrams, blueprints, and reusable design patterns to guide development teams.
Drive technology decisions, platform standards, and architectural guidelines; lead architecture reviews and technical decisions.
Lead design and implementation of data collection, pre-analysis, analysis, forecasting, alerting, and VMM pipelines, ensuring data quality, reliability, and validation.
Own solution delivery on Azure based platforms including Azure Functions, pipeline scheduling, cloud storage, and data lake integration across DEV, UAT, and PROD environments.
Define and enforce best practices for CI/CD pipelines, infrastructure automation, and observability frameworks (logging, alerting, monitoring dashboards).
Lead integration of measurement systems (lidar, met mast, sensor networks) with enterprise IT platforms, ensuring data consistency, reliability, and traceability.
Lead end-to-end feature delivery (Design Build Deploy Operate); review and approve code across multiple services, enforcing security, performance, and coding standards.
Define and enforce engineering standards, coding guidelines, and architectural patterns; ensure compliance with enterprise architecture and data governance policies.
Act as the primary technical liaison between product owners, business stakeholders, and development teams, translating business requirements into technical architectures and delivery plans.
Mentor engineers across multiple levels; lead knowledge sharing, documentation, and capability-building initiatives that foster engineering excellence and continuous improvement.

Requirements

Bachelor’s degree in Computer Science, Software Engineering, or a related field, or equivalent industry certification or experience.
One or more Azure certifications (eg, AZ-900, AZ-204, AZ-305) or equivalent cloud platform credential is strongly preferred.
Minimum 7 years of relevant software engineering or data platform experience, including hands-on development and technical leadership roles.
Proven experience designing and managing complex, multi-stage data pipelines (ingestion, analysis, forecasting, alerting) with strong data cleaning, transformation, and validation workflows.
Experience architecting and delivering solutions on Azure based platforms, including Kubernetes (AKS), containers, Azure Functions, data lake integration, and multi-environment deployment management.
Experience with distributed systems, microservices, messaging systems (RabbitMQ), event-driven architecture, and API design and integration.
Experience designing access control for applications, including human-to-system and system-to-system access schemes.
Experience leading enterprise-scale IT implementations with coordination of multiple workstreams and stakeholder communications, using Agile or hybrid methodologies.
Strong frontend development background with React and TypeScript, backend development with C#, Python, Terraform, or equivalent IaC, plus SQL and experience with large datasets and data formats such as Parquet and CSV.
Experience with Linux and Windows servers, the ability to own Dockerfiles for both platforms, and ownership of SDP's third-party integrations (such as Bullfrog).
Experience working with global teams across Asia, Europe, and the Americas.
This position is an office-based role with some travel and visits to other RWE Americas office and field locations.
Must be able to sit, walk, or stand for long durations of time.
Additional experience in one or more of the following is a plus: Dapr, measurement systems (lidar, met mast, SCADA, sensors), time-series ML pipelines, climate or wind resource data processing, Azure Databricks and Delta Lake with medallion architecture, Delta Live Tables, Lakeflow, Apache Airflow, and cross-domain problem solving.
Strong diagnostics skills and the ability to design cost-effective systems for suboptimal deployments; excellent collaboration and communication abilities; and a commitment to mentoring and engineering excellence.

Technologies

Azure
Azure Functions
Azure Kubernetes Service (AKS)
Kubernetes
Containers
Data Lake
Azure Databricks
Delta Lake
Delta Live Tables
Lakeflow
Bullfrog
React
Typescript
C#
Python
Terraform
SQL
Parquet
CSV
Linux
Windows
Docker
Dapr
Apache Airflow
RabbitMQ
SCADA
Azure DevOps
Jira
Confluence
GitHub

Benefits

Medical coverage
Dental insurance
Vision coverage
Life insurance
Short-term disability
Long-term disability
401(k) with company match
Flexible Spending Accounts
Employee Assistance Program
Education assistance
Parental leave
Paid time off and holidays
Short-term incentive program

Location

Onsite in Austin, Texas, USA (78701). This role requires presence in the office with occasional travel to other RWE Americas offices and field locations as needed.

View full job page →

Today

Functional Analyst/ Developer (Senior)

JANSON Fairfax, VA 🇺🇸 Remote

Fairfax, VA

Today

$170k - $190k/yr Full time Senior Advanced Analytics Analyst

Apply Now

Job Description

A Senior Functional Analyst / Developer role at JANSON in Fairfax, VA with remote work options, focuses on leading analytics delivery, instructional design, and technical mentorship to enable Army Vantage. The position functions as the primary bridge between operational needs, analytics solutions, and training delivery.

Responsibilities

Lead the development and delivery of prototype predictive analytics and dashboards.
Act as the primary instructor for ORSA level and advanced analytics training.
Direct the creation of reusable analytics patterns and governance standards.
Mentor mid-level and junior staff.
Ensure analytics products are instructional, reusable, and sustainment-ready.
Design and oversee Army Vantage specific training curricula aligned to USAREUR-AF personas (executives, staff, analysts, ORSAs).
Lead advanced training events, including ORSA training and executive-level courses.
Ensure training materials emphasize Army Vantage workflows, access permissions, classification, and reuse over generic analytics tools.
Review and approve training content produced by junior staff for technical accuracy and instructional quality.
Lead the development of instructional analytics products with dashboards, workflows, and predictive analytics suitable as teaching exemplars.
Incorporate data lineage, version control, and collaborative development practices into all training demonstrations.
Mentor mid-level personnel on instructional techniques, classroom delivery, and facilitation of Army training events.
Serve as the primary technical advisor supporting data program design and execution.
Ensure training events support organizational change objectives and adoption milestones defined in the SOW.
Lead after-action reviews for training events and integrate feedback into subsequent deliveries.

Requirements

Master’s degree in Computer Science, Data Science, Engineering, Operations Research, Information Systems, or a related technical field.
7 to 10 years of experience in analytics, data engineering, software development, or operational research.
At least two years delivering analytics or data products using an enterprise analytics platform such as Palantir Foundry, Army Vantage, Databricks, Snowflake, or similar cloud-based platforms.
Strong proficiency in Python and SQL.
Experience developing dashboards, data pipelines, or analytical workflows that support operational decision-making.
Demonstrated experience teaching, mentoring, or leading technical personnel or delivering structured training.

Technologies

Python
SQL
Palantir Foundry
Army Vantage
Databricks
Snowflake
Git

Benefits

Competitive base salary
Hybrid work flexibility
401K with matching incentive
Short and Long-term disability
Life insurance
PTO
Paid holidays
Professional development opportunities

Position Details

Location: Fairfax, VA (remote)
Start: Immediately
Clearance: Secret or must be clearable
Security Clearance: Secret (Required)
Work Location: In person
Job Type: Full-time
Pay: $170,000.00 – $190,000.00 per year

Application Questions

What is your salary range?
Are you comfortable with working remotely in the US for 4-6 months prior to deploying to Germany for a minimum of 12 months?

View full job page →

Today

Principal Data Engineer

Oracle Nashville, TN

Nashville, TN

Today

Full time Senior Analytics Big Data

Apply Now

Job Description

The Principal Data Engineer role at Oracle, based onsite in Nashville, TN, leads Data Engineering, BI, and Analytics initiatives for OCI, combining hands-on data engineering with program leadership to design scalable data platforms and reporting solutions across Oracle Cloud Infrastructure.

RESPONSIBILITIES — DATA ENGINEERING & ANALYTICS LEADERSHIP

Design, build, and scale data pipelines that aggregate information from multiple OCI systems and services.
Develop robust data models, datasets, and reporting frameworks that provide actionable insights for engineering, operations, customer success, and executive leadership.
Architect and implement scalable analytics platforms that support strategic customer programs and operational decision-making.
Design and maintain enterprise-grade data solutions that improve visibility into customer adoption, operational health, service performance, and business outcomes.
Build and automate data ingestion, transformation, and reporting processes to reduce manual effort and improve data accuracy.
Establish data quality, governance, lineage, and observability standards across critical business datasets.
Partner with engineering teams to define telemetry, instrumentation, and data collection strategies.
Perform deep analysis of large and complex datasets to identify trends, opportunities, risks, and operational bottlenecks.
Drive adoption of modern data engineering best practices, tools, and technologies across the organization.

RESPONSIBILITIES — BUSINESS INTELLIGENCE & EXECUTIVE REPORTING

Design and deliver Business Intelligence solutions that provide actionable visibility into customer health, operational performance, and strategic business objectives.
Develop executive dashboards, scorecards, KPI frameworks, and reporting solutions used by senior leadership for decision-making.
Partner with business leaders to define success metrics, operational indicators, and reporting requirements.
Build scalable semantic models and reporting datasets that enable self-service analytics across multiple organizations.
Transform raw operational and engineering data into meaningful business insights and recommendations.
Standardize reporting methodologies and establish trusted sources of truth for key organizational metrics.
Support strategic planning, investment decisions, and customer engagement initiatives through data-driven analysis.

RESPONSIBILITIES — TECHNICAL PROGRAM MANAGEMENT & STRATEGIC EXECUTION

Lead large, complex, cross-functional initiatives spanning engineering, product, operations, and executive leadership teams.
Break down ambiguous business problems into actionable technical workstreams and measurable deliverables.
Develop functional specifications and drive successful execution from concept through delivery.
Identify process gaps and establish scalable mechanisms that improve organizational efficiency and execution.
Manage program schedules, dependencies, risks, and stakeholder communications.
Anticipate bottlenecks, proactively manage escalations, and balance technical constraints with business priorities.
Drive alignment across OCI organizations toward shared objectives and customer outcomes.
Lead interactions with cross-functional teams consisting of Engineers, Product Managers, Architects, Customer Success leaders, and Executive Leadership.
Thrive in a fast-paced, highly ambiguous environment while maintaining focus on delivering measurable business value.

RESPONSIBILITIES — DATA PROCESSING & PIPELINING

Mentors less experienced team members to identify data requirements and business objectives of a project or initiative.
Provides expertise on the design and participates in building of data infrastructure to optimize data processing from a variety of data sources.
Independently analyzes, designs, and troubleshoots data flows based on business needs.
Participates in architecture, performance, and security reviews of the technical solution.
Adjusts data collection processes that involve indexing and query optimizations for optimal performance.
Builds ETL pipelines to support efficient and scalable data collection and extraction.
Engages with and holds upstream and downstream teams accountable for predefined SLAs.
Manages relationships with data providers.

RESPONSIBILITIES — DATA GOVERNANCE

Independently designs and implements data governance policies and procedures for data handling to maintain consistency, integrity, accuracy, and reliability.
Leads the redaction of PII and PHI data to ensure privacy and security compliance.
Ensures minimal data collection and usage in accordance with data minimization principles.
Implements data security measures to protect data from unauthorized access or disclosure and escalates issues as needed.
Ensures data compliance with relevant laws, regulations, and industry standards.

RESPONSIBILITIES — DATA VALIDATION & QUALITY ASSURANCE

Contributes to the design and implementation of rigorous data validation and integrity checks to mitigate quality issues.
Mentors team members to define data annotation and labeling processes to ensure data quality.
Identifies opportunities for automation of data validation and governance processes.
Independently corrects deviations and non-conformance when identified.

RESPONSIBILITIES — DATA PIPELINE DESIGN

Leverages advanced knowledge of ETL processes to design, develop, and optimize automated, scalable data pipeline architectures.
Implements advanced data storage solutions to store processed data for analysis and access.
Mentors less experienced team members to manage day-to-day data pipeline and storage operations.

RESPONSIBILITIES — DATA SOLUTIONS ENGINEERING

Works independently and collaboratively in an agile environment to develop, maintain, and debug advanced data solutions that are scalable, efficient, cost-effective, and reliable.
Reviews runnable code and supports testing and debugging with junior team members.
Evaluates new technologies to enhance data solutions.
Enforces and documents code standards and guidance within the team.
Creates documentation for design decisions and gathers broader architectural feedback before implementing.
Gathers data and evidence to secure necessary approvals.

CORE RESPONSIBILITIES — PLANNING & EXECUTION

Manages and coordinates moderately complex tasks, ensuring timely completion and alignment with requirements for a moderately sized project.
Delegates, monitors, and prioritizes work across multiple projects, providing technical oversight and adapting plans as resources or timelines shift.

CORE RESPONSIBILITIES — COLLABORATION & PARTNERSHIP

Collaborates across the organization to align on expectations and achieve shared objectives.
Understands business leaders, stakeholders, and customers to ensure solutions meet their needs.
Fosters inclusivity by seeking diverse perspectives and ensuring others feel heard and respected.

CORE RESPONSIBILITIES — PROBLEM SOLVING

Identifies and addresses moderately complex issues by analyzing data to determine solutions.
Escalates unresolved or critical issues with a thorough assessment and suggests solutions.
Documents problem solving strategies and contributes to improvements.

CORE RESPONSIBILITIES — CONTINUOUS LEARNING

Pursues learning opportunities to expand knowledge and stay abreast of industry trends.
Seeks feedback and training to improve skills.
Coaches and mentors junior teammates, promoting knowledge sharing.

CORE RESPONSIBILITIES — CONTINUOUS IMPROVEMENT

Develops ideas and collaborates on process improvements across teams, evaluating impact for stakeholders.
Solicits feedback on alternative approaches for ongoing improvement.

CORE RESPONSIBILITIES — PERFORMANCE AND DEVELOPMENT

Contributes to the talent development pipeline by participating in candidate interviews and providing hiring recommendations.

REQUIREMENTS

BS degree or equivalent experience in Computer Science, Engineering, Information Systems, Data Science, or related field
7+ years of experience in Data Engineering, Analytics Engineering, Technical Program Management, Software Engineering, or related technical roles
Strong experience designing, building, and maintaining large-scale data pipelines, ETL/ELT frameworks, and cloud-based data platforms
Experience developing BI solutions, executive dashboards, KPI frameworks, and operational reporting systems
Advanced SQL skills and experience working with large-scale datasets
Experience with data modeling, data warehousing, analytics platforms, and reporting architectures
Strong understanding of cloud technologies, distributed systems, and software development lifecycles
Demonstrated ability to analyze complex datasets and translate findings into actionable business recommendations
Experience partnering with engineering, product, operations, and business stakeholders to define requirements and deliver scalable data solutions
Strong written and verbal communication skills across technical and executive audiences
Proven ability to lead large, cross-functional initiatives and drive execution across organizational boundaries
MS degree or equivalent experience in Computer Science, Data Engineering, Analytics, or related field
10+ years of experience in Data Engineering, Analytics Platforms, BI, Technical Program Management, or Software Development
Experience building enterprise-scale data lakes, data warehouses, and analytics platforms
Experience with cloud-native architectures, distributed systems, and OCI services
Experience with Spark, Kafka, Airflow, Databricks, Snowflake, BigQuery, OCI Data Flow, or similar platforms
Experience with Oracle Analytics Cloud (OAC), Tableau, Power BI, Looker, or comparable BI platforms
Experience implementing data governance, data quality, metadata management, and observability frameworks
Experience developing self-service analytics solutions and semantic data models
Experience working directly with large enterprise customers and strategic cloud initiatives

TECHNOLOGIES

Spark, Kafka, Airflow, Databricks, Snowflake, BigQuery, OCI Data Flow
Oracle Analytics Cloud (OAC), Tableau, Power BI, Looker
SQL, Oracle Cloud Infrastructure (OCI)

WHAT SUCCESS LOOKS LIKE

Trusted data platforms and BI solutions become the foundation for decision-making across OCI Strategic Customer Engineering.
Executive leaders have real-time visibility into customer outcomes, operational performance, and business health.
Manual reporting processes are automated and replaced with scalable, self-service analytics capabilities.
Strategic customer programs execute more effectively through improved data accessibility, insight generation, and operational transparency.
Cross-functional teams align around a common set of metrics, objectives, and business outcomes.
Data-driven insights directly influence customer success, operational excellence, and OCI growth initiatives.

View full job page →

1d ago

Data Engineer IV

Hudson Manpower Cincinnati, OH

Cincinnati, OH

1d ago

$68 - $70/hr Full time AWS Big Data Cloud

Apply Now

Job Description

What you get at this onsite Data Engineer IV position in Cincinnati, OH include a competitive base pay of USD 68 - 70 per hour, comprehensive medical, dental, and vision coverage, optional life and disability insurance, a 401(k) plan with company match and potential profit sharing, paid vacation time, paid bench time, a training allowance, and referral bonuses. You will join Hudson Manpower and focus on building scalable data pipelines that power machine learning models deployed in AWS SageMaker, collaborating within the Machine Learning Data Enablement squad of the Data Insights Tribe.

Benefits

Competitive base salary
Medical, dental, and vision insurance coverage
Optional life and disability insurance provided
401(k) with a company match and optional profit sharing
Paid vacation time
Paid Bench time
Training allowance offering
Referral bonuses

Responsibilities

Join the Machine Learning Data Enablement squad in the Data Insights Tribe.
Help build high quality, scalable data pipelines that power machine learning models across the enterprise, deployed in AWS SageMaker.
Deliver curated, production-ready datasets in collaboration with data scientists and ML engineers.
Work closely with data scientists and ML engineers to shape how machine learning data is delivered across the bank.
Maintain in-office presence: in office 4 days a week minimum (Monday-Thursday).
Help define how data powers machine learning at scale.

Requirements

Proficiency in Python
Proficiency in SQL
Location: Cincinnati, OH - onsite
USC and GC Holder candidates only
In-office 4 days a week minimum (Monday-Thursday)

Technologies

Python
SQL
AWS Sagemaker
DBT
Snowflake
Databricks

Top skills

Python
SQL
AWS Sagemaker
DBT
Snowflake

View full job page →

1d ago

Lead Data Engineer

INSPYR Solutions Doraville, GA 🇺🇸 Remote

Doraville, GA

1d ago

$60 - $70/hr Contract Artificial Intelligence Big Data Bigdata

Apply Now

Job Description

Lead Data Engineer at INSPYR Solutions will architect and optimize distributed data pipelines, establish Lakehouse architecture, enable ML workflows, ensure data quality, and improve production reliability.

Responsibilities

Architect, build, and optimize distributed data pipelines using Apache Spark in a high-volume, mission-critical environment.
Design and maintain enterprise Lakehouse infrastructure with Delta Lake, ensuring ACID compliance, lineage, auditability, and governance.
Develop automated ingestion frameworks (batch, streaming, and event-driven) across multiple cloud services and integration points.
Prepare feature-ready datasets and establish reproducible machine learning deployment patterns to enable ML workflows.
Lead platform-wide data quality, access control, and cataloging frameworks.
Implement cost optimization, cluster tuning, and performance engineering strategies.
Collaborate with Finance, BI, Operations, and ML teams to translate complex business needs into scalable data solutions.
Own production reliability, troubleshooting, and root cause analysis for data and ML pipelines.

Requirements

7+ years of experience in advanced data engineering with distributed compute technologies.
Expert Spark engineering experience including performance tuning, cluster configuration, partition strategies, and large dataset optimization.
Hands-on experience with Lakehouse architectures featuring ACID transactions, schema evolution, and governance frameworks.
Strong Python and SQL skills for large-scale data transformations.
Experience supporting machine learning pipelines or model operationalization.
Proven track record architecting cloud-native data platforms on Azure, AWS, or GCP.
Experience integrating diverse, complex data sources at enterprise scale.
Demonstrated ability to own mission-critical production systems.
Experience with distributed streaming frameworks such as Kafka or Event Hubs.
Experience building or supporting ML platforms, feature stores, or experiment tracking systems.
Background in data security, compliance controls, or audit-ready governance.
Experience automating data operations with CI/CD and infrastructure as code.