DeveloperJobs.io
← Back to all jobs

Job Description

Build cloud data platforms and analytics pipelines using Azure Databricks with strong financial domain experience.

Responsibilities

  • Develop services, controls, and reusable patterns that help the team deliver value safely, quickly, and sustainably in public cloud environments
  • Design and develop highly scalable Azure cloud applications using microservices and Azure Functions
  • Enable secure public cloud operations by applying repeatable implementation patterns
  • Build and optimize ETL/ELT solutions on Azure Databricks using LakeBase and Spark
  • Develop and deploy large-scale data pipelines to support machine learning, insights generation, and business intelligence dashboards
  • Design, build, and manage modern data pipelines for data integration that support analytics and ML feature development
  • Write advanced and complex SQL, including performance tuning and query optimization
  • Build AI-based solutions to support business needs, process automation, and improved operational efficiency
  • Improve data reliability, data integrity, system efficiency, and overall data quality
  • Decompose new product capabilities into functional changes for verification and validation
  • Work effectively with business stakeholders and third-party technical organizations
  • Contribute to the architectural evolution of data engineering patterns, frameworks, systems, and platforms, including defining best practices and standards
  • Mentor other data engineers by providing technical direction and teaching teams how to leverage cloud data platforms
  • Gather and analyze data to support decision-making and provide detailed, realistic estimates
  • Promote high-performance, collaborative technical execution to maintain quality output

Requirements

  • 7+ years of experience with Python and Scala
  • 5+ years with Apache Spark including PySpark and Spark SQL
  • 3+ years of DevOps automation using Terraform
  • 3+ years building and managing API lifecycles and designs (e.g., GraphQL, REST)
  • 3+ years building and deploying cloud-based solutions on Azure Databricks with UC, Snowflake, Azure Functions, and Service Bus
  • 5+ years with SQL, including designing complex data schemas and query performance optimization
  • Experience building LLM integrations for workflow automation and business needs
  • Experience with CI/CD and tools including GitHub Actions, GIT, Artifactory, and Sonar

Technologies

  • Python, Scala
  • Apache Spark, PySpark, Spark SQL
  • Terraform
  • GraphQL, REST
  • Azure Databricks (UC), Snowflake
  • Azure Functions, Service Bus
  • SQL
  • LLM integrations
  • CI/CD, GitHub Actions, GIT, Artifactory, Sonar
  • Microservices
  • LakeBase, ETL/ELT, AI/ML

Compensation

  • $70.00 - $80.00 per hour

Schedule

  • 40.0 hours per week

Location

  • Eden Prairie, MN (onsite)

Minimum experience: 3 years

Similar Jobs