Build reliable data foundations for property and casualty insurance while working with Azure Databricks lakehouse technology. Builders offers a collaborative, customer-centric environment backed by strong financial strength, including A (AM Best) and a culture recognized by Top Workplaces with Culture Excellence honors.
In this onsite role in Atlanta, GA, you will design, develop, and maintain data pipelines that support Builders’ migration from Guidewire and legacy systems into the Azure Databricks lakehouse. You will contribute to Unity Catalog governance, data quality, pipeline performance, and the long-term sustainability of pipelines originally built by contractors.
What you’ll do
- Build pipelines for phased migration from Guidewire (PolicyCenter, ClaimCenter, BillingCenter) and legacy systems into Azure Databricks, including historical loads, incremental/CDC (Change Data Capture), and reconciliation.
- Create bronze and silver layer transformations using PySpark and Spark SQL on Delta Lake following the medallion architecture.
- Support data modeling for property and casualty insurance, including policies, insured risks and exposures, coverages, endorements, premiums, claims, reserves, payments, and recoveries.
- Integrate third-party data sources such as geospatial and weather data.
- Schedule and run production pipelines using Databricks Workflows (Lakeflow Jobs) and/or Azure Data Factory, including dependencies, parameters, retries, alerting, and investigation of pipeline failures.
- Apply Unity Catalog controls defined by the Data Engineering Lead and Data Governance lead, including permissions, PII (Personally Identifiable Information) masking, tags, and lineage.
- Implement data quality checks, error handling, and monitoring.
- Tune pipeline performance and cost using techniques such as partitioning, liquid clustering, and right-sized compute.
- Use Git version control (GitHub or Bitbucket) and CI/CD with Databricks Asset Bundles to version, review, and deploy code across development, test, and production.
- Write documentation and runbooks, participate in code reviews, and gradually take ownership of contractor-built pipelines.
- Perform other duties as assigned.
What we’re looking for
- Associate degree or an equivalent combination of education and experience.
- Databricks engineering certification.
- 3+ years in data engineering, including 1+ years hands-on with Databricks (PySpark, Spark SQL, Delta Lake).
- Experience with property and casualty insurance data, ideally Guidewire, plus experience migrating data from legacy source systems.
- Knowledge of Python/PySpark and SQL.
- Knowledge of Unity Catalog: permissions, row/column-level security, and lineage.
- Experience with incremental loading patterns such as CDC, MERGE, SCD (Slowly Changing Dimension), and Auto Loader.
- Knowledge of data modeling, including dimensional modeling.
- Experience scheduling and monitoring pipelines with Databricks Workflows and/or Azure Data Factory.
- Experience with Git-based version control (GitHub or Bitbucket), including branching, pull request reviews, and Databricks Git folders.
- Experience with Azure data services such as ADLS Gen2, Azure Data Factory, and Key Vault.
- CI/CD fundamentals (Databricks Asset Bundles, Azure DevOps, GitHub Actions, or Bitbucket Pipelines).
- Experience with streaming (Structured Streaming, Event Hubs).
- Experience with API and integration tools such as MuleSoft is an added advantage.
Benefits at Builders
- Competitive Salary
- Bonus Structure
- Profit Sharing
- Medical, Dental, Vision Insurance
- Employer Paid Short Term Disability
- Employer Paid Long Term Disability
- Employer Paid Life Insurance
- Voluntary Life Insurance
- 401K with Company Match
- PTO
Technology stack
Azure Databricks, Delta Lake, Unity Catalog, Lakeflow, Databricks SQL, PySpark, Spark SQL, Databricks Workflows, Azure Data Factory, Git, GitHub, Bitbucket, Databricks Asset Bundles, Python, SQL, ADLS Gen2, Azure DevOps, GitHub Actions, Bitbucket Pipelines, Key Vault, Structured Streaming, Event Hubs, Auto Loader, MERGE, SCD, CDC, MuleSoft, API.
Builders Insurance Group is an Equal Opportunity Employer. We welcome applicants from all walks of life and do not discriminate based on any protected status.
