Bigquery

Bigquery Developer Jobs

34 jobs found

1d ago

Data Engineer II

Fisher Dynamics Saint Clair Shores, MI

Saint Clair Shores, MI

1d ago

Full time Apache Airflow Big Data Bigdata

Apply Now

Job Description

Fisher Dynamics is looking for a Data Engineer II to help power its ERP platform and embedded AI and LLM capabilities. In this onsite role in Saint Clair Shores, MI, you will design and deliver scalable, reliable data pipelines and cloud data infrastructure, supporting real-time streaming and feature pipelines, data governance, and data migration and integrations. You will partner closely with ML/AI engineers, software engineering, and business stakeholders to ensure trusted data foundations for operational and analytical use.

Responsibilities

Design and build scalable, fault-tolerant data pipelines for ERP data ingestion, transformation, and loading.
Implement ETL/ELT workflows to migrate legacy ERP data into the new ERP system, including data validation and quality checks.
Build real-time streaming pipelines using Kafka, Spark, or similar technologies for continuous data flow.
Develop batch processing jobs for scheduled transformations and aggregations.
Ensure pipelines support large volumes, complex transformations, and operational resilience.
Establish data governance policies, standards, and procedures for ERP data.
Create monitoring and validation frameworks for data accuracy and consistency, including data profiling, cleansing, and quality tooling.
Document data lineage, metadata, and data dictionaries to support transparency and compliance.
Monitor data quality metrics and SLAs, alert on issues, and drive resolution.
Design and implement cloud-based data architecture on AWS, GCP, or Azure (data warehouses, data lakes, and related components).
Build and optimize storage solutions for ERP transactional and analytical data.
Implement data security controls, including encryption and access controls for sensitive financial and operational data.
Optimize infrastructure for performance, cost, and scalability, and troubleshoot pipeline and platform issues.
Collaborate with ML/AI engineers on feature requirements and data needs, including feature stores and feature pipelines for model training and inference.
Engineer ML-ready features from raw ERP inputs (transactions, master data, time-series), and support real-time feature serving for low-latency inference.
Support exploratory data analysis and data debugging for ML/AI initiatives.
Lead data migration from Plex to the new ERP system with validation and reconciliation.
Build integrations with external sources (suppliers, customers, market data), including synchronization and consistency checks.
Manage historical data and archive strategies, support data cutover activities, and perform validation.
Conduct load testing and capacity planning for data infrastructure.

Requirements

Bachelor’s degree in Computer Science, Data Science, Engineering, or a related field.
4-6 years of professional data engineering experience building production data systems.
Master’s degree preferred.
Demonstrated experience designing and implementing large-scale ETL/ELT pipelines and required data architectures.
ERP system data integration or data warehousing experience strongly preferred.
Advanced proficiency in Python, Scala, Java, or similar data engineering languages.
Expert-level SQL and relational/dimensional database design.
Strong experience with orchestration tools such as Airflow, Prefect, Dagster.
Expertise with cloud data platforms including AWS Redshift/S3, Google BigQuery, Azure Data Lake.
Experience with big data technologies such as Spark, Hadoop, Kafka, Flink.
Knowledge of data warehousing, data lakes, and data architecture patterns, including data quality practices.
Experience with version control (Git) and data pipeline version management.
Proficiency with containerization (Docker) and orchestration platforms.
Understanding of data governance, security, and compliance requirements.
Feature store and ML data pipeline experience is preferred, along with familiarity with ERP systems and business data models.
Strong problem-solving, debugging, and collaboration skills, including clear communication with data scientists and engineers.

Benefits

401(k) and 401(k) matching
Dental insurance, Vision insurance, Health insurance, and Life insurance
Health savings account and Flexible spending account
Employee assistance program
Employee discount
Flexible schedule
Paid time off
Professional development assistance and tuition reimbursement

Work Environment

This position is in-person. The working environment and physical requirements are typical of an office setting and manufacturing environment. The role requires collaboration with technical teams and business stakeholders.

Physical Demands

Ability to lift 40 lbs.

Technologies

Python, Scala, Java, SQL, Airflow, Prefect, Dagster, AWS, AWS Redshift, S3, Google BigQuery, Azure, Azure Data Lake, Spark, Hadoop, Kafka, Flink, Docker, Git

View full job page →

2d ago

Machine Learning Engineer

XPO Boston, MA

Boston, MA

2d ago

$100k - $120k/yr Full time Artificial Intelligence Automation Bigquery

Apply Now

Job Description

In this hybrid role in Boston, MA, XPO is hiring a Machine Learning Engineer to build and maintain the systems that make ML programs reliable in real production settings. You will focus on data preparation and validation tooling, ML infrastructure for training and deployment, and MLOps capabilities such as CI/CD, monitoring, and feedback loops.

This position supports the full path from experimentation to deployment by partnering with applied and data scientists and coordinating with data engineering to ensure dependable, accessible data pipelines.

What you’ll work on

Build and maintain data preparation and validation tooling to ensure high-quality inputs for ML and optimization models
Design and implement ML infrastructure for model training, evaluation, and deployment
Create and maintain CI/CD pipelines for machine learning models, including automated testing and validation
Implement model monitoring, drift detection, and feedback loops to track performance in production
Partner with applied and data scientists to productionize models and streamline the path from experimentation to deployment
Collaborate with data engineering teams to ensure reliable, accessible data pipelines
Contribute to shared MLOps tooling and best practices across the AI/ML organization

Minimum qualifications

Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent related work or military experience)
1 year of experience in software or machine learning engineering, including hands-on experience building data pipelines, ML infrastructure, or MLOps tooling
Experience developing data preparation, validation, or quality-checking tooling for machine learning pipelines
Proficiency in Python and SQL
Experience with cloud data or ML platforms, such as AWS, GCP, or BigQuery
Strong collaboration skills working with data science/applied science teams and data engineering teams
Master’s degree in Computer Science or related field
3+ years of experience building ML infrastructure for training, evaluation, and deployment at scale
Experience building and maintaining CI/CD pipelines for machine learning models
Experience with model serving and inference infrastructure (batch and real-time)
Experience implementing model monitoring, drift detection, and feedback-loop tooling
Experience with containerization and orchestration (Docker, Kubernetes)
Experience partnering with data engineering teams on data pipeline reliability and access

Technologies

Python
SQL
AWS
GCP
BigQuery
Docker
Kubernetes

Compensation and benefits

Annual salary range: $100,000 to $120,000 (actual compensation may vary based on experience and skill set)
Full health insurance benefits available on day one
Life and disability insurance
Earn up to 15 days of PTO over your first year
9 paid company holidays
401(k) option with company match
Education assistance
Opportunity to participate in a company incentive plan
This is an incentive-based position, which may include bonuses, incentive or commission plans

Location

Boston, MA (hybrid)

Education

Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent related work or military experience).

View full job page →

3d ago

Full Stack Developer — Backend and Data Platform Focus

Advanced Management USA LLC Remote 🌍 Remote

Remote

3d ago

$70k+/yr Full time Backend Developer Python APIs

Apply Now

Job Description

This is a hands-on individual-contributor position centered on backend engineering for a Python and SQL data platform that powers patient care-gap work for accountable care organizations and payors. Work is primarily in the data pipeline and database layers, with frontend and browser-extension tasks treated as secondary.

Role Overview

The platform reads CMS assignment files, Part A and Part B claims, Medicare eligibility data, and live hospital ADT and FHIR feeds from BigQuery. Scheduled Databricks jobs generate and manage patient care-gap items, and results are written to a PostgreSQL (Neon) database. A Next.js application and a Manifest V3 Chrome extension use the data for downstream workflows.

Correctness and auditability are core priorities. Records are append-only and fully audited, and automated runs must not overwrite work already completed by a human. Join correctness directly affects which patients are targeted, so engineering judgment focuses on blast radius, idempotency, and proof of what each script changes.

Responsibilities

Own the opportunity-generation pipeline.
Extend and operate a Python pipeline that transforms Medicare claims, eligibility data, and hospital ADT feeds into prioritized, workable care-gap items for a portfolio of ACOs and commercial payors.
Write heavy SQL against BigQuery, including correct large-scale joins and aggregation.
Query and tune multi-million-row datasets, including assignment files, Part A and B claims and line items, eligibility responses, FHIR encounter and ADT feeds, plus deduplication and late-arriving-data handling.
Build and run Databricks jobs.
Package pipeline scripts as scheduled jobs (Python script tasks, cluster and dependency configuration, DBFS, secret resolution), and run controlled one-off production jobs for remediation.
Design and migrate PostgreSQL and manage schema, indexing, and query performance on Neon.
Use versioned SQL migrations, staging and historical mirror tables, JSONB metadata columns, read-replica usage for batch reads, and database-enforced immutability.
Ensure generators are idempotent and safe to re-run.
Implement deterministic record IDs, upsert paths that preserve human work, and supersede or carry-forward logic across reporting periods and status lifecycles.
Maintain an append-only audit log behind every state change.
Protect production data through dry-run-then-commit scripts, volume abort thresholds, environment guards, backups, and rollback artifacts, plus remediation that repairs affected rows without destroying history.
Extend automated data-quality monitoring with invariant and drift checks on live production data, alerting, and anomaly investigation through written root cause documentation.
Test pipeline logic with pytest, ephemeral PostgreSQL fixtures, frozen clocks, and pre-commit gates to prevent regressions in nightly runs.
Publish and defend data contracts by documenting record contents and field-level constraints so the application team can build against them, coordinating breaking changes across repositories.
Move into application-layer work when needed, including Next.js, React, and TypeScript surfaces, REST APIs with role-based access control and HIPAA-compliant audit logging, Firebase Cloud Functions and other serverless services, and the Manifest V3 Chrome extension integrating with web-based EHR systems.
Debug end-to-end by tracing incorrect application outputs back through PostgreSQL, the generator that produced the records, the Databricks run source, and upstream BigQuery feeds.
Collaborate across a small team and two repositories, including code review and coordination with application and clinical-operations stakeholders on data needs.
Handle Protected Health Information responsibly with attention to performance, reliability, scalability, and security across every layer.

Required Qualifications

4+ years of professional software development experience.
Strong Python for production data work, including pandas, with scheduled batch jobs and command-line tools (not notebook-only experience).
Advanced SQL including CTEs, window functions, large multi-table joins, aggregation correctness, and query optimization on large datasets.
Strong PostgreSQL skills: schema design, indexing, EXPLAIN-driven tuning, transactions, upserts, and JSONB.
Experience writing and applying SQL migrations against a live production PostgreSQL database.
Experience with a cloud data warehouse, with BigQuery strongly preferred, including partitioning and query cost/performance awareness.
Experience building and operating scheduled data pipelines, including idempotent re-runs, backfills, duplicate and late-arriving sources, and reconciliation between systems.
Experience with Databricks or Spark, including jobs, clusters, dependency management, secrets, and DBFS, or a comparable managed job platform.
Operational judgment with production data: dry runs, blast-radius estimates, auditability, and rollback planning before writing changes.
Ability to debug complex issues across warehouse, batch jobs, databases, and application components.
Working ability in TypeScript with React and Next.js to ship frontend or API changes when required.
Experience designing and building REST APIs.
Comfort with Git and pull-request workflows, including giving and receiving code review.
Clear written communication to explain data defects, causes, and fixes in actionable documentation.
Strong problem-solving skills and attention to detail.

Technology Stack

Python, SQL, pandas
Databricks, BigQuery, Spark
PostgreSQL, Neon, JSONB
Next.js, React, TypeScript
Chrome extension (Manifest V3)
FHIR
REST APIs
Firebase Cloud Functions, serverless services
pytest, Git

Role Level

This is a hands-on individual-contributor role and is not suited for entry level or new graduate candidates.

Location and Compensation

Location: Remote
Salary: USD 70,000 per year
Minimum Experience: 4 years

Benefits

401(k) 6% Match
Bereavement leave
Dental insurance
Family leave
Flexible spending account
Health insurance
Health savings account
Life insurance
Paid parental leave
Parental leave
Vision insurance
401(k) matching
Employee discount
Paid time off
Professional development assistance
Referral program
Tuition reimbursement

Nice to Have

Healthcare data experience: CMS Part A and Part B claims, Medicare beneficiary identifiers (MBI), HL7 v2 ADT feeds, FHIR, or eligibility services such as HETS.
Value-based care domain knowledge: MSSP or ACO programs, risk adjustment and HCC coding, Annual Wellness Visits, transitional care management, or quality measures.
Experience with healthcare technology, including EHR integrations (eClinicalWorks, Athena, Epic, or similar).
Experience with Firebase Cloud Functions, Firestore, and Google Cloud Platform (IAM, service accounts, Secret Manager).
Experience with Prisma ORM.
Experience developing Chrome Extensions using Manifest V3.
Data quality and observability work: anomaly detection, invariant checks, drift monitoring, and alerting.
Experience integrating LLMs such as Claude, OpenAI, or Gemini.
Experience building speech to text or AI powered workflows.
Experience building internal operator tooling for running, configuring, and monitoring pipeline jobs.
Familiarity with modern Python tooling (uv, pytest plugins, pre-commit hook managers) and multi-package repositories.
Familiarity with Rust for performance-sensitive components, browser tooling, or systems programming.
Familiarity with HIPAA compliance and secure handling of healthcare data.

Application Questions

Have you worked on software handling PHI or in a HIPAA-regulated environment?
Describe a production data pipeline you owned end to end: the data sources, the schedule, the volume, and what you did the first time it produced bad data.
What is your hands-on experience with BigQuery or another cloud data warehouse, and with Databricks or Spark?
How comfortable are you writing and applying SQL migrations against a live production PostgreSQL database, and what precautions do you take?

View full job page →

8d ago

Quantitative Data Engineer - Fixed Income and Mortgages

Selby Jennings New York, NY

New York, NY

8d ago

$350k - $450k/yr Full time 3d Design Tools Big Data Bigdata

Apply Now

Job Description

This hands-on Quantitative Data Engineering role supports end-to-end data workflows for loan-level and structured credit modeling, with a focus on building scalable pipelines and production-ready datasets used in quantitative investment and risk analytics.

Responsibilities

Design and maintain large-scale data pipelines for credit, mortgage, and structured product analytics.
Develop and optimize loan-level feature engineering workflows and model input datasets.
Create reproducible data processing frameworks that support research, validation, and production deployment.
Collaborate with quantitative researchers to implement new features, validate methodologies, and improve model performance.
Partner with engineering teams to productionize research outputs and enhance platform scalability and reliability.
Support ad hoc quantitative analysis and investigate portfolio, collateral, and performance datasets.

Requirements

Strong Python development experience, including production-quality code, testing, packaging, and code review practices.
Deep experience with distributed data processing using Spark and PySpark, including optimization of joins, partitioning, caching, skew management, and execution performance.
Advanced SQL skills, including querying large columnar data warehouses such as Snowflake, Redshift, BigQuery, Vertica, or similar platforms.
Experience building analytical datasets and feature engineering workflows for machine learning, statistical modeling, or quantitative research.
Strong understanding of reproducible data pipelines, experiment tracking, artifact management, and version-controlled development.
Experience working in shared engineering environments using Git, automated testing, and CI/CD processes.
Ability to work directly with quantitative researchers and convert research requirements into scalable engineering solutions.

Technologies

Python
Spark, PySpark
SQL
Snowflake, Redshift, BigQuery, Vertica
Git, CI/CD

Preferred Qualifications

Experience with loan-level, mortgage, consumer credit, or structured finance datasets.
Exposure to prepayment, default, transition, or loss modeling in credit or securitized products.
Familiarity with market and reference data providers, securitization cash flows, collateral reporting, or structured product analytics.
Experience with Databricks, Delta Lake, workflow orchestration tools, and modern cloud-based analytics platforms.
Exposure to model deployment, scoring frameworks, experiment tracking, or machine learning operations.
Experience with high-performance analytics tools such as Polars, DuckDB, Pandas, and scikit-learn.
Familiarity with workflow scheduling, data quality monitoring, and pipeline validation.
Comfortable using AI-assisted development tools to accelerate coding, refactoring, testing, and codebase navigation.
Knowledge of cloud infrastructure, object storage, access controls, and cost-efficient data architecture.

Education

Bachelor’s, Master’s, or PhD in Computer Science, Data Science, Statistics, Financial Engineering, Mathematics, or Economic

Location and Compensation

Location: New York, NY (onsite)

Salary: USD 350,000 - 450,000 per yearly

View full job page →

13d ago

Software Engineer, Monetization

Unity Technologies SF Burbank, CA

Burbank, CA

13d ago

$188k - $188k/yr Full time Backend Developer Ad Tech APIs

Apply Now

Job Description

Unity Technologies SF is seeking a Software Engineer, Monetization in Burbank, CA for a hybrid role. You will help build and maintain Unity’s Monetization platform, products, and services, with the goal of enabling developers to grow their businesses through advertising and in-app purchases. The work emphasizes scalability, performance, large data volumes, and applying machine learning to meet system needs.

What you’ll do

Design, implement, test, review, and maintain Unity’s Monetization platform, products, and services to support developer monetization via advertising and in-app purchases.
Write high-quality, scalable code to support current and future product functionality and drive successful delivery.
Address system scalability and performance, process large volumes of data and functional requirements, and move toward the use of machine learning.
Manage individual project priorities, deadlines, and deliverables.
Translate business requirements into engineering features.

What you’ll bring

Master’s degree (or foreign equivalent) in Computer Science, Information Systems, Information Technology, or a related field, plus three (3) years of related work experience.
Experience designing APIs, backend services, and systems.
Ability to implement and review code in Golang, Nodejs, Typescript, and Python.
Hands-on work with SQL, NoSQL, and key-value data stores such as Postgres, MongoDB, and Redis.
Experience with Bigquery or other data warehousing solutions.
Experience with streaming platforms and queues such as Kafka and Nats.
Use of Terraform to manage cloud infrastructure.
Creating Python Notebooks to analyze datasets and generate useful insights.
Coordination with product to deliver product outcomes end to end.

Technologies

Golang, Nodejs, Typescript, Python
SQL, NoSQL, Postgres, MongoDB, Redis
Bigquery, Kafka, Nats
Terraform, Python Notebooks, Machine learning

Compensation and schedule

Location: Burbank, CA (hybrid)
Salary: USD 187,574 per year
Schedule: 40 hrs/week, Mon-Fri, 8:30 a.m. to 5:30 p.m.
Telecommuting: permitted on a hybrid schedule as determined by the employer

Benefits

Comprehensive health, life, and disability insurance
Commute subsidy
Employee stock ownership
Competitive retirement/pension plans
Generous vacation and personal days
Support for new parents through leave and family-care programs
Office food snacks
Mental Health and Wellbeing programs and support
Employee Resource Groups
Global Employee Assistance Program
Training and development programs
Volunteering and donation matching program
Equity awards and participation in company incentive plans (such as annual discretionary bonuses or sales commissions)

Additional information

International relocation support is not available for this position.
Work visa/immigration sponsorship is not available for this position.
Salary listed as $187,574.00/yr.

View full job page →

14d ago

Sr. Software Engineer

Viant Technology Irvine, CA

Irvine, CA

14d ago

$150k - $180k/yr Full time Backend Developer Senior APIs

Apply Now

Job Description

Viant Technology’s Potens team builds internal platforms that power workflow orchestration and large-scale data exploration, and the team is expanding AI-powered capabilities. In this onsite role in Irvine, CA, you’ll help evolve Magnus and Goliath with a hands-on, full-stack approach across backend services, data platforms, and production AI integrations.

You’ll collaborate on modernization work for established systems while developing new capabilities for how users and authorized AI agents create, manage, and execute workflows and explore BigQuery data.

What you’ll do

Maintain and modernize Magnus Java applications, REST APIs, and Goliath JavaScript/jQuery frontend
Develop and support backend capabilities using Java and Go
Build production AI features and integrations using LLMs, tool calling, guardrails, evaluation, and MCP
Integrate with BigQuery, MySQL, GCP services, and internal APIs
Diagnose and resolve application, workflow, database, performance, and production issues
Build automated tests and deliver reliable, production-quality software
Own projects end to end, from technical proposal and implementation through deployment and production support
Participate in code reviews and contribute to sound engineering practices across the team

How Magnus and Goliath work

Magnus is a visual workflow orchestration platform supporting BigQuery, Cloud Storage, MySQL, REST APIs, email, FTP, conditional routing, parallel execution, Go and JavaScript scripting, YAML workflow definitions, debugging, audit history, and access controls.

Goliath is a lightweight, high-performance BigQuery development interface for writing and executing SQL, browsing datasets and tables, visualizing results, and managing cost and quota allocation without requiring direct access to the GCP Console.

Benefits

Fully paid health insurance
Paid parental leave
Unlimited PTO

Technologies you may work with

Java, Go, JavaScript, jQuery
BigQuery, MySQL, GCP, REST APIs
LLMs, tool calling, guardrails, evaluation, MCP (including MCP server/client development)
Docker, Kubernetes, GKE, CI/CD pipelines, GitHub Actions
Python, Java 8, Tomcat, JSP, CodeMirror, Chart.js, YAML
Cloud Storage, REST, FTP, Email
MySQL performance optimization, GCP Console

Compensation: USD 150,000 - 180,000 per year.

Great to have

MCP server/client development and AI agent or tool-calling architectures
Go development in production environments
GCP and BigQuery
Docker, Kubernetes, or GKE
MySQL performance optimization
CI/CD pipelines, including GitHub Actions
Python development
Java 8, Tomcat, JSP, CodeMirror, or Chart.js
Incremental modernization of mature applications while maintaining production stability

View full job page →

15d ago

Senior Data Engineer

Brooksource Kansas City, MO

Kansas City, MO

15d ago

$55 - $60/hr Contract Senior Azure Big Data

Apply Now

Job Description

Brooksource is seeking a Senior Data Engineer for a 12-month contract role based onsite in Kansas City, MO. In this position, you will design and build dependable data pipelines and turn raw inputs into clean, consumable datasets that support enterprise analytics and reporting.

The work is hands-on across the full data lifecycle, including ETL and ELT development, data modeling, data quality and lineage, governance, and operational support. You will also collaborate with business and technical teams to translate data needs into efficient engineering solutions.

What you’ll do

Design, develop, and maintain scalable, reliable data pipelines and platforms for enterprise analytics and reporting.
Work with business and technical teams to understand data requirements and convert them into data engineering solutions.
Build data ingestion frameworks using Kafka Confluent and integrate data from multiple cloud and on-premises sources.
Implement ELT and ETL pipelines using PySpark, SQL, Dataform, DBT, and Dataproc to transform and process data efficiently.
Design and manage data models, schemas, and data warehousing structures optimized for performance and scalability.
Maintain data quality, consistency, and integrity through validation and monitoring mechanisms.
Apply data governance practices covering data lineage, metadata management, privacy, and compliance.
Create automated monitoring and alerting to support reliability, availability, and performance of pipelines.
Partner with cross-functional teams to enable downstream reporting, analytics, and AI/ML model consumption.
Use Git and infrastructure-as-code practices with documentation and CI/CD workflows.
Provide operational support for data pipelines and proactively troubleshoot performance issues.
Optimize data processing, query execution, and resource utilization in Azure environments.
Develop and maintain RESTful API integrations to support ingestion and synchronization with web applications and enterprise systems.
Support lightweight web application development efforts where data pipelines interface with front-end or full-stack components.

What you bring

Bachelor Degree in Computer Science or related field with 8 years of related professional experience (required).
Associate Degree with 10 years of related professional experience (required).
High School Diploma/GED with 12 years of related professional experience (required).
Strong programming skills in Python and PySpark for large-scale data processing.
Proficiency in SQL for data manipulation, analysis, and performance tuning.
Experience with Dataform, Dataproc, and BigQuery for pipeline development and orchestration.
Understanding of data governance principles and implementation practices.
Experience using Git for version control and CI/CD pipelines.
Experience building and consuming RESTful APIs, including enterprise integrations such as OAuth2, OpenID Connect, webhooks, pagination, rate limiting, retry logic, and error handling.
Some hands-on web application development experience (full-stack or front-end).