Senior Data Engineer
Senior
Apache Airflow
Apache Beam
Azure
Big Data
Bigdata
Cloud
Cloud Data Engineering
Cloud Data Platform
Cloud Engineer
Cloud Native
Cloud Operations
Cloud Platform
Cloud Platforms
Cloud Platforms Cloud Platforms
Data
Data Analysis
Data Architecture
Data Engineer
Data Engineering
Data Engineering Lead
Data Integration
Data Pipeline
Data Pipelines
Data Platform
Data Processing
Data Warehouse
Database
ETL
Google Cloud
Lead Data Engineering
Software Development
Streaming Data
Job Description
Own the design and evolution of a biomedical data platform built for reliable, scalable scientific and ML workflows.
Responsibilities
- Architect and lead end-to-end data systems for large-scale biomedical datasets (chemical, biological, toxicology, omics, assay, clinical, and experimental data).
- Define and evolve schema-driven data models that reconcile noisy, semi-structured, and heterogeneous sources into coherent, interoperable representations.
- Set best practices for data quality, validation, provenance, lineage, and versioning aligned to scientific and ML workflows.
- Build and maintain cloud-native data infrastructure, including data lakes, warehouses, object storage, and streaming systems, with a focus on scalability and reliability.
- Design pipelines that support both batch and streaming access for ML training, evaluation, and inference.
- Collaborate with ML engineers, scientists, and product leads to translate research needs into durable data abstractions.
- Make principled trade-offs across performance, cost, flexibility, and correctness for production systems.
- Provide technical leadership through design reviews, architectural guidance, and mentorship for other engineers.
- Proactively identify and address systemic risks in data integrity, scalability, and operational complexity.
Requirements
- 5+ years of experience in data engineering, platform engineering, or ML infrastructure roles, with clear ownership of production systems.
- Proven experience designing and operating large-scale, production-grade data pipelines.
- Strong proficiency in Python and data-centric software engineering practices.
- Deep experience with cloud platforms: AWS, GCP, or Azure, including storage, compute, and security primitives.
- Familiarity with distributed data processing and orchestration systems such as Spark, Beam, Ray, Airflow, and Dagster.
- Experience supporting ML/AI workloads, including dataset generation, feature pipelines, and reproducible training workflows.
- Strong architectural judgment and the ability to communicate technical decisions across disciplines.
Technologies
- Python
- AWS
- GCP
- Azure
- Spark
- Beam
- Ray
- Airflow
- Dagster
Benefits
- Flexible remote or hybrid work arrangements.
- Deeply technical, low-ego culture focused on learning and rigor.
Bonus if you have
- Competitive compensation with meaningful equity participation.
- Prior work with biomedical or life-science data (e.g., omics, assays, molecular representations, clinical or toxicology data).
- Experience with streaming platforms (Kafka, Pub/Sub, Kinesis).
- Exposure to ontology-aware data modeling or schema evolution in scientific domains.
- Infrastructure-as-code and systems experience (Terraform, Docker, Kubernetes).
- Experience in early-stage startups or research-heavy environments.
- Open-source contributions or technical publications.
What you’ll focus on
- Think in systems and interfaces, not just pipelines.
- Own ambiguous problems and converge on robust solutions through iterative design.
- Apply schema design, normalization, and semantics as first-order engineering problems for biomedical data and AI-driven settings.
How to apply
- Submit your resume and a short note on why this role resonates with you.
- GitHub, technical writing, or relevant project links encouraged.
- Compensation Range: $115K - $200K