Sciforium is seeking a Foundation Model Data Engineer to drive end-to-end dataset strategy, creation, and curation for foundation models.

Own end-to-end pre-training dataset creation for LLMs, including defining the optimal mix of web data, code, books, and technical papers for downstream performance.
Design and implement pipelines for data cleaning, exact and fuzzy deduplication, and high-quality signal extraction from petabytes of raw, unstructured data.
Lead development of post-training datasets, including SFT instructions, multi-turn dialogues, and preference modeling data for RLHF and DPO.
Drive acquisition and processing of vision and video data, addressing multimodal alignment, video compression, and temporal data consistency.
Build high-throughput data processing scripts in Python, using multiprocessing and multithreading to scale ingestion and transformation without bottlenecks.
Perform deep-dive statistical analysis on training corpora to surface biases, gaps in knowledge, and quality regressions, ensuring the model “diet” is mathematically balanced.
(Added Value) Create pipelines for high-reasoning synthetic data to fill gaps in natural datasets, leveraging existing models for labeling and refinement.

5+ years of industry experience in Data Science or Machine Learning, with a track record building and managing datasets for foundation models.
Expert-level proficiency in high-performance code, focused on multiprocessing, multithreading, and efficient memory management for large-scale data work.
Proven experience working with petabyte-scale datasets used to train production-grade LLMs or Large Vision Models.
Hands-on experience building massive LLM training sets from scratch, including raw web crawls (e.g., Common Crawl) and specialized domain data.
Practical experience creating datasets for RLHF, DPO, and multi-turn instruction following, including managing human-labeling workflows and quality gold-sets.
Mastery of data-at-scale tooling such as Spark, Ray, or high-performance data formats including WebDataset and Parquet.

Experience building large-scale image or video datasets from scratch (e.g., LAION-style pipelines).
Familiarity with large-scale crawling of multimodal data, including video processing, codecs, and compression.
Experience designing complex labeling schemas for reasoning, coding, and mathematical benchmarks.
A Master’s or PhD in a quantitative field with focus on data-centric AI or information retrieval.

Foundation Model Data Engineer

Job Description