AI Data Engineer - AI Training data
Python
Artificial Intelligence
Azure Data Engineer
Azure Data Lake
Azure Machine Learning
Big Data
Data
Data Analysis
Data Analytics
Data Engineer
Data Integration
Data Integration Tools
Data Pipeline
Data Platform
Data Processing
Database
Databases
Engineer
ETL
Informatica
Information Technology (IT)
Software Development
SQL
Job Description
Blueprint Technologies is seeking an AI Data Engineer to help productionize training data pipelines that turn structured and unstructured sources into datasets ready for AI use. This role focuses on Python-based data processing, OCR and text workflows, automated quality checks, and reliable dataset management in Azure, working closely with data science and engineering teams.
What you’ll do
- Design, develop, maintain, and optimize Python-based data pipelines for large-scale ingestion, processing, and transformation.
- Build training datasets from raw inputs by collecting, organizing, cleaning, validating, and preparing structured and unstructured data.
- Create workflows for OCR processing, text extraction, normalization, filtering, segmentation, and deduplication.
- Run and monitor large-scale batch jobs, troubleshoot failures, and add automated retry and recovery mechanisms.
- Implement automated data quality checks to detect missing or corrupted files, incomplete outputs, duplicates, formatting issues, and metadata inconsistencies.
- Produce training-ready datasets using filtering, tokenization, text segmentation, and conversion into standardized formats.
- Manage bulk data uploads, downloads, and synchronization between local processing environments and Azure storage.
- Maintain organized datasets with consistent naming, metadata, version control, and source traceability.
- Optimize processing performance, memory usage, storage efficiency, and cloud resource consumption.
- Track dataset statistics, processing coverage, quality metrics, and exceptions, and generate operational reports.
- Collaborate with applied scientists and engineering teams to translate AI training requirements into scalable, repeatable data workflows.
- Maintain documented, tested, and version-controlled code and follow established data security and governance requirements.
Required qualifications
- 5+ years of professional experience in data engineering, data pipeline development, or data science with substantial hands-on programming responsibilities.
- Strong proficiency in Python, including reusable modules, debugging, testing, logging, and error handling.
- Experience building datasets from scratch, including ingestion, cleaning, transformation, validation, and preparation.
- Hands-on experience processing large-scale structured and unstructured datasets using memory-efficient, parallel, or distributed techniques.
- Practical experience with Azure Blob Storage or Azure Data Lake Storage, including bulk data transfers, storage organization, and access troubleshooting.
- Experience with Python data-processing libraries and formats: pandas, NumPy, JSON/JSONL, CSV, and Parquet.
- Working knowledge of SQL, Git, Linux, and command-line tools.
- Strong troubleshooting, analytical, and problem-solving skills focused on data quality and integrity.
- Ability to communicate technical challenges and collaborate with cross-functional engineering and data science teams.
Technologies
- Python, Azure Blob Storage, Azure Data Lake Storage
- pandas, NumPy, JSON, JSONL, CSV, Parquet
- SQL, Git, Linux
- OCR, tokenization
- Azure Databricks, Azure Machine Learning, Azure Data Factory
- Hugging Face Datasets, PySpark, CI/CD
Benefits
- Medical, dental, and vision coverage
- Flexible Spending Account
- 401k program
- Competitive PTO offerings
- Parental Leave
- Opportunities for professional growth and development
Preferred qualifications
- Experience developing OCR pipelines, processing scanned documents, multilingual text, or document-layout data.
- Experience with Azure Databricks, Azure Machine Learning, Azure Data Factory, or other workflow orchestration platforms.
- Familiarity with Hugging Face Datasets, AI/ML tokenization, near-duplicate detection, and training/evaluation data separation.
- Experience with large-scale dataset curation and preparation for AI or machine learning model training.
- Knowledge of distributed data processing frameworks, including PySpark.
- Experience optimizing data pipelines for performance, reliability, scalability, and cost efficiency.
- Experience with dataset lineage, reproducibility, containerized environments, and CI/CD practices.
Salary: USD 150,000 - 160,000 per year (Washington State anticipated range).
Location: Seattle, WA, 5 days onsite.
Applicant accommodations: If you need a reasonable accommodation to participate in any part of the application or interview process, contact [email protected].