Data Engineer, Applied AI Solutions
Amazon Web Services
Apache Airflow
Artificial Intelligence
Automation
AWS
Aws Bedrock
Aws Data Warehouse
Aws Glue
Aws Sagemaker
Big Data
Bigdata
Cloud
Cloud Computing
Cloud Data Warehouse
Cloud Data Warehouse
Data Architecture
Data Engineer
Data Integration
Data Lake
Data Pipeline
Data Platform
Data Processing
Data Warehouse
ETL
Spark
SQL
Workflow Orchestration
Job Description
Build and run next-generation data infrastructure for applied AI solutions, with support for both human analytics and autonomous AI systems.
Key responsibilities
- Design, build, and maintain next-generation data infrastructure that serves both human analysts and AI systems.
- Partner across data science, engineering, analytics, and business stakeholders to deliver innovative, scalable solutions using organizational data assets.
- Maintain dataset accuracy, consistency, and observability by implementing monitoring, lineage tracking, and self-healing mechanisms for data quality at scale.
- Deliver infrastructure interfaces, documentation, and metadata for analysts/scientists and autonomous AI agents.
- Support an evolving data infrastructure roadmap that anticipates AI system needs while maintaining business reliability and performance.
- Help shape technical direction for data systems that support the organization’s AI transformation journey.
- Build and operate production pipelines and warehouses.
- Develop machine-consumable data infrastructure for AI systems and autonomous agents, including metadata and documentation.
- Implement GenAI data patterns end to end: chunking, embeddings, and vector stores for retrieval-augmented generation (RAG).
- Build and maintain datasets and feature pipelines for ML/GenAI training, fine-tuning, and inference using Amazon SageMaker, Amazon Bedrock, or equivalent.
- Implement data quality, lineage, and observability required by AI workloads, including validation, freshness or anomaly monitoring, and alerting at scale.
- Create analytical, operational, and AI/retrieval data models and schemas.
- Use workflow orchestration tools such as Step Functions, Airflow, or Glue Workflows.
Requirements
- 3+ years of data engineering experience.
- 1+ years of developing and operating large-scale data structures for business intelligence analytics, including data modeling experience.
- Bachelor’s degree (or foreign equivalent) in Computer Science, Engineering, Information Systems, Mathematics, or a related field.
- Experience with data modeling, warehousing, and building ETL pipelines.
- Experience building data infrastructure for AI systems and autonomous agents, including machine-consumable interfaces, metadata, and documentation.
Technologies
- Python, Scala, Java
- SQL (advanced; performance tuning at scale)
- Amazon SageMaker, Amazon Bedrock
- Amazon Glue
- Amazon EMR, Spark
- Amazon S3, Amazon Athena
- Amazon Redshift
- Step Functions, Airflow, Glue Workflows
- Kinesis, FireHose, Lambda
- IAM roles and permissions
- Vector stores, retrieval-augmented generation, chunking, embeddings
Benefits
- Sign-on payments
- Restricted stock units (RSUs)
- Health insurance (medical, dental, vision, prescription)
- Basic Life & AD&D insurance and option for Supplemental life plans
- EAP
- Mental Health Support
- Medical Advice Line
- Flexible Spending Accounts
- Adoption and Surrogacy Reimbursement
- 401(k) matching
- Paid time off
- Parental leave
Similar Jobs
C