F
Lead Data Engineer / Architect
Artificial Intelligence
Big Data
Bigdata
Data
Data Architecture
Data Engineer
Data Engineering
Data Engineering Lead
Data Integration
Data Lakehouse
Data Pipeline
Data Processing
Databricks
Databricks Lakehouse
Databricks Lakehouse Platform
Lead Ai Engineer
Lead Data Engineering
Llm Operations
Programming Languages
Rag Architectures
Software Development
SQL
Job Description
The Lead Data Engineer / Architect role at Fasiokart is focused on hands-on production data engineering and the delivery of production AI and LLM capabilities, including LLM/RAG. The position is based in Cary, NC with a hybrid model and onsite expectations starting from Day 1.
Role Overview
Location: Cary, NC (hybrid)
Work model: Day 1 onsite; hybrid schedule with Tuesday–Thursday onsite
Employment: Full-time
Experience: 10–15 years (minimum 12 years)
Compensation: USD 145,000 - 160,000 per year
Responsibilities
- Design solutions, write production code, review code, and support data engineering teams through technical leadership.
- Design and deploy LLM-based solutions in production, including RAG, embeddings, retrieval, and prompt engineering.
- Design and implement lakehouse / medallion architecture using Bronze, Silver, and Gold layers at enterprise scale.
Requirements
- Production data engineering skills with Python, PySpark, Scala, and SQL.
- Strong Azure Databricks experience, including Delta Lake, Spark performance tuning, Jobs/Workflows, and Unity Catalog.
- Experience with Azure data services, including ADLS Gen2 and Azure Data Factory (ADF).
- Proven experience implementing lakehouse / medallion architecture with Bronze, Silver, and Gold layers at enterprise scale.
- Data pipeline engineering experience, including metadata-driven ingestion frameworks, data transformation, data quality, and large-scale pipeline optimization.
- Production AI/LLM experience delivering LLM-based solutions, including RAG, embeddings, retrieval, and prompt engineering.
- Experience with LLM frameworks: LangChain, LlamaIndex, or LangGraph, plus Azure OpenAI, OpenAI, or Databricks Model Serving.
- Azure DevOps / CI-CD experience for automating data pipeline deployments; Terraform and Databricks Asset Bundles are highly relevant.
- Financial services or insurance domain experience, especially financial reconciliation, general ledger, or accounting data.
- Experience with AI-driven data quality, anomaly detection, and automated reconciliation.
- Knowledge graphs, ontologies, semantic layers, or enterprise Text-to-SQL solutions.
- Azure security and governance experience, including PII protection and fine-grained access controls.
- MLOps/LLMOps experience, including model evaluation and production AI monitoring.
- This is a hands-on Lead Data Engineer position, not a purely managerial or coordination role, and candidates must demonstrate recent production coding experience.
- Willing and able to work onsite in Cary, NC from Day 1, Tuesday–Thursday.
Technologies
- Python, PySpark, Scala, SQL
- Azure Databricks, Delta Lake, Jobs/Workflows, Unity Catalog
- Azure Data Factory (ADF), ADLS Gen2
- Lakehouse / Medallion Architecture
- LangChain, LlamaIndex, LangGraph
- Azure OpenAI, OpenAI, Databricks Model Serving
- Azure DevOps, CI-CD, Terraform, Databricks Asset Bundles
- Text-to-SQL
- MLOps, LLMOps
Benefits
- Flexible schedule
Recruiter Screening Checklist
- Confirm 12–18 years of relevant experience.
- Confirm visa status and eligibility to meet the applicable compensation terms.
- If applicable, confirm an H-1B transfer candidate can start upon receipt of the transfer receipt notice.
- Confirm willingness and ability to work onsite in Cary, NC from Day 1, Tuesday–Thursday.
- Confirm strong hands-on Python, PySpark, Scala, and SQL.
- Confirm strong Azure Databricks, Delta Lake, ADLS Gen2, and ADF experience.
- Confirm proven enterprise lakehouse / medallion architecture experience.
- Confirm production experience building and deploying LLM/RAG solutions.
- Confirm recent production coding and technical leadership experience.
- Confirm compensation expectations before submission.
Candidate Priority
- Highest priority: Candidates who combine Azure Databricks and enterprise data engineering expertise with production LLM/RAG implementation experience.
- Secondary priority: Candidates with strong data platform engineering and production AI experience plus insurance/financial services domain knowledge and governance expertise.
- Not prioritized: Pure people managers, candidates without hands-on production AI/LLM experience, candidates lacking core Azure Databricks expertise, or candidates unable to meet mandatory onsite and compensation requirements.