Lead Data Engineer
Manager
Artificial Intelligence
Data
Data Analysis
Data Analytics
Data Architecture
Data Engineer
Data Integration
Data Pipeline
Data Platform
Data Processing
Data Warehouse
Database
Databases
ETL
Informatica
Integration
Programming
Programming Language
Programming Languages
Reporting and Analytics
SQL
Job Description
Lead enterprise-scale, AI-ready data solutions for Simpalm’s client as an on-site Lead Data Engineer in Dallas, TX.
Responsibilities
- Lead analysis, design, development, and implementation of scalable data solutions across the enterprise data warehouse and modern data platform
- Collaborate with Product Engineers, Data Engineers, Architects, Data Scientists, and AI Engineers to turn business requirements into data and AI solutions
- Design, build, and optimize ETL/ELT pipelines using Snowflake, Nexla, SQL, and Python
- Develop and manage Apache Airflow DAGs for scheduling, reliability, and operational observability
- Design AI-ready data architectures for analytics, machine learning, GenAI, RAG, vector databases, and agentic AI solutions
- Use AI-assisted development tools (Microsoft Copilot, Claude, Cursor) to accelerate delivery and improve code quality and documentation
- Define ingestion, transformation, governance, and serving patterns for AI training and inference workloads
- Support metadata management, data lineage, data observability, and governance practices for enterprise AI initiatives
- Evaluate and recommend emerging AI technologies, frameworks, and best practices to strengthen engineering capability
- Manage agile sprint execution: prioritize work, support backlog refinement, estimate stories, track dependencies, and meet delivery commitments
- Coordinate across onsite, offsite, offshore, and cross-functional teams to unblock delivery and maintain momentum
- Create and maintain technical design docs, data flow diagrams, AI solution architecture diagrams, and operational runbooks
- Drive data quality, performance tuning, monitoring, and production support for critical pipelines and Snowflake workloads
- Lead architecture reviews and code reviews, and facilitate root-cause analysis and problem-solving sessions
- Mentor team members in modern data engineering and AI-assisted development practices
- Promote responsible AI practices covering data privacy, security, governance, and ethical use of AI
Requirements
- Bachelor’s degree in computer science, IT, engineering, data science, AI, or a related field, or equivalent experience
- Strong hands-on Snowflake experience: SQL development, performance tuning, data modeling, stored procedures, and scalable processing patterns
- Hands-on ETL/ELT pipeline experience with Nexla and modern data integration tools
- Experience with Apache Airflow (or similar): DAG development, scheduling, monitoring, and troubleshooting
- Strong proficiency in SQL and Python for enterprise-scale data engineering
- Practical experience with AI-powered development tools including Microsoft Copilot, Claude, Cursor, and GitHub Copilot (or similar)
- Working knowledge of GenAI concepts: LLMs, RAG, vector embeddings, prompt engineering, and agentic AI patterns
- Experience designing data architectures and pipelines supporting AI, ML, analytics, and intelligent automation
- Knowledge of data governance, security, privacy, lineage, and compliance for AI and enterprise data platforms
- Proven ability to lead sprint execution, manage competing priorities, clarify scope, identify risks, and coordinate delivery
- Strong collaboration across product engineering, data engineering, AI/ML, BI/analytics, QA, architecture, and business teams
- Experience working in an Agile environment with onsite, offsite, offshore, and distributed teams
- Strong analytical, problem-solving, communication, and troubleshooting skills
- Self-starter with ownership, accountability, and attention to detail in a fast-paced environment
- Demonstrated ability to learn and adopt emerging AI technologies in a rapidly evolving landscape
Technologies
- Snowflake, Nexla, SQL, Python
- Apache Airflow, Apache Airflow DAGs
- Microsoft Copilot, Claude, Cursor, GitHub Copilot
- LLMs, Retrieval-Augmented Generation (RAG), vector databases
- vector embeddings
Preferred Qualifications
- Experience with Snowflake on Azure and related Azure data services
- Experience with Azure OpenAI, Microsoft Fabric, Databricks, Amazon Bedrock, Anthropic Claude, OpenAI, or similar AI platforms
- Experience implementing RAG, semantic search, vector databases, or agentic AI frameworks (LangChain, LangGraph, CrewAI, AutoGen, Semantic Kernel)
- Experience building AI-enabled data products, intelligent document processing, conversational AI, or enterprise knowledge retrieval systems
- Experience with additional ETL/ELT tools: Azure Data Factory, Talend, and Python-based frameworks
- Experience with DevOps, CI/CD, Infrastructure-as-Code, and MLOps supporting AI/data platform deployments
- Experience with Jira, Azure DevOps, GitHub, or similar Agile delivery and tracking tools
- Experience with metadata management, data quality frameworks, or Master Data Management (MDM) initiatives
- Experience in insurance, insurtech, healthcare, or financial services
- Relevant certifications in Snowflake, Azure, Generative AI, Claude, Microsoft Copilot, AI engineering, or cloud data platforms
Location and Salary
- Location: Dallas, TX (onsite)
- Salary: USD 103,000 - 126,000 per year
Similar Jobs
