DeveloperJobs.io
← Back to all jobs

Job Description

Baker Tilly Advisory Group, LP is building AI capabilities through its founding AI Product Group (APG), and you will help create the data infrastructure that makes AI applications and intelligent agents reliable in production. This onsite role in Santa Clara, CA supports scalable, secure, and governed pipelines across lakehouse architecture, knowledge graphs and retrieval, and evaluation-driven data engineering.

What you’ll build

In this senior role, you will design and deliver production-grade data foundations for AI readiness and continuous improvement. Work centers on batch and incremental processing, automated validation and metadata management, identity resolution, and knowledge graph retrieval pipelines that stay accurate as source systems change.

  • Design and build data pipelines for structured and unstructured inputs, producing clean, validated datasets in a modern lakehouse environment
  • Develop lakehouse architecture using modern principles such as medallion architecture, data modeling, and integration of enterprise and legacy sources
  • Ensure data quality, lineage, and provenance with automated validation, reconciliation, lineage tracking, and metadata management
  • Build data models and identity resolution through data contracts and scalable entity-resolution processes for consistent client, engagement, and document identities
  • Implement governance and access controls including data cataloging, classification, permissions, and access management for sensitive information and AI agents
  • Develop knowledge graph and retrieval capabilities including graph structures, search indexes, and retrieval pipelines grounded in properly governed information
  • Support AI data readiness by capturing agent activity, practitioner feedback, reviews, and corrections to improve ground-truth datasets and evaluation outcomes
  • Drive evaluation-based engineering by establishing measurable quality standards, testing frameworks, benchmarks, and validation processes for extraction, retrieval, and grounding accuracy
  • Maintain freshness and reliability with incremental processing, change data capture, versioning, and monitoring for data products, knowledge graphs, and search indexes
  • Own production delivery by deploying, monitoring, troubleshooting, and maintaining production pipelines with automated testing, code reviews, and operational documentation
  • Support knowledge transfer and technical ownership with reusable code, tests, and runbooks while collaborating with external partners
  • Collaborate cross-functionally and globally with AI Engineering, Platform & Production Engineering, enterprise data teams, subject matter experts, and teams in India
  • Contribute to strategic initiatives through special projects and other duties that advance the firm’s AI and data platform capabilities

What you bring

  • Bachelor’s degree in Computer Science, Engineering, or a related field (advanced degree preferred)
  • 3+ years of hands-on data engineering or software engineering experience, including production pipeline delivery and maintenance
  • Experience independently building and operating production pipelines on a modern lakehouse platform, with demonstrated ownership and operational reliability
  • Experience implementing data quality, lineage, or data contracts with automated testing, reconciliation, validation, or traceability practices
  • Strong Python and SQL, with ability to develop, optimize, test, and maintain production-quality solutions
  • Strong data modeling fundamentals for scalable, maintainable structures that adapt to evolving sources and business needs
  • Experience designing batch and incremental pipelines for structured and unstructured enterprise data
  • Software engineering fundamentals: version control, automated testing, code reviews, debugging, and repeatable workflows
  • Ability to validate solutions with measurable outcomes such as tests, benchmarks, experiments, performance metrics, or comparative evaluations
  • Strong preference for lakehouse technologies such as Databricks, Apache Spark, Delta Lake, and/or medallion architecture
  • Cloud experience preferred, especially Microsoft Azure
  • Governance experience preferred, including data catalogs, classification, access controls, lineage, and tools such as Unity Catalog or comparable solutions
  • Preferred experience with knowledge graphs, graph data modeling, vector search, hybrid search, or retrieval-based applications
  • Preferred familiarity with unstructured processing such as document extraction, content chunking, metadata management, or search indexing
  • Preferred experience with entity resolution, master data management, change data capture, or incremental data processing
  • Strong analytical and problem-solving skills, including troubleshooting complex issues and making sound technical trade-offs
  • Excellent written and verbal communication skills for documenting solutions and collaborating with technical and nontechnical stakeholders
  • Ability to work with business subject matter experts and globally distributed stakeholders
  • Capability to manage competing priorities in a fast-paced environment
  • Prior AI/ML/LLM experience not required; engineering fundamentals and willingness to learn emerging technologies are essential
  • No professional certifications or licenses required

Compensation

USD 185,000 - 230,000 per year

Location note: In California, the pay rate range is $185,000 - $230,000.

Tech you may work with

Python, SQL, Databricks, Apache Spark, Delta Lake, medallion architecture, Microsoft Azure, Unity Catalog, knowledge graphs, graph databases, graph data modeling, vector search, hybrid search, retrieval-based applications, document extraction, content chunking, search indexing, entity resolution, master data management, change data capture, incremental data processing, LLM systems

Similar Jobs