DeveloperJobs.io
← Back to all jobs

Job Description

Build foundational data infrastructure for agentic AI by creating ingestion pipelines, knowledge graphs, semantic data preparation, and secure APIs.

Responsibilities

  • Design and run reliable, scalable ingestion and transformation pipelines for structured, semi-structured, and unstructured data sources.
  • Build and maintain connector frameworks to integrate enterprise systems, including ERPs, PLMs, CRMs, legacy platforms, email, Excel, documents, and web APIs.
  • Develop the Data Fabric layer, including knowledge graphs enriched with ontologies, metadata, and relationships.
  • Prepare, enrich, normalize, and vectorize enterprise data to support AI and LLM applications such as RAG, summarization, and intelligent alerting.
  • Establish data contracts and implement access layers, data lineage, governance, and data quality processes.
  • Create secure APIs that allow services, AI agents, and users to access and query enriched semantic data.
  • Collaborate with ML and LLM teams to deliver high-quality enterprise data for model training, tuning, and downstream AI applications.
  • Build production-grade systems with support for schema evolution, observability, scalability, and dependable operations.

Requirements

  • Minimum 5 years of experience designing, building, and operating large-scale data infrastructure in production environments.
  • Hands-on experience with ingestion frameworks such as Kafka, Airbyte, Meltano, or Fivetran, plus orchestration using Airflow, Dagster, or Prefect.
  • Proven experience across structured, semi-structured, and unstructured sources, including PDFs, Excel, emails, logs, CSV, and web APIs.
  • Experience with columnar data stores, object storage, and lakehouse technologies such as Iceberg, Delta, or Parquet.
  • Strong knowledge of knowledge graphs and semantic modeling, with experience using Neo4j, RDF, Gremlin, or PuppyGraph.
  • Experience designing and developing GraphQL and REST APIs, including scalable and developer-friendly data access layers.
  • Knowledge of data governance, including RBAC, ABAC, data contracts, lineage, and data quality.
  • Highly desirable: experience with vector databases and embedding pipelines, including Weaviate, Qdrant, or Pinecone.
  • Advantage: familiarity with enterprise connector ecosystems, ontology versioning, graph diffing, and semantic schema alignment.
  • Plus: exposure to RAG pipelines, LLM fine-tuning, Linked Data, W3C standards, or modern data fabric architectures.

Technologies

  • Kafka, Airbyte, Meltano, Fivetran
  • Airflow, Dagster, Prefect
  • Iceberg, Delta, Parquet
  • Neo4j, RDF, Gremlin, PuppyGraph
  • GraphQL, REST APIs
  • RBAC, ABAC
  • Weaviate, Qdrant, Pinecone
  • RAG pipelines, LLM fine-tuning, Linked Data, W3C standards

Location

  • San Francisco, CA (onsite)

Salary

  • USD 150,000 - 200,000 per year

Benefits

  • Salary: $150k – $200k
  • Equity

Similar Jobs