DeveloperJobs.io
← Back to all jobs

Job Description

Join Disney Entertainment and ESPN Product & Technology in New York, NY (onsite) as a Software Engineer II on the Reliability Insights team. In this role, you will help prove how Disney’s operational AI and observability tools perform in the real world by turning telemetry, agent interactions, and operational logs into actionable reliability metrics and clear business impact. The work focuses on improving AI reliability, reducing noise, and making performance visible to both leadership and technical partners.

Compensation: USD 124,900 - 167,400 per year.

What you’ll do

  • Design, build, and maintain automated feedback pipelines that capture operational outcomes such as confirmed incidents, false positives, and dismissed alerts, then correlate them back to AI model predictions and agent actions.
  • Develop evaluation frameworks that measure model precision, recall, and model drift over time, with automated threshold alerts that support model tuning and help prevent alert fatigue.
  • Build end-to-end data pipelines connecting AI outputs to incident lifecycles, using metrics such as Service to Incident correlation, notification success rates, and Mean Time to Mitigation (MTTM) to quantify business impact.
  • Design and deploy scalable APIs, metrics layers, and single-pane model health dashboards that communicate AI ROI, operational health, and performance narratives for stakeholders.
  • Create mechanisms to collect structured end-user feedback so real-world usage signals continuously feed model improvement cycles.

What you bring

  • 3+ years of applicable experience in backend and data pipeline engineering, including building scalable APIs (for example, FastAPI, Flask) and metrics or insight generation platforms.
  • Strong data platform engineering experience using distributed processing and analytics tools such as PySpark, SQL, Pandas, Databricks, and Snowflake to process and analyze telemetry.
  • Practical understanding of model evaluation concepts including precision, recall, model drift, and ground-truth labeling systems.
  • Hands-on familiarity with modern engineering practices: GitHub, Docker, and cloud-native deployments on AWS and EKS.
  • Ability to use AI-assisted development tools such as Cursor and Claude Code to increase engineering velocity.
  • Strong analytical skills to translate system telemetry and operational logs into meaningful reliability metrics and business insights.
  • Excellent cross-functional collaboration and communication skills, including partnering with operational teams (SRE, DevOps) to turn raw data into actionable insight.

Technologies you may use

FastAPI, Flask, PySpark, SQL, Pandas, Databricks, Snowflake, GitHub, Docker, AWS, EKS, Cursor, Claude Code, Datadog, Grafana, Conviva, GPT-4, Claude

Preferred qualifications

  • Experience with observability, telemetry, and monitoring platforms such as Datadog, Grafana, and Conviva.
  • Familiarity with agentic workflows and foundation models (for example, GPT-4, Claude) and evaluation of output accuracy.
  • Knowledge of incident management workflows and SRE metrics, including MTTR, MTTD, and alert suppression.
  • Prior experience in data attribution, metrics aggregation, or financial and performance visibility tooling in high-scale environments.

Education: Bachelor’s degree in Computer Science, Engineering, or equivalent experience.

Similar Jobs