Staff, Robotics ML/Data Engineer

Persona AI Inc is hiring a Staff, Robotics ML/Data Engineer to architect and scale multimodal robotics data pipelines for foundation model training.

Responsibilities

• Design cross-modal validation systems that check agreement across video, proprioception, force or haptic signals, and language annotations.
• Run validation approaches such as reprojecting robot state into the image plane to verify video-state consistency.
• Use VLM-assisted checks to confirm instructions align with observed behavior.
• Orchestrate multimodal modules including hand-tracking, segmentation, depth estimation, 3D reconstruction, and pose-tracking.
• Retarget human demonstrations into robot trajectories.
• Apply simulation-in-the-loop validation to ensure physical grounding, including kinematic feasibility, physics replay, and motion-consistency filtering.
• Implement robust data augmentation for expert trajectories using spatial transformations, temporal scaling, synthetic viewpoints, and sensor noise injection.
• Unify state-action representations across embodiments, coordinate frames, rotation conventions, gripper or hand parameterizations, and sampling rates.
• Add per-dimension validity masking and per-source normalization so onboarding a new robot or sensor is configuration-driven rather than a rewrite.
• Build dataset tooling for researchers to query, visualize, and audit data, including clip browsers, trajectory viewers, and annotation review UIs.
• Translate model failure analyses into new curation rules and targeted re-collection requests.
• Architect end-to-end ingestion pipelines from raw recordings such as egocentric video, teleoperation sessions, and third-party open datasets.
• Produce indexed, queryable, training-ready datasets including temporal segmentation into action clips.
• Extract metadata and scene-graph information, support embedding-based retrieval, and implement language annotation workflows.

Requirements

• M.S. or Ph.D. in Computer Science, Data Engineering, Machine Learning, Robotics, or a related field.
• Deep expertise in Python and extensive experience with PyTorch, including custom dataloaders for multimodal datasets.
• Experience processing complex time-series data from force-torque (F/T) sensors, load cells, or tactile arrays with precise alignment to visual frames.
• Strong video processing pipeline experience and familiarity with libraries including OpenCV, FFmpeg, and Decord, including managing I/O bottlenecks for terabyte-scale video datasets.
• Working knowledge of 3D geometry and robotics data: coordinate frames and transforms, rotation representations, camera intrinsics and extrinsics, forward and inverse kinematics, and URDF.
• Ability to implement programmatic and generative data augmentation techniques for computer vision and time-series data.

Technologies

• Python, PyTorch, OpenCV, FFmpeg, Decord
• URDF
• Ray, Apache Spark
• Open X-Embodiment, DROID, AgiBot World, EgoDex
• SAM-family, MANO, SMPL
• Omniverse, MuJoCo
• NVIDIA robotic software stack, NVIDIA's robotic software stack
• VLM

Bonus Skills

• Experience with NVIDIA’s robotic software stack, including Open X-Embodiment, DROID, AgiBot World, EgoDex, or similar.
• Familiarity with modern perception toolbox workflows as a user: segmentation (SAM-family), monocular depth, hand or body pose estimation (MANO/SMPL), 6-DoF object pose tracking, and point tracking.
• Comfort composing and evaluating these components in a data pipeline without needing to train the models.
• Familiarity with distributed data processing for cluster computing (Ray, Apache Spark).
• Background generating or utilizing synthetic robotic data via simulation (Omniverse, MuJoCo).
• Experience integrating spatial awareness or tactile data representations (for example, Fourier encoding) into visual pipelines.

Benefits

• Competitive compensation
• Performance-based bonus
• 99% employer covered medical benefits
• Early-stage equity
• Competitive PTO
• Company-wide paid winter break between December 24th and January 2nd
• Full access to advanced tools

Job Details

• Job Title: Staff, Robotics ML/Data Engineer
• Department: Software
• Reports To: Teleoperations Lead
• Employment Type: Full-Time
• Location: Houston, TX (onsite); Houston, TX or Pensacola FL