AI Engineer, Large Scale Model Evaluation - Helix Team

Company: Tbwa Chiat/Day Inc
Location: Sunnyvale
Posted on: March 27, 2025

Job Description:

San Jose, CAFigure is an AI Robotics company developing a general purpose humanoid. Our humanoid robot, Figure 02, is designed for commercial tasks and the home. We are based in San Jose, CA and require 5 days/week in-office collaboration. It's time to build.Figure's vision is to deploy autonomous humanoids at a global scale. Our Helix team is looking for a Model Evaluator to take learned robot models to the next level. As a key member of our Helix team, you will be responsible for leading user studies, data collection efforts, and evaluations for AI models across multiple modalities. Your work will directly impact robots that we ship into the real world to perform useful work.Responsibilities:

• Evaluate Model Performance: Develop, implement, and refine rigorous methodologies for assessing AI model accuracy, robustness, and efficiency across multiple modalities (e.g. vision, language, proprioception).
• Framework & Tooling Development: Collaborate with internal teams to build or integrate new evaluation frameworks, simulation environments, and metrics tailored to our humanoid robot applications.
• Baseline & Benchmark Creation: Establish and maintain benchmarks, gold-standard datasets, and systematic test procedures for continued performance comparisons of new and existing models.
• Continuous Evaluation & Monitoring: Implement ongoing monitoring pipelines to detect model drift or performance degradation, and propose retraining strategies when necessary.
• Collaboration with Engineering Teams: Work closely with roboticists, software engineers, and data scientists to ensure end-to-end integration of model evaluation feedback into production systems.Requirements:
• The ideal candidate will have a strong computer science background, excellent attention to detail, and a passion to make an impact.
• Track record building and maintaining distributed systems.
• Thrive in a high pace environment, where solutions are often unclear and require exploration.Bonus Qualifications:
• Prior experience working with robotic learning systems or large generative models.The US base salary range for this full-time position is between $200,000 - $400,000 annually.The pay offered for this position may vary based on several individual factors, including job-related knowledge, skills, and experience. The total compensation package may also include additional components/benefits depending on the specific role. This information will be shared if an employment offer is extended.Apply for this job
  #J-18808-Ljbffr

Keywords: Tbwa Chiat/Day Inc, Sunnyvale , AI Engineer, Large Scale Model Evaluation - Helix Team, Engineering , Sunnyvale, California