The MetaFine Open Competition is a central component of our workshop at CoRL 2026. It aims to move robot manipulation evaluation beyond binary task success by diagnosing the specific capabilities that determine fine-grained performance.
The competition is built on MetaFine, a diagnostic meta-evaluation framework that evaluates robotic systems along three complementary dimensions: understanding, perception, and controlled behavior.
Submissions will be evaluated using a unified diagnostic protocol. In addition to task success, the competition will report dimension-specific capability profiles and representative failure cases. Separate rankings will be provided for each track, together with an overall leaderboard.
To ensure fairness and reproducibility, participants will be required to submit executable code or standardized evaluation packages. Final rankings may be verified on hidden task configurations and perturbation sets.
The competition is open to researchers, students, and practitioners from academia and industry. Public baselines, documentation, starter code, and evaluation tools will be released to support participation.
Teams may participate in all challenge tracks. Further details on registration, submission format, compute requirements, and evaluation rules will be announced on this page.
The top three teams will be selected before CoRL 2026 and invited to present their methods during the workshop. Finalists will also participate in interactive failure analysis and community discussions with invited speakers and workshop attendees.
Competition results will serve as shared evidence for discussing how fine-grained robotic capabilities should be measured, diagnosed, and improved.
Awards will recognize the overall best-performing systems as well as outstanding performance in understanding, perception, and controlled behavior. Additional awards may be given for reproducibility, open-source contributions, and insightful failure analysis.