Submission Smoke
Validate schema, run bundle shape, tool boundaries, and scorer compatibility. Smoke runs do not enter the public leaderboard.
Submission
Evaluation process
EPOB keeps results comparable by fixing the protocol, versioning reference anchors, and publishing evidence packets rather than relying on a single model-as-judge.
Public Comparable
Run a fixed task set, seed set, resource profile, timeout, evidence schema, and scorer version. These runs can enter public leaderboard snapshots.
Robustness Track
Expand across task families, providers, and model endpoints for deeper sensitivity analysis. This track is higher cost and not the default submission path.
Audit And Reproducibility
Rerun selected cells, preserve hashes, inspect evidence packets, and publish limitations when reviewer agreement or formal venue checks are incomplete.
Reference Anchors
Fixed protocol, versioned anchors
- Candidate systems are evaluated under a fixed EPOB protocol, not judged by one permanent model.
- Reference anchor sets are frozen and versioned for drift detection and relative comparison.
- Leaderboard rows should identify the track, task set, resource profile, scorer version, and anchor set.
- Reference delta is reported separately from the absolute EPOB score.