Contribute to ContextBench
Submit a result
Share your system's performance with the community. Submit results and logs to our GitHub repository for review before they appear on the leaderboard.
- 01
Prepare your results
Run ContextBench and save your evaluation outputs, execution logs, and agent trajectories. Record the model, dataset revision, evaluated instances, and reproduction commands.
- 02
Open a pull request
Fork the leaderboard repository, copy the submission template, and add your results and logs. Run the format check, then open a PR against main using the submission checklist.
- 03
Review and publication
Maintainers review the evaluation setup, logs, and reported scores. Once approved, they add your result to the leaderboard and merge the update.
What to include
Create one folder per system and evaluation run. Include failed attempts and disclose any retries or task filtering so reviewers can check coverage and reproduce the scores.
submissions/YYYY-MM-DD_system-name/
metadata.json # system, submitter, evaluation setup
results.json # reported leaderboard metrics
README.md # method and reproduction instructions
logs/ # raw evaluation outputs and execution logs
trajectories/ # recorded agent interactions per instanceChoose Fixed harness for a model evaluated with the benchmark's adapted mini SWE-agent, or Agent systems for an agent and model combination. Use fractions between 0 and 1 for Pass@1 and retrieval scores; omit optional metrics that were not evaluated.
Keep artifacts in your submission folder. Remove credentials and personal data before committing; document any redactions. The guide covers large logs and the full review checklist.
Ready for review?
Push your submission branch to your fork, then select it under “compare across forks” on GitHub. Automated checks validate the format; maintainers verify the results and publish approved scores.