The people behind the tasks

Interaction Bench is built by EvalSys, but every task in it starts as a defect someone already found and fixed in a product they work on. We revert that fix, pin the product to the commit before it, and ask an agent to find its way back. The benchmark exists because these people did the hard part first.

39Upstream authors
42Merged fixes turned into tasks
24Open-source projects

Benchmark Contributors

Contribute on GitHub

The EvalSys team: harness, tasks, runs and results.

Arvin Xu@arvinxx
Initiator · Leader · Contributor
Rylan Cai@cy948
Leader · Contributor
CanisMinor@canisminor1990
Contributor
Allison Zhan@AllisonXinyuan
Contributor
Neko@nekomeowww
Contributor
Innei@Innei
Contributor
MaxForAI@MaxForAI
Contributor

Upstream authors

All 39 authors

Every task is built from one of their merged fixes.

Acknowledgements

Repository

BibTeX

@misc{ibench,
  author = {{EvalSys}},
  title = {Interaction Bench},
  url = {https://github.com/eval-sys/ibench}
}