Pass@k: What the Metric Family Hides
Pass@k measures coverage, not reliability. How to estimate it without bias, why HumanEval+ and pass^k reorder models, and what to report for agent loops.
Pass@k measures coverage, not reliability. How to estimate it without bias, why HumanEval+ and pass^k reorder models, and what to report for agent loops.