Is it mode collapse in LLMs, or incoherent training data?
What mode collapse is in LLMs, how to detect it with pass@k, and why a sharp drop after peak-only data selection behaved differently in our run.
What mode collapse is in LLMs, how to detect it with pass@k, and why a sharp drop after peak-only data selection behaved differently in our run.