There Is No Neutral Harness: Modern LLM Leaderboards Are Manufactured by Config-Fragile Items

arXiv:2608.21382v1 Announce Type: new Abstract: Multiplechoice benchmarks fix the questions and the correct answers, but not the harness: the order of the options, the wording of the prompt, and whether a language model's answer is read from generated text or from peroption likelihoods.