The flattening problem
Ask one question — was this okay? — and you get one bit of information. Averaging a million such bits gives you a smooth, confident, nearly useless number. Two judges can both say "not okay" for reasons that have nothing in common, and the average erases the reasons.
The five benches are our refusal to flatten. Every case gets scored on moral reasoning, social cognition, preference modeling, epistemic calibration, and ambiguity resolution — by the AI, and implicitly by the crowd through how cases of each type resolve.
Where the interesting failures live
The reason this matters: machine judgment does not fail uniformly. In our data, models track human moral intuitions on clear-harm cases surprisingly well, wobble on social-context cases where the same act reads differently between cultures, and systematically overrate hype — treating confidence as evidence in exactly the way humans warn each other not to.
A single alignment number would average those into "pretty good." The bench breakdown shows a machine that is strong, weak, and overconfident at the same time, in different places. That texture is the research value.
Dynamic weights, honest composites
Each case type weights the benches differently, because an ethical dilemma is not a product launch. The verdict score is the weighted composite, and the weights are published per case — you can always decompose the number you are looking at. A score you cannot decompose is a vibe with digits.