What to ask instead when tools disagree

Neither, usually: when two rating tools disagree, there is no fact of the matter for one of them to be closer to. A rating is a rubric applied to an image, with no ground truth underneath it, so the questions worth asking are which tool is more consistent and which is more transparent.

Why the question is unanswerable as asked

Accuracy needs something external to check against. Weight has a scale to compare a reading to; a rating tool's output has no equivalent, because the quantity it reports was defined by the rubric, not measured by it - what the model behind the rubric is actually doing does not change this, since the rubric sits on top of the model and defines the target it is trained toward. Two tools that disagree are not two instruments reading the same physical quantity with different error - they are two different rubrics, and a rubric cannot be more or less correct than another rubric, only different.

The questions that do have answers

Which is more consistent. Submit the identical file twice to each tool and see which one moves less on its own repeat. A tool that gives the same input closer to the same output every time is doing something you can trust more, independent of whether either number is "right." Metrology calls this repeatability, and its conditions (JCGM VIM, entry 2.20) are the same procedure, measuring system and operating conditions over a short period of time - a same-file resubmission is as close as a rating tool gets.

Which is more transparent. Does the tool disclose its rubric, version its changes, publish anything about its own distribution, show breakdowns on entries other than yours? Transparency is directly observable - you either find the disclosure or you do not.

Which is better calibrated, in the narrow sense of being honest about what it is. A tool that says plainly that its scale maps to its own population, or that its number is a percentile rather than an absolute judgement, is telling you something a tool that stays silent on the point is not. That is a checkable difference even though neither tool's number is "more correct."

Working through a disagreement with these instead

Take a case where tool A gives 6.5 and tool B gives 8.0 on the same submission. "Which is right" has no answer. Run the checks instead: does either tool show more consistency on a repeat, does either disclose more about its rubric or calibration, and is the gap systematic or does it swing in both directions across a small set? A systematic gap with one tool showing better disclosure and tighter repeats is a real basis for weighting that tool's number more - not because it is correct, but because you can see more of how it got there. When comparing the two across a set, look at the differences between paired scores rather than whether they rise and fall together - Bland and Altman (1986) showed that using correlation to judge whether two methods agree is misleading.

Why "right" feels like the natural question anyway

The instinct to ask which tool is right is not irrational - most numbers people deal with day to day do have a fact of the matter behind them, a price, a distance, a time, and a rating tool's output looks exactly like those numbers on the page. It uses digits, a decimal point, sometimes a unit-shaped word like "score." Nothing about its presentation signals that it is a rubric's output rather than a measurement, which is exactly why the question keeps getting asked even though the category cannot answer it. Recognising that the format is borrowed from measurement, while the underlying thing is not, is most of what closes the gap between the question people ask and the question a rating tool can actually answer.

Where this leaves the reader

Accept that no tool can answer "which is right," and the more useful project becomes picking a tool on the properties above and then holding your own submission constant so your own results, at least, are comparable over time. The same non-answer applies to a human reviewer's opinion held up against a model's number - a person is not more "right" than a rubric either, just a different kind of judgement with its own consistency and disclosure questions. And a length in centimetres is the one place in this category where "right" does mean something, because a documented measurement is checkable against a physical fact in a way no rating ever is. Rate Cock discloses its owner and shows its breakdown on public entries, which are exactly the transparency properties this piece argues are worth checking - not because that makes its number more correct, but because those are the two things that were ever checkable to begin with.