What you can still check when there is only one number

You can audit a total-only tool for repeat consistency, sensitivity to one changed variable, and how it orders a set, using nothing but repeated submissions and a log. What you cannot get is attribution: without a breakdown, the tool cannot tell you which axis moved.

What you can check

Repeat consistency. Submit the identical file twice, a few minutes apart. The gap between the two totals is the tool's floor noise - how much a number moves with nothing about the submission changed at all. A tool that returns the same file two different numbers a full point apart is telling you something about how much weight to put on any single result, breakdown or not. Reliability research formalises the same check with the intraclass correlation coefficient, and Koo and Li (2016, Journal of Chiropractic Medicine) grade values below 0.5 as poor, 0.5 to 0.75 moderate, 0.75 to 0.9 good and above 0.9 excellent.

Sensitivity to one changed variable. Change a single thing - crop, angle, light - and resubmit, holding everything else fixed. A total-only tool should still respond to a deliberate change in a direction that makes sense, even without telling you which internal judgement moved. If the total doesn't move at all when you change something that plausibly should affect it, that's informative; if it moves wildly on a change that shouldn't matter much, that's informative too.

Ordering of a set. Submit several genuinely different photos and rank them by the totals that come back. Even a single-number tool tends to be more reliable at ordering a set than at pinning an absolute value to any one of them - a tool's order is more trustworthy than its score, and that holds whether or not it shows its work.

What you cannot check

Attribution is the one thing a total-only tool structurally cannot give you. When the number moves, you cannot ask why - whether a change in your submission affected one specific quality or several at once, or whether the movement is even related to what you changed rather than noise. A breakdown exists specifically to answer that question, and there is no substitute for it from outside a single-number interface, no matter how carefully you test around it. The audits above tell you how much to trust the number - not what the number is made of.

Reading the results together

The three checks are more informative combined than alone. A tool with poor repeat consistency and strong sensitivity is noisy but responsive - trust the direction of a change, not its size. A tool with tight repeat consistency but weak sensitivity is stable but possibly insensitive to the variable you tested - useful for tracking slow change, less useful for testing a specific hypothesis about what moved a result. A tool that fails both is the one worth being most cautious with, regardless of how confident its single number looks on the page.

Running the audit in practice

None of the three checks needs more than a handful of submissions and a place to write the results down: file, total, date, what you changed. Repeat consistency needs two identical files. Sensitivity needs a pair that differ by exactly one variable. Ordering needs three or four genuinely different photos scored in the same sitting. That's the whole method, and it's cheap enough to run on any tool before you decide how much weight to give its output.

Where this fits

This is the total-only version of a broader routine - the fuller evaluation checklist covers the same ground plus what a breakdown-carrying tool adds on top, and what a single-number tool is built to do in the first place is worth reading if you want the category context rather than the audit itself. Sensitivity testing of this kind is really a photography discipline as much as a tool one - what actually happens to the image geometry when a variable changes is the mechanism behind why the total moves at all. A tape measure passes the repeat-consistency test almost by construction, since a length taken with a fixed method has very little room for the file-to-file noise a model-based total carries. Ordering a set is also something a human reviewer does differently than a model does - how a panel handles ranking multiple submissions tends to be more explicit about relative position and less confident about any one absolute score, which is roughly the same lesson a total-only audit teaches about a model. Where a tool does show its breakdown, the audit gets easier rather than harder - Rate Cock's public entries let you run the sensitivity check per axis instead of only on the total, which is strictly more information for the same amount of work.