The vocabulary of rating tools

This glossary defines the forty-odd words rating tools use and rarely explain, from axis and calibration to signed link, split into score-side and tool-side terms. Terms with a fuller post get a link; the rest get the plain definition.

Score-side terms

Axis. One component a tool scores separately before combining it into a total. What an axis actually promises is independence from the other axes, and that promise is not always kept.

Baseline. The first result in a series, taken under repeatable conditions, that later results get compared against.

Calibration. The mapping between an internal model value and the number printed on the scale. In the stricter machine learning sense, Guo and colleagues (2017) found modern neural networks "poorly calibrated": their stated confidence does not match how often they are right.

Ceiling effect. What happens near the top of a scale when most of a population crowds into the last point or two, compressing whatever distinctions were left up there.

Composite score. A total built from parts by a rule. The rule is the part that matters, and it is the part you are least likely to be shown.

Confidence value. A secondary figure some tools attach to a result, usually describing image quality or model certainty, not the correctness of the score.

Consistency. Whether a tool returns the same result for the same input, repeatedly. A separate property from accuracy, and the only one of the two you can actually check yourself.

Decimal. The second digit in a score like 7.4. It looks like precision; in most tools it is a rounding of an internal value that was never that stable.

Distribution. The shape of all the scores a tool has produced, across everyone who has used it. Two tools with the same average can have very different distributions.

Error bar. The range of results a single submission would plausibly produce if scored repeatedly. Almost never printed, almost always present.

Generosity bias. The tendency of a tool under commercial pressure to score high, because a tool that scores low loses users.

Normalisation. The mapping step between an internal value and the display scale - linear, clipped, or mapped onto a percentile. It decides more about your number than the underlying computation, which is a separate stage entirely.

Ordinal vs interval. Whether a scale only orders results (ordinal) or claims equal gaps between values (interval). Most rating scores are ordinal, printed as if interval.

Percentile. A score expressed as a position within a population, rather than against a fixed rubric maximum. An 8 as a percentile and an 8 as a rubric score are different claims.

Rank. A position among other submissions, as opposed to a value. A tool can offer one, the other, or both, and they answer different questions.

Recalibration. When a tool retunes its scale or rubric. Scores from before and after sit on different scales, even when both are out of ten.

Reference population. The group a score is implicitly compared against - "better than most" depends on who "most" is, and tools rarely say.

Regression to the mean. The tendency of an unusually high or low result to be followed by one closer to the centre, with nothing having changed. Barnett, van der Pols and Dobson (2005) call it "a ubiquitous phenomenon in repeated data".

Rubric. The list of things a tool claims to judge and how much each counts. Most tools have one internally; fewer publish it.

Total. A single number that compresses several judgements into one figure, discarding which one moved.

Weighting. How much each axis contributes to a total - equal, fixed, or non-linear. Different weighting schemes make different submissions look best from the identical breakdown.

Tool-side terms

Cost model. How a tool charges: free-with-account, per-result, subscription, or credits. What matters more than the price is what happens when a generation fails.

Dark pattern. A results-page design that pressures rather than informs - a locked breakdown, a blurred axis, "unlock your full result".

Disclosure. What a tool tells you unprompted: its rubric, scale, owner, update history. The single best predictor of whether a number is worth trusting.

Hidden weights. A total that does not equal a plain average of the shown breakdown, meaning some axes count for more than they appear to.

Multi-axis tool. A tool that reports a breakdown rather than a single number. Multi-axis tools still differ in whether the axes are computed independently or split from one figure for show.

One-shot vs iterative. Whether a tool is built for a single result or a series. Iterative tools keep history and encourage retakes; one-shot tools do not.

Pairwise rater. A tool that compares two submissions and says which is preferred, never assigning either a standalone value. A weaker, more defensible claim than a score.

Presentation layer. Everything a results page adds on top of the number - badge, colour, prose, animation, share card. Cosmetic, generated after the score rather than informing it.

Public entries. A feature that lets you see other people's breakdowns, which is what lets you audit a tool's axes, spread, and consistency from the outside.

Quiz-style rater. A tool that scores a questionnaire rather than an image. The number is a function of self-report, not of anything visual.

Reproducibility test. Submitting the identical file twice. The gap between the two numbers is the floor of a tool's own noise, before changing anything about the photo.

Retention period. How long a tool keeps your submission and result after you are done with it.

Rubric version. A dated record of when a rubric changed. A tool that keeps one is one you can compare against; a tool that changes silently makes every past score uninterpretable.

Scale anchor. Whatever fixes the ends of a scale - a percentile, a rubric maximum, or nothing at all, which is common.

Single-number rater. A tool that returns one figure. Easiest to read, hardest to learn anything from.

Share card. The image generated for posting a result elsewhere. Keeps the number and the brand, drops the breakdown, the conditions, and every caveat.

Signed link. A temporary, expiring URL to your result media, as opposed to a permanent public address. The less advertised privacy setting and the one that matters most.

Total-only tool. A tool with no breakdown. Still testable for consistency and sensitivity; you cannot ask it which judgement moved.

Word label. A term like "good" or "elite" attached to a numeric range. Where the label starts is the tool's decision, and it is usually generous.

The photo side has its own terms in the submission-side glossary, and measurement owns a third vocabulary, kept at the site that does that measuring. For how these words play out in a single disagreement between two services, why two rating tools disagree on the same photo is the worked case. If you got a low score from a human reviewer rather than a tool, the vocabulary is different again: that property belongs to ratepenis.com. And if you want to see what a tool that actually names its axes looks like in practice, Rate Cock publishes six on every public entry rather than one blended figure.