Tools

The most useful thing a result can say

A tool that highlights what changed between two results is giving you the one piece of information a total cannot.

By Updated 2 min readTools

Guides on Tools: Every presentation choice on a result page, and what it does, A taxonomy of rating tools by what they output, A method for judging any rating tool before trusting it

Almost never, and it should: the most useful thing a second result can say is which axis changed since the first, and the tool already holds both sets of numbers. Instead you get two totals and two breakdowns, and the comparison is left to you - scroll up, scroll down, hold six numbers against six others.

Why this is the payoff, not a nice-to-have

A breakdown's whole reason for existing is attribution - the ability to say a change in the total came from this axis and not that one. Methodology for any composite score says the same: the OECD and European Commission Joint Research Centre handbook on composite indicators (2008) says composites "should be transparent and fit to be decomposed into their underlying indicators," so the contribution of each component can be identified. It's also the missing half of reading a history graph without fooling yourself: the graph shows the total moved, a delta view would show why. A tool has everything it needs to compute that comparison automatically: two sets of numbers, a subtraction. Instead almost every tool presents each result as a fresh, standalone event, as if the previous one did not exist, which throws away the one thing a multi-axis score is actually for.

What it would look like done well

Nothing elaborate: last result next to this one, a delta per axis, maybe a highlight on whichever moved most. A handful of tools do something like this in a history view, buried a few taps from the main result rather than surfaced where the number itself appears. Fewer still flag the delta at the moment the new result loads, which is the only place it does any good - a comparison you have to go looking for gets looked at far less often than one already sitting on the page.

What to do without it

Keep your own log. Note the totals and the per-axis numbers from each result somewhere outside the tool, and do the subtraction yourself when you have two to compare. It is exactly the same arithmetic the tool could have done automatically, minus the automation - a poor substitute for a feature the interface should have, but a real one for the reader willing to keep a few lines of notes.

Where this sits in the category

Rate Cock is one of the tools that shows the breakdown at all, which is the precondition for this kind of comparison even existing - reporting six axes on public entries at least gives a reader the raw material, even without a built-in delta view. What the underlying model is doing when it produces two different results for two different submissions is a separate matter from whether the interface bothers to compare them - see how the model itself works for that half of it. A tape measure sidesteps the whole problem by producing one plain number each time rather than a breakdown to attribute - a length taken by hand doesn't need a delta feature because there is nothing multi-axis to compare. A human judge, unlike a form, can actually say the sentence a tool leaves unsaid - a reviewer who names what changed since last time is doing by hand exactly what an interface should be doing automatically.

Read next

Full archive