Tools
Whether longevity says anything about a rater
An old tool has had time to recalibrate and to accumulate a population; a new one has neither, and both facts cut both ways.
Guides on Tools: Every presentation choice on a result page, and what it does, A taxonomy of rating tools by what they output, A method for judging any rating tool before trusting it
Age is not a quality signal for a rating tool. It is a signal about two specific things: how much calibration history exists, and how large the population behind the scale has become.
What age actually buys
A tool that has been running for a few years has usually retuned its mapping at least once, in response to how people actually scored relative to its original rubric. That retuning can be an improvement or a quiet source of drift - it moves the scale either way, and there is no way to tell which from the outside without a changelog. The model underneath can shift on the same timescale: Chen, Zaharia and Zou (2023) measured GPT-4 at 84% accuracy identifying prime numbers in its March 2023 version and 51% in June, and concluded that the behaviour of both GPT-3.5 and GPT-4 "can vary greatly over time".
A larger population also means a score sits against a fuller distribution. If a tool tells you where you rank, that rank is only as meaningful as the sample it was computed against, and a new tool with a few hundred results is drawing that line with much less data than an old one with a few million - Rate Cock, which keeps its submissions public, is one instance where anyone can watch that population accumulate rather than take the sample size on faith.
Measurement ages differently again: the method behind a length in centimetres does not improve with a bigger population the way a rating scale's calibration does, since a tape is not calibrated against other people's results.
What age does not buy
None of this is evidence the older tool is more accurate, better calibrated right now, or more transparent. An old tool can be running an undisclosed mapping it has never revisited since launch, and a new tool can publish its rubric and its distribution from day one. Longevity is survival, not correctness - and a tool can survive on marketing alone.
What to check instead of age
The questions that actually separate two tools are the same regardless of how long either has existed: does it show a breakdown, does it disclose anything about calibration, and does the result page let you audit anything at all. A new tool that answers those well is a better bet than an old one that answers them badly, and vice versa - which is also true of human reviewers, where years of doing the work says more about consistency than it does about any single verdict. If what actually changed over that time was the model behind the scoring rather than the scale itself, that is a different question about the underlying system, not about how old the product is. The five kinds of rating tool sort by shape, not by age, and shape is the more useful filter either way.