Device consistency as a protocol rule

For comparable scores, keep the device you already have a series recorded on, not the best one available. An upgrade feels like progress and behaves like a variable: it swaps lens, processing and colour response at once.

What a new phone actually changes

A phone camera is not one variable, it is a bundle of several that happen to ship together. The lens changes, which changes how proportion is rendered at a given distance. The processing pipeline changes, and most phones now apply sharpening, noise reduction and tone mapping before you ever see the image, none of it documented and all of it different between models. The colour response changes too, sometimes within the same manufacturer's own lineup from one generation to the next. Camera distance is already the single largest source of unrelated score movement; swap the lens behind it and the same physical distance no longer produces the same framing, which reintroduces the exact variable a fixed distance was supposed to remove.

Why quality is the wrong axis

The instinct is to ask whether the new phone takes a better photo, and it usually does. That is not the question a comparable series needs answered. A phone camera is good enough for a repeatable submission on a condition that has nothing to do with megapixels - the condition is that it is the same phone, same lens, same settings, every time - and an upgrade breaks that condition regardless of how much sharper the new sensor is. Focal length is one of the specific ways a lens swap changes what gets rendered, and it is worth reading in isolation because it is the mechanism, not just the symptom, behind most of what a device change does to a score.

The same logic outside a camera

This is not a camera-specific idea. Measurement has the identical rule with a tape rather than a lens - a different tool, even a more accurate one, breaks a series that depended on one consistent instrument, and the fix is the same in both cases: pick one and stay with it. It also holds if the destination is a person rather than a model. A human judge is responding to the material in front of them, and a set shot on three different phones over a year reads as three different photographers, not one consistent submission.

What the tool sees, mechanically

The reason this matters at the model layer is straightforward geometry and processing, not superstition. How the underlying model actually reads an image is a subject worth understanding on its own terms, and the short version relevant here is that the model has no way to know a change in the numbers came from the lens rather than the subject - it only sees the pixels it was given. Hendrycks and Dietterich (2019) built a benchmark of 15 common image corruptions - noise, blur, brightness, contrast, JPEG compression among them - because image classifiers measurably lose accuracy on exactly these changes. Several of those are what a new phone's processing pipeline alters without asking.

Settings within the same device

Consistency does not stop at the device model. Some phones let the camera app switch between lenses automatically depending on distance, apply a "beauty" or smoothing filter by default, or vary exposure settings session to session unless locked. Checking that these settings are the same across sessions, on the same device, closes a gap that "same phone" alone does not fully close - two photos from the same handset with different processing settings applied are still, for this purpose, two different instruments.

When a device change is unavoidable

Phones break, get replaced, get lost. When that happens, the right move is not to keep comparing new results to the old device's numbers and hoping the difference is small. It is to treat the change as one of the events that retires a baseline and start a fresh one, logged against the new device from day one. A short gap in comparability, honestly marked, costs you less than months of results you cannot actually trust against each other. Rate Cock's own guidance on getting an accurate first result is a reasonable place to reset from if you are starting a new baseline anyway - it is written for a first good photo, which is exactly what a fresh baseline needs.

The rule compresses to one line: consistency in the instrument outranks quality of the instrument, for every property you are trying to compare rather than simply produce.