Photos

Why the rest of the frame is part of the submission

The tool encodes the whole image; a busy background is not ignored, it is scored alongside everything else.

By 4 min readPhotos

Guides on Photos: A complete protocol for comparable submissions, Everything about submitting more than one image, How to find out how much of your score is you

Yes, background clutter can change a rating, because the tool encodes the whole frame rather than isolating the subject first. Whatever sits behind the subject is input the tool includes, not noise it filters out, and a busy room contributes differently than a plain wall.

What "the whole frame" means in practice

There is no step where a model crops itself down to the subject before judging it. How the underlying system actually processes a frame is a longer subject than this note needs, but the relevant fact is simple: everything in the image contributes to whatever representation the model builds, and a cluttered room contributes differently than a plain wall does. Object-recognition research shows how far that can go: Xiao, Engstrom, Ilyas and Madry (2020) found adversarially chosen backgrounds made models misclassify images "up to 87.5% of the time" even when the foreground alone was classified correctly, and that models can reach non-trivial accuracy from the background alone. This is not a flaw to route around so much as a property to plan for.

Cluttered against plain

A cluttered background adds visual information that has nothing to do with the subject, and that information is not evenly distributed across two photos taken in two different rooms. One background might be brighter, one darker; one might contain sharp lines and text that draw attention in ways a model's encoding is not indifferent to; one might simply contain more of everything. None of this needs to move the score in a predictable direction to be a problem - the problem is that it moves the score at all, for reasons that have nothing to do with what is actually being rated. A plain background removes that source of variance rather than trying to guess which way it would have pushed.

Why this matters more for a series than a single shot

A single result taken in a cluttered room is not obviously wrong - it is one number, and the reader has nothing to compare it against. The problem shows up the moment a second photo, taken somewhere else, is meant to be compared with the first. If the backgrounds differ, some of whatever moved between the two scores belongs to the room, not the subject, and there is no way after the fact to separate the two. A comparable submission holds everything constant that is not the subject, and background is one of the more overlooked items on that list, since a plain wall is easy to forget matters until the two results disagree.

What "cluttered" actually covers

Clutter is not one thing - it is at least three, and they do not all behave the same way. Visual busyness is the most obvious kind: patterned fabric, stacked objects, anything with a lot of independent detail competing for the same encoding space as the subject. Colour is a second, quieter kind: a background in a strongly saturated or unusual colour can shift how the tool reads tone in the subject, in a way a viewer might not consciously register but a model's encoding still picks up. Scale is a third: a background object of a recognisable, fixed size - a doorframe, a chair - gives the model an implicit reference that a background with no recognisable scale does not, and that reference can interact with proportion-reading axes in ways that are hard to predict from outside the tool. A plain, unpatterned, neutral-toned wall with no recognisable object in it removes all three at once, which is the practical reason it is the standard recommendation rather than any specific alternative.

Background is not the same variable as crop

It is worth being precise here: how tightly a photo is cropped decides how much of any background survives into the frame, but what that background actually contains is a separate question with its own answer. A tight crop against a cluttered wall and a loose crop against a plain one can both remove most of the clutter from the final image, by two different routes, and a protocol has to account for both.

The specific choice between two plain options

Once "plain" is the goal, the remaining question is which plain surface to use, and that comparison - black against white, and which one repeats better - has its own answer worth reading on its own.

Where this stops applying

None of this bears on an actual measurement, which is read from a tape against a method, not from a photograph, background included or not. The same plain-background habit is worth keeping if the photo is headed to a human judge rather than a model - a cluttered room distracts a person's attention the same way it adds noise to a model's encoding, and Rate Cock's photo guide asks for a plain background for exactly that reason.

Read next

Full archive