Why a Smaller Video Can Look Worse

A twelve-second shot can expose what a file-size figure leaves out. Here is how motion, detail and repeated exports change what survives in a video.

A Sony Handycam Hi8 video camera, an example of a source recording device
In this story 5 sections

A video of a person sitting near a window can look excellent after compression. A video made a few seconds later, when the camera follows that person through a crowd, can fall apart in patches. If the files have the same duration and occupy nearly the same space, it may look as though an export program changed its mind. The difference starts with the picture. In the first shot, large parts of the scene remain similar from one moment to the next. In the second, the program keeps having to describe something that has moved, changed shape or appeared for the first time.

File size tells you how many bytes have been used. It cannot tell you which parts of the scene were hard to describe or what was sacrificed to fit. It pays to be precise about that distinction even when the entire job is to send a clip to somebody before the evening ends. You can spare yourself a long series of exports if you begin by finding out what has become harder to see.

What the encoder has to pay for

A digital video frame contains samples representing the picture. In an uncompressed sequence, recording another frame calls for another set of samples even when the camera and everything in front of it stays put. Storing the whole sequence in that form can take a great deal of space. A video codec defines a more compact way to represent pictures and how a decoder turns that representation back into frames.

One of the opportunities is prediction. A region in one frame may look like a region in a previously decoded picture, displaced a little. There is no need to describe a patch of brick wall as if each brick had just come into existence. The coded video can communicate a prediction and describe a difference. Some kinds of prediction also use information within a picture. This is a useful mental model, not a promise that an encoder divides the world into walls, faces and furniture as neatly as a person does.

Prediction is easiest to understand with a slow pan across a building. Where the wall has moved sideways in the frame, some of yesterday's picture, so to speak, is still useful. At the edge where something new comes into view, there is no matching patch of that wall in the preceding frame. Fine leaves in the foreground may be moving independently. Reflections on the glass will not necessarily go where the building goes. A pan is one movement to the person holding the camera and many different changes across the picture.

The detail that does not make it back

Lossy encoding spends bits unevenly on the information needed to reproduce a picture. An encoder can represent some parts of the remaining difference at coarser precision and obtain a smaller file. One practical consequence is that a soft gradation may return in steps. Another is that strands of grass may merge into a texture that seems to flicker or crawl when played. Exactly what appears depends on the codec, the encoder settings, the material and how you watch it.

Noise complicates the process. A video recorded indoors in low light may contain moving grain even if the furniture stays perfectly still. An encoder can have trouble predicting grain because a tiny light or dark patch need not recur in the same place. Preserving that pattern uses information the same file could have spent elsewhere. Smoothing can make compression easier but may be bad for the actual job if the indistinct surface carried fine writing or the marks a researcher hoped to study. There is no general instruction to remove all grain that survives both examples.

You can often spot a lossy encode first where two colours meet. A title might acquire a halo. A clean boundary may look busier when motion resumes. There are other possible causes of softness and uneven colour, so compare with the source before blaming the export. A camera that missed focus did so before the encoder opened the file. Enlarging a low resolution original can give it more pixels without discovering missing details.

A bitrate is a budget over time

Suppose two one-minute exports leave the video stream with an average of four megabits per second. Four million bits each second for sixty seconds comes to 240 million bits, or roughly 30 million bytes. Audio and container overhead would add to the total. That is a rough comparison, useful when planning how much has to move through a network. It does not mean each second receives exactly four million bits. One sequence may spend more during the camera move and less on the still shot before it.

Average bitrate and file size thus answer related questions. Neither identifies the most difficult second in either file. A smaller encode can look better when the encoder makes more useful predictions or spends its budget more effectively on the details a viewer notices. The reverse is true too. If a person needs to compare lettering on a moving sign, a file that preserves the lettering is more useful than one that looks smoother overall. The outcome depends on both the source and the purpose of the comparison.

Changing resolution divides the problem differently. A frame with fewer sample positions may let each part of the remaining picture receive a larger share of the budget, but it loses spatial detail before compression gets to work. Removing frames reduces temporal detail. A long, steady lecture and a quick demonstration with moving hands may therefore prefer different compromises even if their running times match. The best setting is sometimes the one that produces the least irritating imperfection in the specific part someone needs to use.

The file ending can hide several decisions

A container holds tracks such as video and audio, together with information a player uses to arrange them. The video codec describes how the video is encoded. The ending of the filename can give a clue about the container without settling exactly how the picture was compressed. A player has to handle both. The Alliance for Open Media publishes a bitstream specification and a separate description of how AV1 can be carried in the ISO Base Media File Format. That separation is a useful reminder that putting a stream in a package and creating the stream are different jobs.

Copying an already encoded stream into a different compatible container can leave the encoded picture alone. Decoding and producing a new lossy stream is another kind of operation, even if the filename looks almost unchanged. A recipient who says a video will not open may need a compatible container and streams rather than a smaller file. Treating every delivery problem as a demand for more compression can cost time and quality at once.

An encoder is also a program with its own decisions about analysis, speed and where to spend bits. Two programs that produce the same kind of bitstream do not have to make the same choices about a busy frame. A setting that asks an encoder to work more quickly may give it less opportunity to search for useful predictions. Check the result before comparing settings by number alone; the numbers on two different tools may not name an equivalent amount of work or loss.

Test the part you plan to keep

For a useful comparison, find twelve seconds that contain the difficulty. If it is a dance recording, include a turn and a part of the frame where fabric and moving hands cross. If it is a talk, select a moment when a slide changes and somebody gestures at a small word. Export from the same source twice, changing one setting at a time. Play both results at the display size that matters. A frozen frame can help to locate a flaw you saw at speed, but a flaw invisible during normal playback may have little consequence for the intended viewer.

Keep a separate note for the motion, the smallest needed text, the noise in dark areas and the edges of high contrast objects. This is more useful than an overall impression that one example looks clean. A trial on a short clip does not guarantee identical treatment of every minute by an encoder that adjusts its choices over the full running time. It does tell you which tradeoffs deserve a check again on the complete export.

If a file is going to be edited later, retain the highest quality source that is available alongside the small delivery copy. Starting the next edit with a copy that has already smoothed away fine texture requires the next encoder to work from that altered picture. It may spend bits reproducing a compression artifact as though the artifact were part of the original scene. Adding quality at that stage can preserve the altered image; it cannot establish what the deleted detail looked like. Keeping the files separate also helps when there is uncertainty about which copy was sent last time.

A rough size target is often practical. A messaging limit, a slow connection or a very full drive can make a long export pointless. The easiest test is to open the source and pick the moment that would be hardest to lose. Make the smaller file answer to that part of the video. If a piece of writing will need to remain legible, use it as the test; if the movement tells the story, watch the movement. The byte count is then a constraint on the work, not a substitute for looking at it.

Further reading