February 26, 2013

At a Loss with Video Compression? Here Are Some Encoding Basics and Artifact Removal Tutorials

While H.265 has been approved as the next-gen lossy delivery codec, we're still watching a vast amount of video in H.264. In fact, even when H.265 sooner or later takes its place, videomakers will still be dealing with many of the same basic compression principles at work. Knowing all the variables of a delivery encoding job can help optimize bit efficiency, ensure the highest possible quality of media, and reduce the visibility of artifacting such as banding. Read on for a look at what drives the quality-to-compression ratio of your lossy-encoded delivery video, and how you can even 'trick' it in some cases.

Credit for these finds goes to DVXuser MattDavis, responding to a thread started by another user worried his Vimeo compressions artifacts (specifically banding) stem from the Canon C100. While it doesn't appear that the camera is necessarily underperforming in its own encoding of underexposure, the discussion does shed some light on some qualities of footage that can have trouble translating well to H.264. Here's the video of DVXuser draculr's concern:


This raises a question that faces video uploaders to YouTube, Vimeo, or any major video hosting/sharing service: how much of the lossy conversion do we leave to the site, versus how much quality we manually take away ourselves beforehand? Almost invariably, such video services are chewing up and spitting out several encoding passes of your media, for those ubiquitous "HD on/of" or "360p/480p/720p..." options. Each service has its own processes, so a 1:1 match in quality can be difficult to arrive upon, but you can certainly encode for optimization. This tutorial by Jan Ozer of Streaming Learning Center goes into some of the basics for ensuring maximum control over your encodings:


Obviously due to the age of the video, some of the details are a bit out of date -- again, the principles are the key things to take away from this presentation. For those interested, here's a more recent (1 hour+) presentation Jan gave at NYU expanding on this topic, and taking it through to the monetization stage:


Or, for those specifically concerned with banding problems at the encoding stage, Nick Campbell has put together an After Effects tutorial (on GreyscaleGorilla) demonstrating some preemptive correction techniques, several of which emulate dithering:


The nature of compressions such as H.264 and H.265 means a continued tug-of-war with the artifacts demonstrated above, but a deeper understanding of compression may ease the struggle that uploaders face. These issues also carry over into the topic of resolution escalation, where it's sometimes easy to forget that larger spatial dimensions (like 4K) matter progressively less at delivery if co-opted by increased compression visibility. Not every high-efficiency codec will use the same techniques as the H.264 family (REDRAY, for instance, will not), but in the meantime, knowing your encoding can only help!


Your Comment


Great post!

February 26, 2013 at 2:02AM, Edited September 4, 11:21AM


Why encode at h264 for Vimeo / Yourube to re-compress it?
You can compress as PhotoJPEG 75% quality for much better results.

February 26, 2013 at 5:36AM, Edited September 4, 11:21AM


Do those files support audio? Does Vimeo support that for an upload?

February 26, 2013 at 10:32AM, Edited September 4, 11:21AM

David S.

I highly doubt the "much better results" claim, if the end result is highly compressed H264 anyway.

February 26, 2013 at 1:12PM, Edited September 4, 11:21AM


I watched the first video from Jan about H.264 and learned a few things about encoding that I had never picked up either while in school or in the workplace -- namely i-frames, p-frames and b-frames and how they work. Thanks for the information. When I have more time, I'll go back and watch the full-length lecture.

February 26, 2013 at 9:07AM, Edited September 4, 11:21AM


I usually just upload a Prorez file if my clip is not too big. H264 at 10,000 kps looks pretty damn good as well.

February 27, 2013 at 1:32AM, Edited September 4, 11:21AM


The problem is... the flat look. Yes, that flat look that is so pop. Encoders hate it. Especially when there are huge swatches of single color. H.264 files aren't even 8 bit, they drop those bits down in the encoding phase to 6-7 to save space. Only real thing to help it is to add noise and adjust the curves so that the image doesn't JUST have big blocks of same colors. Vimeo/youtube encoders also drop the bitrate a lot when they notice that the image doesn't have much in it.

February 27, 2013 at 7:55AM, Edited September 4, 11:21AM

mikko löppönen

wow, I always thought it was because of the sensors... I was playing with my 60D couple days ago and I shot 120 short 5-8sec. clips with different combinations of picture styles and camera settings. Then I uploaded them online and programmed a little app for browsing them - have a look at it if you are interested, it's at http://picturestyle.hoodiemode.com

Shots there are underexposed on purpose ;)

March 1, 2013 at 8:01PM, Edited September 4, 11:21AM


Hi Stefan,

Nice idea!
But what do you want to tell us with these? What conclusion do you have?


March 2, 2013 at 5:23AM, Edited September 4, 11:21AM


I'm sorry,I should have also attached the corresponding article I wrote. http://www.hoodiemode.com/playing-with-picture-styles-on-canon-dslrs/
I was basically looking for the best picture style and combination of settings that would give me the lowest possible noise in this underexposed shot. I pretty much wanted to verify the theory that more contrast will give you less grain.

March 2, 2013 at 1:03PM, Edited September 4, 11:21AM


If the sensor can only do 8 bit then it is the sensor's fault along with the h.264 long gop encoding.

As someone said, h.264 can look really good with a high arbitrate and higher pixel density such as 10 bit or more. I am surprised that Jan said nothing about bitrate, I suspect he does in the long one.

Two things on h.264 variants, x.264 is the best according the the latest in depth Moscow study. MainConcept is second but stay away from the Cuda and Matrox accelerated versions unless you just need something quick and dirty. The quality drops drastically with the accelerated versions.

BTW, the Nikon D4 and D800 allow for 422 video out of the HDMI that can be recorded externally without any internal codec, as in no h.264 or whatever. It may even be 10 bit but I have not been able to verify that.

Take it EZ,
Robert A. Ober

March 2, 2013 at 12:56PM, Edited September 4, 11:21AM


March 2, 2013 at 6:48PM, Edited September 4, 11:21AM


x.264 is definitely better than H.264 ... agree on that point.

May 22, 2013 at 3:25AM, Edited September 4, 11:21AM


the for sure solution to banding is encoding in 10-bit even if the source is 8-bit. In low bitrate coding 10-bit solves banding issues. The problem is that 10-bit is software decode only with 50% more cpu.

February 13, 2014 at 7:01PM, Edited September 4, 11:45AM


I am extremely inspired together with your writing skills and aso with
the layout for your weblog. Is that this a paid subject or ddid you customize it yourself?
Anywway keep up the nice high quality writing, it's uncommon to see
a great blog like this one today..

May 13, 2014 at 5:10PM, Edited September 4, 11:56AM