February 9, 2013

Will 6K with RED Dragon Mean Unmanageable File Sizes? Plus, New RED Rocket Card in Development

While companies like Sony and Canon are just starting to offer their first affordable 4K cameras, RED is forging ahead to even higher resolutions. 5K is so 2012. Thanks to RED we're about to get 6K. But it's not all about resolution, we're also getting other improvements with their new Dragon sensor, including better sensitivity and more dynamic range (Dragon will likely surpass the best 35mm film stocks with the latter). With all of these advancements comes a potential data headache for any working professional -- what are we going to do with all of this data? RED and Sony both work in compressed RAW formats, but 4K compressed RAW is still a lot of data, and 5K pushes that even further. How will we deal with 6K RAW? According to CEO Jim Jannard, that's where the power of RED's wavelet codec comes into play.

RED's CEO Jim Jannard explained, in a post that was deleted after this article was completed:

The Dragon sensor has less than half the noise of the Mysterium-X sensor. That means that you can use approximately twice the REDCODE RAW compression settings to get the same quality noise results at 6K as you have been getting with 5K. Dynamic range increases are not affected by compression settings. It also means that you can raise the ISO of the Dragon by (more than) double to get similar results as compared to the Mysterium-X sensor.

In the end... 6K is 45% more resolution than 5K. With double the compression settings, the files are actually likely to be SMALLER than what you are currently shooting. And dynamic range goes up significantly.

More resolution. Much more dynamic range. Higher ISO performance. Smaller files. Win.

And you don't have to buy a new camera...

You don't have to buy a new camera if you are a current EPIC owner, that is (SCARLET owners will receive a yet-undetermined trade-in credit if they want to upgrade to Dragon).

Noise and grain generally fight against compression schemes since they are random fine details, which brings us to Jim's statement. It's unclear why the post was actually deleted, but it brings up an important topic nonetheless. Theoretically having a cleaner image will mean less random fine details, and so the codec can compress more information for similar visual quality. That all makes sense, but it also means that you're physically losing more information the harder the compression scheme is working. REDCODE is a lossy wavelet codec, but in many cases if you're working with one of the lower compression settings, the results can end up being visually lossless (though they are not in a mathematical sense).

My knowledge of codecs certainly isn't near engineering levels of understanding (not by a long shot), but I do know that when you're dealing with lossy video codecs, even REDCODE RAW, the more you compress it, the more information you lose that you'll never get back if you need to push it in post-production. If you're currently using REDCODE 6:1 on an EPIC at 5K, does that mean Dragon could get the same visual quality at 12:1 on 6K? If so, that would go a long way towards making 6K file sizes manageable... but the smart money is likely on a more modest increase in the compression's efficiency (more modest than 2X, that is).

I think this whole conversation brings up an even bigger point that many are just starting to find out: uncompressed RAW -- especially 4K RAW -- can be very unwieldy. For example, Canon's 4K RAW is 1 Terabyte per hour. The forthcoming Aaton Penelope Delta shoots 3.5K RAW CinemaDNGs at a similar data rate as Canon's RAW implementation. Using the same CinemaDNG codec, 2.5K RAW on the Blackmagic Cinema Camera is roughly 512 Gigabytes per hour. RED 5K at 5:1 compression (which is what was used for Prometheus and Spiderman) is a little over 256 Gigabytes per hour. Hard drives might be getting cheaper, but when you consider your footage needs to be backed up somewhere, the numbers add up very, very fast. For all of the misinformation that gets spread about the company, when it comes to actually having to go out and shoot something, REDCODE RAW is one of the best supported, and most efficient codecs out there. REDCODE may have its issues (like any compression method), but your workflow is as important, if not more important, than the camera itself.

Let's take a quick detour into what "wavelet codec" means, since we mentioned it above and since we receive comments all the time from people assuming that editing 4K RED files on a laptop or desktop computer without special hardware is difficult. Via RED's site:

This efficiency is achieved in part because wavelet compression encodes image features at different scales separately. A wavelet file therefore contains a low-resolution base image, plus progressively higher resolution components — all the way up to the final full-resolution image, which is the net combination of all these:

As a result,low-resolution previews can be generated without having to process the entire high-resolution file. For example, a quarter resolution preview could be generated from just the above base image plus the leftmost component. Wavelets therefore make it easier to directly view and edit large videos on desktop computers.

So if you're editing a 4K file at 1/8 resolution, it's basically like editing a 512x270 video file. This is one of the other aces up RED's sleeve, especially now that Premiere Pro, Final Cut Pro X, and other NLEs natively support the .R3D format.

When it comes to more efficient compression thanks to lower noise, we'll have to wait and see if Jim's comments are accurate, but there is another bit of good news from Jarred Land. It seems Dragon footage will work with the current RED Rocket cards, and he also mentioned in the forum that there will be a newer heavy-duty card at some point down the road:

Its inevitable... but it's not a priority right now. It wont really be a replacement, The Turbo Rocket is going to be more for the power mongers out there.. insane speeds with some expensive horsepower. Current Rocket will work just fine, and you can double current Rocket up if you want to go even faster.

He also mentioned that we probably won't see a price reduction on the current card, and the new Turbo Rocket will cost RED twice as much to make -- so it's not going to be cheap. As technology gets better, every piece of the workflow should see improvements, and it looks like that is no different with the Rocket card. He didn't mention when we might see this new card, so I wouldn't expect it for some time.

What do you guys think? If anyone more knowledgeable than me can add to the compression conversation, we'd love to hear your thoughts in the comments.

Link: 6K Dragon and file size...

Your Comment

45 Comments

I think its also important to point out that some compression schemes also come at the sacrifice of increased processing power needs on the back end of things. To me, the resolution of these cameras is there but the dynamic range and native "look" still has a long way to go. Many love the Alexa because it produces stunning images directly out of camera with great skin tones. However, it lacks the resolution and the power of a RAW workflow. I love the Epic, its a much more powerful camera in my opinion, but it takes a lot more work on the back end to get amazing images. If RED could figure out a better "out of camera" color space that puts priority on skin tones and then up their image latitude (which they claim to have done)...they'll have the perfect camera.

Only time will tell...

February 9, 2013 at 7:48PM, Edited September 4, 11:21AM

2
Reply

Well some of this logic is flawed. First is from anyone complaining about the data rates. DPX which was the mastering format for years with the film workflow is about 650GB/hour with 2k footage. We are getting 10x the resolution at less than half the file sizes. To many professionals (people who can afford dragon) using the RED is probably a significant reduction in data storage needs than they are used to.

Now for RED rocket. The rocket has been out for a few years now and with all computing hardware I'm sure it costs significantly less to make now than it used to. If the turbo rocket costs twice as much to make as this current rocket (2 or 3 years old now) than it probably costs the same as the old rocket did when it was new. By that logic they could easily sell the new one for the price of the old and drop the old one if they WANTED to.

If this was their only rocket they could easily drop the price to sell more units now but since they are the only ones making it and its still a good product they dont have to. Many PC desktop users will be able to get away from using it as machines with 24 and 32 virtual cores become common but for those invested in mac machines and software that will be forced to use imacs and macbook pros this will be a powerful tool when connected via thunderbolt.

February 9, 2013 at 8:03PM, Edited September 4, 11:21AM

0
Reply

Jarred said that Red Rocket won't become cheaper because of the boards it uses aren't becoming cheaper either. It has something to do with http://en.wikipedia.org/wiki/Field-programmable_gate_array but I honestly have no idea what it is.

February 10, 2013 at 3:57PM, Edited September 4, 11:21AM

0
Reply
Denis Murin

You guys should first carefully compare the output of the RED ROCKET vs. REDCINE without a RED ROCKET.

I did, I researched that for quite some time, quite deeply. Its a fact that the output is DIFFERENT and I prefer the software only output BY FAR.

Software output (done using Graeme's fantastic code) looks softer. So the RED ROCKET does apply sharpening (its no longer RAW in that context) and creates blocky artefacts in the blacks (push gain/gamma heavily to reveal that). Those artefacts hit compression seriously - I measured up to 70% LARGER files from RED ROCKET than with software only. And those artefacts survive even down into a BluRay release copy, still increasing data rate either 10+% or reducing quality if you go fixed data rate.

While the ROCKET is fantastic for editing or proxy transcoding, its IMHO not if you want to work with the results.

Imagine: RED RAY is going to be 20 MBit. Do you really want to trade some serious percent of the image quality due to using the RED ROCKET instead of software only?

Btw, adding some very small gaussian sharpening to the software only output makes the image match the sharpness level of the RED ROCKET, while remaining much cleaner, without jaggies, when zooming in.

If you don't believe it, let me know or test it yourself.

Again, the RED ROCKET is not a bad tool, and if you like its results, I am fine with that. We use it as well. But not for final rendering, that is important to know. Then you can make a serious decision, if you know the facts.

February 14, 2013 at 3:34PM, Edited September 4, 11:21AM

7
Reply

I am also curious why the forum topic disappeared...looking forward to more Dragon news in the future...

February 9, 2013 at 8:51PM, Edited September 4, 11:21AM

0
Reply
Chase

It won't be long before Arri introduces their next camera. Sorry RED but 6k, 12k, 100k, doesn't matter to me. Alexa still has a more organic and pleasing look than RED.

February 9, 2013 at 11:36PM, Edited September 4, 11:21AM

10
Reply
Mark

Amen to that

February 10, 2013 at 8:40AM, Edited September 4, 11:21AM

0
Reply
Hampus

Sony F35 beats the Alexa!
FAN BOY!

February 10, 2013 at 12:09PM, Edited September 4, 11:21AM

4
Reply

Although being a RED owner I agree on Alexa having the more 'gentle' and pleasing image. But I wouldn't automatically put Arri's next camera over Red Dragon, as I understood the Dragon sensor will also get a whole new color science. Nonetheless, the opions we have already at our hands are great enough if you ask me :)

February 10, 2013 at 1:36PM, Edited September 4, 11:21AM

1
Reply
hansd

Ok guys its time to give it a rest, no camera is better than any other camera, its just what has the look for the project. House of Cards shot on Epic would not have looked as good on Alexa just because of the style they went for.

February 10, 2013 at 3:40PM, Edited September 4, 11:21AM

0
Reply
ryan

Organic, pleasing, blah blah blah. It's just good DR, color science and low-contrast filters to get that sweet look. Sorry, Arri, but I don't want to see your low res in IMAX near me, you can't effectively replace 65mm. Release the Kra... Dragon.

February 10, 2013 at 11:14PM, Edited September 4, 11:21AM

0
Reply
Natt

If it's so easy, why haven't there been a single project shot on RED that looks "organic"?

February 11, 2013 at 7:28AM, Edited September 4, 11:21AM

0
Reply
Hampus

And you know that Alexa with proper up-res looks more sharper and detailed than RED at 4K?

February 11, 2013 at 7:30AM, Edited September 4, 11:21AM

0
Reply
Hampus

Watched sky fall in IMAX. Looked soft. Sorry.

February 12, 2013 at 3:09AM, Edited September 4, 11:21AM

2
Reply

So do features shot on RED. A 1:1 crop of RED footage is about as sharp as a 5D mkII. And seeing a movie in IMAX when it's not shot in 70mm isn't worth the extra price of a ticket. Just like a post-converted 3D movie.

February 12, 2013 at 6:59AM, Edited September 4, 11:21AM

8
Reply
Hampus

Since for some. Reason I can't reply below I will reply here. FUD is unappreciated on no film school. Please troll elsewhere.

February 12, 2013 at 9:31AM, Edited September 4, 11:21AM

0
Reply

First time I've been called a troll, thank you sir. I am anything but scared, uncertain and doubtful, I speak from personal point of views. Is Alexa soft in IMAX? Yup, and so is RED and any other camera than actual 70mm film. And it's no secret that Deakins softened the picture on "Skyfall" for what he wanted.

February 13, 2013 at 6:56AM, Edited September 4, 11:21AM

0
Reply
Hampus

Manufacturers that went the Cinema DNG way need to offer the new Compressed Cinema DNG format that was announced by Adobe. Clearly uncompressed Raw is cumbersome and requires a huge quantity of fast storage.

REDCODE is clearly the strongest selling point of RED, it's efficient and practical.

And to those who are thinking "there is no need for 6K", 6K is just the perfect oversampling to produce a 3K image which allows some resizing to produce a 2K image.

February 10, 2013 at 12:13AM, Edited September 4, 11:21AM

3
Reply
Nej

Redcode and the relatively easy Red workflow get chronically disregarded :)

February 10, 2013 at 1:41PM, Edited September 4, 11:21AM

1
Reply
hansd

The funny thing about wawelet compression is that the artifacts you get from higher compression is noise so à theory about why they removed that statement is exactly that case. It may be that the chip is less noisy but the fact that you have 45% more resolution and you are compressing it harder is going to generate more noise. It is important to note that the percieved noise wouldnt be due to the sensor but due to the compression(well more of it). Wille on the subjekt of noise there are so many missconceptions. The most stupid thing that is spred over the internet is that higher quality codecs are more noisy. I dont know how many times I have read on forums about people how have avc codec cameras that they connect to an external recorder and then they say that it makes the picture becomes more noisy. While that is true few of them seem to realise is that the better encoding that is able to record the noise while avc does not. You guys should really do a post about different codecs.

February 10, 2013 at 3:50AM, Edited September 4, 11:21AM

0
Reply
T

Just to clarify what I meant regering the avc codec is that it is either unable to capture the noise structure or applies noisereduction before encoding or both. Meaning that the codec Is removing detail from the recorded picture not that it is inherently less noisy than a high quality codec.

February 10, 2013 at 3:57AM, Edited September 4, 11:21AM

0
Reply
T

I'm not sure where you're getting your info on wavelet compression...wavelet compression softens fine details the more it gets compressed, it definitely doesn't add noise. In general higher amounts of compression removes random fine detail, which means noise (you correctly note that in regard to the use of external recorders).

February 10, 2013 at 4:23PM, Edited September 4, 11:21AM

4
Reply
Gabe

First off, red compression ratios are just that. Ratios. Having a 2k frame @3:1 is far less data than 6k frame @3:1. There is no set bit rate, the camera will just record bigger files the bigger the frame size. So there is no additional compression.

Second, wavelet compression when stressed gets softer, and to a point cleaner. Recording at 3:1 is the grainiest (but sharpest) a red will ever record.

February 12, 2013 at 3:15AM, Edited September 4, 11:21AM

0
Reply

Compressed RAW, wouldn't you call that a oxymoron?

February 10, 2013 at 6:41AM, Edited September 4, 11:21AM

0
Reply
Film genius

No. RAW means "not debayered". And it can be compressed. The RAW still files from every DSLR manufacturer are compressed.

February 11, 2013 at 3:56AM, Edited September 4, 11:21AM

0
Reply

Compressed RAW is not an oxymoronic statement. Raw simply means the computer has to create an image from the data (e.g. the image is not debayered in the camera). Usually, raw means you can apply white balance and iso selection after shooting the footage without any degradation in the footage. However, Canon chose to bake in the white balance and iso in their C500 (you can check the article here: http://nofilmschool.com/2012/11/canon-c500-shipping-raw-4k/

In the end, it is just a different approach to maintaining quality with manageable file sizes. I appreciate each approach because they all have different pros and cons.

February 10, 2013 at 10:41AM, Edited September 4, 11:21AM

0
Reply
Morgan C.

Mark made an interesting comment that I've been tackling mentally regarding all these new advancements in cameras: is resolution really that important? In other words, why does anyone need 5k, 6k, etc.? I'm not being obstinate about it; I really want to know.

One of the more impressive technological advancements I just heard about is a new codec being developed that would enable vectorized images. That would circumvent some of this resolution/codec trouble wouldn't it?

February 10, 2013 at 10:46AM, Edited September 4, 11:21AM

6
Reply
DIYFilmSchool.net

Re: the vector codec, I doubt that'll have a practical application any time soon. Hello artifacting!

Debayering brings with it resolution loss. A 5K bayer pattern sensor yields 4K of real resolution after debayering. If you want to give yourself freedom to do any reframing in post, a 6K image should yield roughly 4.8K of real resolution -- and keep in mind many using the 6K Dragon sensor will be shooting in "only" 5K or perhaps 5.5K if that's a setting, due to the Dragon sensor being larger than the traditional Super35 image circle.

6K is there for shooting stills as well -- that's a 19MP image, which is DSLR territory (at much higher frame rates).

February 10, 2013 at 1:40PM, Edited September 4, 11:21AM

0
Reply
avatar
Ryan Koo
Founder
Writer/Director

where did you hear that? and what does that even mean? I am very curious

February 10, 2013 at 8:58PM, Edited September 4, 11:21AM

0
Reply

February 10, 2013 at 10:37PM, Edited September 4, 11:21AM

4
Reply

It's a long ways away, though.

February 10, 2013 at 10:38PM, Edited September 4, 11:21AM

0
Reply

Why more resolution? Ask Paul Thomas Anderson why he shot The Master on 70mm, not on conventional 35mm. Or maybe you can get some answers from Chris Nolan and Wally Pfister, fresh out of TDKR, half of which was shot in huge IMAX format. Or the creators of Baraka and Samsara. Now, oversampled 4K is your friend. F65 does it internally, but there's no room to do that inside Epic. They may release a module for that at some point in the future, dunno.

February 10, 2013 at 11:22PM, Edited September 4, 11:21AM

3
Reply
Natt

Exactly.

February 11, 2013 at 4:47PM, Edited September 4, 11:21AM

1
Reply
Mark

Oversampled 4K from 6K sensor is now. Just ask Sony.

February 10, 2013 at 10:39PM, Edited September 4, 11:21AM

0
Reply
Natt

In an interview Anderson said it was because Panavision recommended it and he didn't really know what he was getting into, he said it was audibly noisy and cumbersome
as well hahaha

February 11, 2013 at 11:09AM, Edited September 4, 11:21AM

3
Reply
Aaron G

that was in reply to Natt about the Master

February 11, 2013 at 11:10AM, Edited September 4, 11:21AM

0
Reply
Aaron G

Basing my assumption that wavelet compression is similar to JPEG compression I've created a test.
Opened photoshop and created 3x3 inch file, with a blue gradient. Jpeg at 60% compression would be saved at 2.8kb. I have applied little bit of noise to the same gradient image. When I export it as jpeg it is 9kb.

The logic is there, the less noise the image has, the less information it has to compress, which will result in smaller files. The statement has some truth, but I bet that the data rate will depend on your subject matter. If you record detailed scene in the forest full of pine trees, don't expect any reduction in data rate.

February 11, 2013 at 6:59PM, Edited September 4, 11:21AM

6
Reply

Wavelet and jpeg are not very similar. Their artifacts are nothing alike.

February 12, 2013 at 3:21AM, Edited September 4, 11:21AM

5
Reply

You are correct. I looked it up and it is actually JPEG 2000 that is using wavelet compression. I got the information from here: http://www.red.com/learn/red-101/redcode-file-format

I have performed the same test with JPEG2000 with even more dramatic results. Slightly larger file resulted in 2kb file without added noise, and 46kb with added 1% noise.

I believe my point continues to be valid.

February 12, 2013 at 6:44PM, Edited September 4, 11:21AM

0
Reply

I'm not 100% but I think jpeg2000 compressing an rgb value is different than raw data. Raw data is a stream of information before linearization. It's effect is somewhat different because its pre RGB value.

February 14, 2013 at 1:31AM, Edited September 4, 11:21AM

0
Reply

It looks like the camera industry is doing the same thing what the computer industry did back in the 90s.
It is all about the specs... we know how that ended up. Specs don't mean nothing!

Will 6K make your story better? Nope!
Why is everybody jumping on this resolution race. 2K for Cinema has been is fine! Besides if it's true and download/on demand over the net is a rising market. Everybody will be watching there films on Tablets.... that can't even compete with the Blu-Ray quality! ;)
Or, they download them in Youtube quality.

I like to see companies putting there money there where it really counts, workflow, file formats, tools and etc...

February 12, 2013 at 3:07AM, Edited September 4, 11:21AM

1
Reply
Tim

Why does everyone keep brining up 6K? that is only one of the improvements on this sensor. DR and clean high ISO is what everyone has been asking for from RED. You dont need to shoot 6K. So much misinformation on here its ridiculous

February 12, 2013 at 1:08PM, Edited September 4, 11:21AM

0
Reply
carlos

http://community.sony.com/t5/F5-F55/Red-v-Sony-shootout/m-p/71715#U71715

"Accordingly RED seeks a preliminary and permanent injunction enjoining
Defendants from making, using, importing, offering to sell and/or
selling its F65, F5, or F55 cameras."

FUCK YOU, Jim.

This is what you get when dealing with megalomaniac asshole.

February 13, 2013 at 11:57AM, Edited September 4, 11:21AM

3
Reply
Juhan-i

I first privately suggested the chip set in the red rocket to them in the days if the redone, as a short term solution to performance needs until gpu processing could do away with it cheaply. Long has been the time gpu's have been capable. The thing with custom chip sets, is the shear cost of them compared to a mass market product, that is subsidised by maybe hundreds of thousands of times the sales. Maybe development of the rr chips has been paid off and they dropped in price and moved to cheaper production process, and the manufacture is selling heaps to other clients, then is it closer to $1000 now? So, how does a $500 gpu card compare?

February 14, 2013 at 10:13PM, Edited September 4, 11:21AM

2
Reply
Diy

Though no one will probably ever read this because this is a few years past the original posting, I'm going to answer this anyway.

I own a Red Dragon, and the files are completely manageable. They are easy to store, easy to use, and I can edit them natively right on computer without changing a thing. I'm finishing up a feature documentary shot on a mixture of 5K and 6K footage, and it's all been done on my Mac Pro and Raid Drive. So... there you go.

August 27, 2015 at 5:16AM

0
Reply