December 21, 2011

With New Magic Lantern Firmware, Canon DSLRs Get Real-Time HDR Video*

Is real-time High Dynamic Range imaging, ala RED's HDRx, coming to Canon DSLRs with the next version of Magic Lantern firmware? Well, yes and no -- thus the asterisk. The system they've developed splits a 24p stream into two 12FPS exposures, one high and one low, and then interpolates the frames. It's an interesting hack, demonstrated here:

It should be available tomorrow for the Canon 60D, T2i, and T3i. More details from the Vimeo User Group:

You will get two streams at half frame rate. For example, if you record 25p, you will get two 12.5fps streams. Interpolate them to 25fps and use your favorite HDR software to blend the frames. Our workflow is cross-platform and uses only free software: enfuse, AviSynth, InterFrame and VirtualDub... Will it work on 5D Mark II, 50D and 500D? Maybe. Right now there are some synchronization issues on these cameras. More details coming soon.

While this has a limited application, it looks like an interesting addition, and quite frankly even RED's more advanced HDRx exposure hack has limited capabilities when shooting motion. More on that soon.

Also coming in the next firmware: variable frame rates. They are limits and the cameras can overheat quickly, but here's a peak at the feature:

Link: Magic Lantern Wikia page (HDR feature should be available as of 12/22)

[via CheesyCam]

Your Comment

39 Comments

Hmmm... I'm going to download it no matter what. I just hope I can keep my sanity

December 21, 2011

1
Reply
Julian

UFFFF!!! But why no 7D?!?!? =( i want the hack on it!!!

December 21, 2011

1
Reply
Luigi

I have been following the Magiclantern development for a little over a year now, and Sadly the 7D has been made VERY hack-proof, to the point of many firmware pioneers' 7D's being "Bricked" in their hacking efforts. Canon probably upped the security when designing their 7D for the fact that it has dual Digic 4 image processors, whereas all of the other ML-capable cameras (to the best of my knowledge) have one image processor. So, considering that all of these magical and wondrous things are achievable with one processor, imagine what one could do if a dual processor camera was unlocked.... It would make the 7D quite a formidable (even more so) device, competing more aggressively with their own (proper) video cameras. Who knows if Canon did this out of foresight, thinking that someone like Trammel, Alex, and company would make such strides with a hacked firmware, or if they were simply escalating security in proportion to such significant hardware upgrades.

Disclaimer: I myself am merely an avid follower of the ML development group, and could have missed a recent update stating "7D development breakthrough!!!!", but I highly doubt it, considering what the current data suggests. BUT, those of you possessing one of the listed, ML compatible cameras, I would highly recommend giving the latest stable release a try, I know the firmware has been featured here before, but here is a link to the wiki-page: http://magiclantern.wikia.com/wiki/Unified

Have fun with the coming HDR* feature!

December 21, 2011

0
Reply

Nothing is imposible, yes the dual digic 4 system is locked down (and canon wont spill the beans about how to work with it) but i have faith in the community to make a breakthrough soon

December 21, 2011

-1
Reply
John Jeffreys

It also has to do with the fact that the firmware updater has a limit, so even trying to crack the firmware is difficult because they can only update it X amount of times; then they can`t even upgrade Canon`s firmware.

December 26, 2011

0
Reply
DavidJFulde

I mean: They can`t even upgrade with Canon`s firmware. Once the limit is reached; they are stuck.

December 26, 2011

-1
Reply
DavidJFulde

Soooooo... that's kind of neat, and also kind of dumb. I will not be making any film at 12.5fps. Perhaps if you needed some fast-motion HDR, you could doublespeed it? Other than that... bleh. What if they did this same thing with a 720p60 mode, though?

December 21, 2011

-1
Reply
Rev. Benjamin

It is 2 streams combines to make a 25fps video (in the case of 12.5fps streams)...

December 21, 2011

0
Reply

Ah I see. I stand corrected. Still kinda looks jittery in my opinion. But a cool advent.

December 21, 2011

1
Reply
Rev. Benjamin

you CAN shoot in 720P 50 (or 60, or 48) with HDR mode and then combine it that way :)

December 26, 2011

0
Reply
DavidJFulde

Meh. (disclaimer, I'm on a GH2 now)

For scenes with little or no motion, you can probably get away with it, like landscapes. But if there is motion, as evidenced in the sample video, you get "weird jittery frame incongruity artifacts." The result of interpolating 2 12fps streams.

Better to do this though and push the envelope. However, those jitters are a non starter for anything I'd shoot, professionally or personally.

December 21, 2011

0
Reply

I've done a few little experiments with the HDR feature and it's pretty good !

(Unless there's ANYTHING moving in your shot. Then it's crap).

December 23, 2011

1
Reply
fabdex

if you shoot in 720P 60 (or 50 or 48) the jitteryness is FAR far less and is actually usable!

December 26, 2011

0
Reply
DavidJFulde

Hopefully a 720x60/50fps option IS enabled, and who's to say that the workflow they have demonstrated is the optimum, maybe (as with so much of ML) the development team, or someone else, will surprise us all and figure a way to at least smooth some of the blemishes.

Though, at worst, this is a new way to experiment with the video capabilities of a camera that is meant primarily for stills. I feel that any developments past the stock features, are gravy on top of an already delicious meal.

December 21, 2011

0
Reply

It is possible. You can shoot at 720P 60 (For 30FPS) 720P 50 (For 25) or even 720P 48 (For 24)>>There are alternate workflows, you can look them up on Vimeo (I have one that I feel is pretty good. Be sure to read the desc.)

December 26, 2011

0
Reply
DavidJFulde

December 21, 2011

0
Reply

If I can shoot even 20 second shots at 50fps, this will be amazing.

December 21, 2011

0
Reply
Hahibundy

How about a stable ML for the 5D instead?

December 21, 2011

-1
Reply
Jack L.

It`s being added to the Unified tree to have all of the features of the T2i, T3i, 60D etc. soon!

December 26, 2011

-1
Reply
DavidJFulde

So cool! Gotta try this asap when it comes out.. I've got a 550D, do I need some software to combine these two videostreams to one hdr video?

Here's my last work with the 550D: http://www.youtube.com/watch?v=9Mm8QyuRlDk

December 21, 2011

-1
Reply
Britti Faija

DUDE that video is SIIIICK!!!

December 21, 2011

0
Reply
TSM

Hey. I just had a great time watching that video. What I am not a fan of though is how you can see a equipment (camera on a tripod...6 times) in the frame. You should have put that away. It really distracts. If it's meant to be in the video...then I don't get why. Otherwise, fun stuff, keep it coming.

December 22, 2011

0
Reply
Raphael

Philip Bloom<s blog has a handful of different workflows with different software.

December 26, 2011

0
Reply
DavidJFulde

This will, I think, be extremely valuable. The scenes where I really notice clipped highlights or shadows are typically not fast-paced action scenes (and honestly DSLRs aren't very good for fast-paced action scenes in the first place). I'm very excited about this. I think now that there are reasonable algorithms for frame interpolation, there are going to be a lot of interesting software developments like this. Things like Twixtor and the Warp Stabalizer, for instance, do totally different things, but they're both based on similar shape/edge tracking systems. I imagine there are still a lot of untapped applications of this.

Speaking of which, does anyone happen to know if the "line skipping" on DSLRs is under the control of firmware, or is it built directly into the chip? I think that if it were possible to alternate which lines are read every other frame it might be possible to extract a significantly higher-detailed image at the end. Of course, long GOP compression could completely eliminate any benefit, but it's an interesting thought.

December 21, 2011

0
Reply
cows

Oh, one other thing. It looks like they're interpolating the frames on the two "streams" independently of each other, which seems to work basically fine. It's possible (in theory, at least) to get an even better result by taking into account, in the weighted interpolation, the data from both streams simultaneously.

That is to say, when getting the "between" frame from two high-exposure frames, you can take into account the information from the appropriate low-exposure frame (which is chronologically in exactly the right position as the high-exposure "between" frame) as well to get a much smoother result.

The right algorithms to do this already exist but I'm doubtful that there's an implementation that could handle this specifically. Still, it shouldn't be that hard to do. Maybe in my vast quantities of imaginary free time I'll try to make a proof of concept. I'm not sure that it would perform substantially better than generating the between frames from each stream independently, but it couldn't possibly perform worse.

December 21, 2011

-2
Reply
cows

This will only be able to be used on shots with very little movement, look at the man's legs as he walks in the second example his leg actually vanishes every other frame.

December 22, 2011

-1
Reply

There are other methods which result in much less artifacting :)

December 26, 2011

-1
Reply
DavidJFulde

This is awesome news. Now I can rewrite my article on how to achieve HDR video without having a RED http://www.reelseo.com/hdr-video-3d/

Until now, the only way to achieve this was by combing multiple exposures from different cameras on a 3D rig, or using simultaneuos exposures.

Hopefully Canon is paying attention.

December 22, 2011

-1
Reply

I don't wonder if it would work shooting at 50/60fps. Could make it very useful indeed.

December 22, 2011

0
Reply
Monty

Hey I use Magic Lantern and I love some of the features.
I'm not sure if these will get much use but if you use the 2 new features together couldn't you shoot at 50fps and end up with 2 24fps streams?

Cheers

Rob

December 22, 2011

-1
Reply
Rob

Yep! You can shoot at 48FPS and get 2 24FPS streams (One of highlights and one of lowlights)

December 26, 2011

0
Reply
DavidJFulde

Thanks guys, I'll do an update - great stuff.

December 26, 2011

0
Reply
avatar
Ryan Koo
Founder
Writer/Director

Have they fixed the hdmi output problems for the T2i yet? That was the only thing putting me off magic lantern tbh. Although I am shooting stop motion for the next year anyway, so I guess it doesn't matter.

December 22, 2011

1
Reply

How is this different from using Picture Styles, for instance, Cinestyle, to get a better dynamic range out of your camera? At least with pictures styles, you don't get jittery footage...it's an actual adjustment to the footage itself rather than combining exposures taken every other frame...doesn't this make more sense and work better overall?

December 22, 2011

0
Reply
Jake

It's different because the sensor can only capture a limited amount of color data in a single frame, and the encoder can only store a limited amount of color data in a single frame. By using two streams you can get (in theory) twice the dynamic range you can get with any Picture Style as well as twice the color depth (that is, 16 bits per color).

December 22, 2011

0
Reply
cows

Not really. From what I've seen (I might be mistaken) it switches between two ISO's at 1 stop intervals. So you don't get double the dynamic range. You'd be getting 1 extra stop in the shadows, 1 extra stop in the highlights. The rest is the shared range between both frames. And two 8bit streams at odd/even frames don't give you a 16 bit output (unless interpolation creates non-existing, non-recorded values), you can't create information that wasn't recorded to begin with.

Again, I might be mistaken, but it doesn't make much sense to me.

December 23, 2011

0
Reply

I assumed that it alternated between the set exposure length and half (or otherwise less than) the exposure length. That'd make much more sense than ISO. You're right in any case that there won't practically be double the dynamic range, but in theory you could design it such that the lowest useful values from the high-exposure correspond to the highest in the low-exposure.

Likewise for color depth. You get 8 bits per color in the high-exposure frame and another 8 bits in the interpolated (fake) low-exposure frame, or vice versa. The end result (assuming the correct workflow in post) is 16 bits per color, again, in theory.

In short: each frame has half the color data. Taking data from the adjacent frame yields (at most) twice the normal color data.

December 23, 2011

-2
Reply
cows

Wouldn't setting the highest/lowest ISO possible introduce grain in one frame, and clean frame in another? That would result in some ugly-looking footage, when the highest ISO possible on a T2i is 6400, I think. I find a similar conundrum with switching shutter speeds. One frame will have less (or more) motion blur than the next, wouldn't that cause weird artifacts?

I downloaded it yesterday, still have to try out the HDR functions. I still think there should be another formula to get smoother motion... maybe combining the frames for HDR to get an HDR 12fps stream (or 15 if working on 30p), then interpolating with Twixtor for smooth(er) motion?

Either way, I think it's an interesting (and quite impressive) addition to the firmware hack. And now that we can force 20fps, I'll probably try out some fast-moving fight scenes, a-la Jackie Chan. :D

December 23, 2011

1
Reply

Why isn't there someone hacking Nikon software so we can get some of this goodness?

December 22, 2011

0
Reply