Description image

With New Magic Lantern Firmware, Canon DSLRs Get Real-Time HDR Video*

12.21.11 @ 12:28PM Tags : , , , , , , , , ,

Is real-time High Dynamic Range imaging, ala RED’s HDRx, coming to Canon DSLRs with the next version of Magic Lantern firmware? Well, yes and no — thus the asterisk. The system they’ve developed splits a 24p stream into two 12FPS exposures, one high and one low, and then interpolates the frames. It’s an interesting hack, demonstrated here:

It should be available tomorrow for the Canon 60D, T2i, and T3i. More details from the Vimeo User Group:


You will get two streams at half frame rate. For example, if you record 25p, you will get two 12.5fps streams. Interpolate them to 25fps and use your favorite HDR software to blend the frames. Our workflow is cross-platform and uses only free software: enfuse, AviSynth, InterFrame and VirtualDub… Will it work on 5D Mark II, 50D and 500D? Maybe. Right now there are some synchronization issues on these cameras. More details coming soon.

While this has a limited application, it looks like an interesting addition, and quite frankly even RED’s more advanced HDRx exposure hack has limited capabilities when shooting motion. More on that soon.

Also coming in the next firmware: variable frame rates. They are limits and the cameras can overheat quickly, but here’s a peak at the feature:

Link: Magic Lantern Wikia page (HDR feature should be available as of 12/22)

[via CheesyCam]

COMMENT POLICY

We’re all here for the same reason: to better ourselves as writers, directors, cinematographers, producers, photographers... whatever our creative pursuit. Criticism is valuable as long as it is constructive, but personal attacks are grounds for deletion; you don't have to agree with us to learn something. We’re all here to help each other, so thank you for adding to the conversation!

Description image 40 COMMENTS

  • Hmmm… I’m going to download it no matter what. I just hope I can keep my sanity

  • UFFFF!!! But why no 7D?!?!? =( i want the hack on it!!!

    • I have been following the Magiclantern development for a little over a year now, and Sadly the 7D has been made VERY hack-proof, to the point of many firmware pioneers’ 7D’s being “Bricked” in their hacking efforts. Canon probably upped the security when designing their 7D for the fact that it has dual Digic 4 image processors, whereas all of the other ML-capable cameras (to the best of my knowledge) have one image processor. So, considering that all of these magical and wondrous things are achievable with one processor, imagine what one could do if a dual processor camera was unlocked…. It would make the 7D quite a formidable (even more so) device, competing more aggressively with their own (proper) video cameras. Who knows if Canon did this out of foresight, thinking that someone like Trammel, Alex, and company would make such strides with a hacked firmware, or if they were simply escalating security in proportion to such significant hardware upgrades.

      Disclaimer: I myself am merely an avid follower of the ML development group, and could have missed a recent update stating “7D development breakthrough!!!!”, but I highly doubt it, considering what the current data suggests. BUT, those of you possessing one of the listed, ML compatible cameras, I would highly recommend giving the latest stable release a try, I know the firmware has been featured here before, but here is a link to the wiki-page: http://magiclantern.wikia.com/wiki/Unified

      Have fun with the coming HDR* feature!

      • John Jeffreys on 12.21.11 @ 2:55PM

        Nothing is imposible, yes the dual digic 4 system is locked down (and canon wont spill the beans about how to work with it) but i have faith in the community to make a breakthrough soon

        • DavidJFulde on 12.26.11 @ 9:44AM

          It also has to do with the fact that the firmware updater has a limit, so even trying to crack the firmware is difficult because they can only update it X amount of times; then they can`t even upgrade Canon`s firmware.

          • DavidJFulde on 12.26.11 @ 9:46AM

            I mean: They can`t even upgrade with Canon`s firmware. Once the limit is reached; they are stuck.

  • Rev. Benjamin on 12.21.11 @ 2:39PM

    Soooooo… that’s kind of neat, and also kind of dumb. I will not be making any film at 12.5fps. Perhaps if you needed some fast-motion HDR, you could doublespeed it? Other than that… bleh. What if they did this same thing with a 720p60 mode, though?

  • Meh. (disclaimer, I’m on a GH2 now)

    For scenes with little or no motion, you can probably get away with it, like landscapes. But if there is motion, as evidenced in the sample video, you get “weird jittery frame incongruity artifacts.” The result of interpolating 2 12fps streams.

    Better to do this though and push the envelope. However, those jitters are a non starter for anything I’d shoot, professionally or personally.

    • I’ve done a few little experiments with the HDR feature and it’s pretty good !

      (Unless there’s ANYTHING moving in your shot. Then it’s crap).

    • DavidJFulde on 12.26.11 @ 9:46AM

      if you shoot in 720P 60 (or 50 or 48) the jitteryness is FAR far less and is actually usable!

  • Hopefully a 720×60/50fps option IS enabled, and who’s to say that the workflow they have demonstrated is the optimum, maybe (as with so much of ML) the development team, or someone else, will surprise us all and figure a way to at least smooth some of the blemishes.

    Though, at worst, this is a new way to experiment with the video capabilities of a camera that is meant primarily for stills. I feel that any developments past the stock features, are gravy on top of an already delicious meal.

    • DavidJFulde on 12.26.11 @ 9:48AM

      It is possible. You can shoot at 720P 60 (For 30FPS) 720P 50 (For 25) or even 720P 48 (For 24)>>There are alternate workflows, you can look them up on Vimeo (I have one that I feel is pretty good. Be sure to read the desc.)

  • If I can shoot even 20 second shots at 50fps, this will be amazing.

  • How about a stable ML for the 5D instead?

    • DavidJFulde on 12.26.11 @ 9:48AM

      It`s being added to the Unified tree to have all of the features of the T2i, T3i, 60D etc. soon!

  • Britti Faija on 12.21.11 @ 8:08PM

    So cool! Gotta try this asap when it comes out.. I’ve got a 550D, do I need some software to combine these two videostreams to one hdr video?

    Here’s my last work with the 550D: http://www.youtube.com/watch?v=9Mm8QyuRlDk

    • DUDE that video is SIIIICK!!!

    • Hey. I just had a great time watching that video. What I am not a fan of though is how you can see a equipment (camera on a tripod…6 times) in the frame. You should have put that away. It really distracts. If it’s meant to be in the video…then I don’t get why. Otherwise, fun stuff, keep it coming.

    • DavidJFulde on 12.26.11 @ 9:50AM

      Philip Bloom<s blog has a handful of different workflows with different software.

  • This will, I think, be extremely valuable. The scenes where I really notice clipped highlights or shadows are typically not fast-paced action scenes (and honestly DSLRs aren’t very good for fast-paced action scenes in the first place). I’m very excited about this. I think now that there are reasonable algorithms for frame interpolation, there are going to be a lot of interesting software developments like this. Things like Twixtor and the Warp Stabalizer, for instance, do totally different things, but they’re both based on similar shape/edge tracking systems. I imagine there are still a lot of untapped applications of this.

    Speaking of which, does anyone happen to know if the “line skipping” on DSLRs is under the control of firmware, or is it built directly into the chip? I think that if it were possible to alternate which lines are read every other frame it might be possible to extract a significantly higher-detailed image at the end. Of course, long GOP compression could completely eliminate any benefit, but it’s an interesting thought.

    • Oh, one other thing. It looks like they’re interpolating the frames on the two “streams” independently of each other, which seems to work basically fine. It’s possible (in theory, at least) to get an even better result by taking into account, in the weighted interpolation, the data from both streams simultaneously.

      That is to say, when getting the “between” frame from two high-exposure frames, you can take into account the information from the appropriate low-exposure frame (which is chronologically in exactly the right position as the high-exposure “between” frame) as well to get a much smoother result.

      The right algorithms to do this already exist but I’m doubtful that there’s an implementation that could handle this specifically. Still, it shouldn’t be that hard to do. Maybe in my vast quantities of imaginary free time I’ll try to make a proof of concept. I’m not sure that it would perform substantially better than generating the between frames from each stream independently, but it couldn’t possibly perform worse.

  • This will only be able to be used on shots with very little movement, look at the man’s legs as he walks in the second example his leg actually vanishes every other frame.

  • This is awesome news. Now I can rewrite my article on how to achieve HDR video without having a RED http://www.reelseo.com/hdr-video-3d/

    Until now, the only way to achieve this was by combing multiple exposures from different cameras on a 3D rig, or using simultaneuos exposures.

    Hopefully Canon is paying attention.

  • I don’t wonder if it would work shooting at 50/60fps. Could make it very useful indeed.

  • Hey I use Magic Lantern and I love some of the features.
    I’m not sure if these will get much use but if you use the 2 new features together couldn’t you shoot at 50fps and end up with 2 24fps streams?

    Cheers

    Rob

  • Have they fixed the hdmi output problems for the T2i yet? That was the only thing putting me off magic lantern tbh. Although I am shooting stop motion for the next year anyway, so I guess it doesn’t matter.

  • How is this different from using Picture Styles, for instance, Cinestyle, to get a better dynamic range out of your camera? At least with pictures styles, you don’t get jittery footage…it’s an actual adjustment to the footage itself rather than combining exposures taken every other frame…doesn’t this make more sense and work better overall?

    • It’s different because the sensor can only capture a limited amount of color data in a single frame, and the encoder can only store a limited amount of color data in a single frame. By using two streams you can get (in theory) twice the dynamic range you can get with any Picture Style as well as twice the color depth (that is, 16 bits per color).

      • Not really. From what I’ve seen (I might be mistaken) it switches between two ISO’s at 1 stop intervals. So you don’t get double the dynamic range. You’d be getting 1 extra stop in the shadows, 1 extra stop in the highlights. The rest is the shared range between both frames. And two 8bit streams at odd/even frames don’t give you a 16 bit output (unless interpolation creates non-existing, non-recorded values), you can’t create information that wasn’t recorded to begin with.

        Again, I might be mistaken, but it doesn’t make much sense to me.

        • I assumed that it alternated between the set exposure length and half (or otherwise less than) the exposure length. That’d make much more sense than ISO. You’re right in any case that there won’t practically be double the dynamic range, but in theory you could design it such that the lowest useful values from the high-exposure correspond to the highest in the low-exposure.

          Likewise for color depth. You get 8 bits per color in the high-exposure frame and another 8 bits in the interpolated (fake) low-exposure frame, or vice versa. The end result (assuming the correct workflow in post) is 16 bits per color, again, in theory.

          In short: each frame has half the color data. Taking data from the adjacent frame yields (at most) twice the normal color data.

          • Wouldn’t setting the highest/lowest ISO possible introduce grain in one frame, and clean frame in another? That would result in some ugly-looking footage, when the highest ISO possible on a T2i is 6400, I think. I find a similar conundrum with switching shutter speeds. One frame will have less (or more) motion blur than the next, wouldn’t that cause weird artifacts?

            I downloaded it yesterday, still have to try out the HDR functions. I still think there should be another formula to get smoother motion… maybe combining the frames for HDR to get an HDR 12fps stream (or 15 if working on 30p), then interpolating with Twixtor for smooth(er) motion?

            Either way, I think it’s an interesting (and quite impressive) addition to the firmware hack. And now that we can force 20fps, I’ll probably try out some fast-moving fight scenes, a-la Jackie Chan. :D

  • Why isn’t there someone hacking Nikon software so we can get some of this goodness?

LEAVE A COMMENT