September 2, 2013

Edit like You're Tom Cruise in 'Minority Report' with Leap Motion Controller

Video thumbnail for youtube video Editing 'Minority Report' Style: Editors Keys' Gesture-Based Interface for Final Cut Pro - No Film SchoolWe've seen a few early attempts at touch-based editing interfaces, using devices like the iPad. It may take some time for touch-based tools to mature into something better/faster than good ol' fashioned keyboard input, if ever, but the developments are interesting to say the least. Now, Editors Keys are working on a totally gesture-based system for the Leap Motion Controller, for 'lift of a finger' interfacing with Apple Final Cut Pro X, Adobe Premiere & Photoshop, Avid & Pro Tools, and beyond. As always, editors will have to judge whether Minority Report-style control over their NLEs is viable. That said, it will be quite fascinating to see where technology like this goes. Read on for a demonstration by Editors Keys.

First, here's a brief intro to the Leap Motion Controller (retail $80), courtesy LM's YouTube channel:

http://www.youtube.com/watch?v=_d6KuiuteIA

Next is a YouTube review of the device in general by Engadget. The full-text review asserts the following, as of July 22, which may be useful to keep in mind (my emphasis):

It's clear that it's not cut out to replace a touchscreen or mouse as a primary input device. Not yet, anyway. Some developer may well figure out a way to take full advantage of the Leap's capabilities with a novel UI, but for now, it's best suited for creative pursuits, not productivity.

http://www.youtube.com/watch?v=xzn-mLEjRZw

None of which amounts to a "this and only this is the future -- NOW!" type of endorsement by any means, but again, this is all very young. It's probably worthwhile to read and watch with a fair amount of skepticism at present, but the review also underlines Leap Motion's future potential.

Finally, with kudos to Wolfcrow for the find, here's Editors Keys on YouTube with their custom app for the Leap Motion -- still a work in progress, for sure:

http://www.youtube.com/watch?v=tDQODtxSAKM

Not convinced? I'm sure you're not alone. This tool will surely come with its growing pains, to be felt by both developers and early users alike. Things can only be improved as time goes by, though, and I'm sure we'll be hearing more about this tool, or ones like it, further down the line.

I'm not entirely convinced, but I do think it's amazing we can even take this into consideration. At the very least, I think Leap Motion and Editors Keys should be commended for their efforts in ironing out the possibilities that gesture-based interfaces may hold.

Editors Keys has responded to comments on YouTube, including the following:

We're developing this app further so -- it will learn the degree and speed at which you shuttle through video to make it very accurate. We're thinking of doing some two hand gestures too.

The developers at Editors Keys seem quite receptive to feedback, so if you have any constructive comments to make, be sure to let them know! The app for Leap Motion Controller doesn't seem to have a name yet (that I could find, at least,) nor has any release date been announced. According to PremiumBeat, "The finished app should support over 50 unique gestures for video editing and retail for 99 cents."

What do you guys think? Will the lack of physical feedback prove to be a deal breaker for editors? How could a Leap Motion-style interface complement your working style? How do you envision the future of technologies like this benefitting editors and filmmakers?

Links:

[via Wolfcrow]

Your Comment

31 Comments

o.0... that is all. Seriously this is revolutionary. I can't wait to see how this pans out.

September 2, 2013 at 10:40PM, Edited September 4, 11:21AM

16
Reply

Although, I have to say, some of these gestures seem to make the editing take longer than simple mouse clicks or pen movements.

September 2, 2013 at 10:43PM, Edited September 4, 11:21AM

0
Reply

Agreed. Some are cool but most gestures seem to actually require more energy and take longer than just clicking a short cut.

September 3, 2013 at 2:14AM, Edited September 4, 11:21AM

0
Reply

After an hour or so with your arms in the air, you'll have a a huge pain in the back, trust me

September 2, 2013 at 10:48PM, Edited September 4, 11:21AM

1
Reply
Daniel

It'll take a few years for this to be any good. There's some serious lag there...

September 2, 2013 at 11:28PM, Edited September 4, 11:21AM

0
Reply

Too imprecise. You'd have to be in front of an 80 inch screen to really be able to take advantage of such a program; you would really have to go full on "Minority Report" for this to work...

Unfortunately, this might be fun to use if you're just having fun, but to edit something complex "Minority Report" style would become tiresome quickly.

September 2, 2013 at 11:48PM, Edited September 4, 11:21AM

0
Reply
neil

It seems like a stylus on a touch screen would be superior with the visual image editing while a regular mouse should do the trick with bars, files and arrows.

September 2, 2013 at 11:54PM, Edited September 4, 11:21AM

0
Reply
DLD

i could see stuff like this being more useful in audio, specifically surround sound. You take your finger and move it around in the 3d space while the program calculates the speaker levels.

for video, i agree your hand would get tired. Maybe lay the device down (which i think i read is possible) and turn your desk into a touch surface. do all the gestures that way. If it was right beside my mouse i would use it more than if i had to move my hands all over IMHO.

September 3, 2013 at 12:08AM, Edited September 4, 11:21AM

1
Reply
kevin

It would be better to have the sensor pointed up, so you can have your hands down as you gesture.

Ideally you'd want to combine gesture with a multi-touch screen. It would be interesting to handle a timeline like you would film.

September 3, 2013 at 12:51AM, Edited September 4, 11:21AM

0
Reply
Vizar

I agree with a couple of the previous comments. While this may be "cool," it's simply not a practical as demonstrated. Technological advancements in tools are meant to make things more efficient, not less. This requires significantly more physical movement and therefore also more time per action. It just doesn't make much sense in actual use. The gestures may be more intuitive, which could potentially decrease the learning curve of editing software (though I can't imagine by much), but for any serious, or even just enthusiastic amateur editors, a bit of up-front learning of hot keys will be drastically more productive than this. I do, however, commend the exploration of the tech and look forward to seeing it implemented in a useful way in the future.

September 3, 2013 at 12:58AM, Edited September 4, 11:21AM

3
Reply

Rather bring something like ctrl+ console to android!

September 3, 2013 at 1:18AM, Edited September 4, 11:21AM

0
Reply

Maybe it can be usefull right now, if we combine Leap Motion with a regular mouse. For example you can use your right hand to control mouse as usual, while scrubbing with your left hand using Leap Motion.

September 3, 2013 at 3:03AM, Edited September 4, 11:21AM

0
Reply
Ruslan Randzhabar

mouse clicks rules for ever... Leap motion for kids only....

September 3, 2013 at 3:07AM, Edited September 4, 11:21AM

2
Reply
Sergey

While I too feel that this will ultimately be left at the wayside like most motion-controllers for console gaming (it's fun for a few days, but then you find yourself reaching for the dualshock and forgetting all about motion-control) I also remember that Minority Report was the first thing I thought of when I first saw the Leap-videos.

However. The ultimate combination would be to use Leap for gesture-commands and Occulus Rift for displays. Surfing the web in the ultimate style and I WANT ROOMSERVICE!!!

September 3, 2013 at 5:08AM, Edited September 4, 11:21AM

0
Reply

I have a Leap Motion controller and it's a piece of junk. It takes half a dozen attempts to get it set-up whenever I want to use it and the reaction times are so slow it's almost impossible to use. To add insult to ninety quids worth of injury their tech support sucks and it took me very publicly outing them on social networks before they responded to my requests for help, even then their advice was rubbish and hasn't worked. Any suggestion that they might replace the unit has been met with stony silence. Seriously, avoid them like the plague.

September 3, 2013 at 7:05AM, Edited September 4, 11:21AM

13
Reply
Robert

It sounds like you've had a rough experience, which is unfortunate. But from my perspective, in the history of human innovation this is still an incredible evolution forward for this type of technology. The lesson here may not be that LEAP doesn't work so well, but that rough edges come with the territory of being an early adopter. I'm incredibly excited about this technology and could easily see myself using it to edit more rapidly or at least in complement to driving with the keyboard, but I understand that the technology may not quite be there yet. To put down LEAP and not take into account the positive aspects of what they've accomplished doesn't seem fair.

September 3, 2013 at 3:00PM, Edited September 4, 11:21AM

0
Reply

Everything is always " an incredible evolution / Leap forward" until it is realized its always related to ,or IS the same thing over again. Just buy again what you already own in How many formats now?

September 5, 2013 at 8:24PM, Edited September 4, 11:21AM

0
Reply
Dheep'

I was on the preorder, and it took several times for them to get my credit card info correct. Then it never came and they still didn't have correct info, so I just cancelled my pre-order. Then I got two in the mail and they billed my card twice after the cancellation. It took over a month to get my money back after talking to at least 8 different reps. Doesn't bode well IMHO.

September 5, 2013 at 6:03PM, Edited September 4, 11:21AM

0
Reply
Blasst

Meh. I work in the video game industry and even coming up with practical applications for the Kinect and Sony Eye Camera in games where motion control isn't necessary is difficult at best and gimmicky at worst. (I won't include the Wii or Wii U in this discussion because there you don't have a choice but to have an idea first, then implement it, my studio makes games that were franchises before the whole motion control movement) This I would absolutely categorize as gimmicky. Like most games, it's just not practical, even less practical than gaming. At least we can assume our users are in a living room or a room that allows room for movement. Many editors work in cubicles. I work in a cubicle, there isn't a lot of room for movement. It makes testing various motion control applications difficult because of your limited space to move around in. I couldn't imagine using such a system in a cubicle 8+ hours a day. Our telemetry also indicates that once the novelty wears off, most of our users don't even use the motion control features and go back to the controller. I would imagine the same would happen here. After a week of such nonsense editors will revert back to what's comfortable for them.

September 3, 2013 at 1:17PM, Edited September 4, 11:21AM

0
Reply
Amy

The difference is that film editing has a long history of tactile work, namely, editing on film itself. For those of us who cut our teeth editing on film, this sort of thing is really really great and welcome.

September 3, 2013 at 2:59PM, Edited September 4, 11:21AM

5
Reply

This is what I've been dreaming of ever since Apple started delving into multitouch. Want it right now.

The best would be an iPad satellite interface. You could run all the media, etc. off a base machine and then use the iPad on the go as a control surface, connecting to the base machine on your home network or even through the internet at large, editing on the go but not containing any of the media on your iPad.

Still. This is enough to make me want to get a Leap.

September 3, 2013 at 2:58PM, Edited September 4, 11:21AM

0
Reply

I would add, the more the gestures can mimic the old motions of cutting on film - the splicing, taping, joining, pulling of film from the reel, etc. the better and more intuitive it will be to use. This is a good start though.

September 3, 2013 at 3:02PM, Edited September 4, 11:21AM

1
Reply

I have a leap motion and it's certainly exciting to see the potential it has or at least that the technology has birthed an opportunity for motion editing.

Some apps have way better response with the sensor. I think the device itself is capable of a lot but it's up to the programmers to make sure their apps actually work the best.

I'd like to see better UI for the apps that's for sure.

September 3, 2013 at 4:08PM, Edited September 4, 11:21AM

7
Reply

I think a touch screen would be much more ergonomic. Manipulating something with your hands feels more precise than waving them in the air. Even if a cursor were to be implemented, it just feels easier to touch what you want, rather than hold your hands up and move them into position.

That said, the best way may even be to use the two technologies together. For example, a flick gesture to make the image full screen (cinema mode in Premiere), or a gesture to change tools; but then actually editing the project by touching the screen.

September 4, 2013 at 2:29AM, Edited September 4, 11:21AM

8
Reply
Ok

I can see it to be very useful in many other fields as for editing, I'll stick with my mouse, keyboard, shuttle and my Wacom tablet.

September 4, 2013 at 4:20PM, Edited September 4, 11:21AM

0
Reply
PayDro

For editing, a better invention would be a touch screen that lays at a 35 degree angle on your desk that you can make tactile edits with.

September 4, 2013 at 5:30PM, Edited September 4, 11:21AM

5
Reply
Steve

I tend to agree with this perspective—for tactile control and for the sake of your arms! Unless you are a pro boom operator AND an editor, motion control is gonna wear your arms down fast! ;)

September 4, 2013 at 5:36PM, Edited September 4, 11:21AM

8
Reply

wtf

why is everyone saying that this is revolutionary

-helloooo-

We've had motion detection for years. This is stupid. LEAP is a waste of time. A bandwagon will just ride past and everyone will forget it. Technically it shouldn't even be a bandwagon. It's something riding on the motion control bandwagon. Bandwagonception.

September 4, 2013 at 8:07PM, Edited September 4, 11:21AM

4
Reply
Tyler

To quote a wise old Proverb: "If it ain't broke ,don't fix it"

September 5, 2013 at 8:21PM, Edited September 4, 11:21AM

4
Reply
Dheep'

The main problem probably is that they want to bring back a tactile feeling to film-editing... by having us edit in the air without any tactile sensation.

But also, what made me really dissapointed in videos of this app and others is how strange the gestures have been made. I absolutely try out the Leap-PPro combo, but jogging by spinning finger constantly? none of those gestures feel any sort of intuitive. Multi-touch really took off because how easy they were to learn. What these guys need is to actually look at movies like Minority Report... and just copy those gestures. I'd accept having to wear the glowing dots-glove just for the heck of being able to do the edit-dance that Cruise showed. Though... physical buttons are hard to beat when it comes to real memorizing of commands.

September 6, 2013 at 5:16AM, Edited September 4, 11:21AM

0
Reply

We spend all day editing using mouse and keyboard, Sitting in a chairs. Our bodies have not evolved to do this. The leap motion and other devices that follow it, will work out the bugs to the point we can operate, standing up. Moving our limbs like in Tai Chi. Good for Posture, good for muscle strength, good for respiration. I'm looking forward to it. Time for a walk now.

September 29, 2013 at 8:02AM, Edited September 4, 11:21AM

9
Reply
Digital Allstar