June 15, 2013

What Would a 360 Degree Bullet Time Replay of LeBron James Dunking Look Like?

NHK Bullet Time camera systemAmazing. It would look -- amazing. And this isn't something I thought up in a wonderful wonderful dream I had. (If it was, it would've been Muggsy Bogues dunking on Hakeem Olajuwon and then high-fiving David Robinson before dismounting off of the backboard in a fiery double gainer.) This is what we could see every time we turn on a game if broadcasters use a bullet time camera system developed by Japanese broadcasting company NHK. The multi-viewpoint remote-controlled camera setup could take instant replays in sports to a whole new -- very exciting level.

Though the concept of bullet time is no new thing (The Matrix being only one, but the most obvious example,) NHK has created a much more versatile and sophisticated setup. Developed specifically for live sports broadcasting, the setup employs an array of 8 motion tracking cameras that record from all different angles of a shot and are controlled by a central camera. Action shots can be more easily captured and render a much smoother image, since the object can be tracked over a larger area. An NHK rep says:

Using this system, you can create the effect of stopping time and moving the viewpoint all around the subject -- Previous methods used a fixed camera, so they could only capture subjects moving in a narrow or limited space. But this multi-viewpoint robot camera system can film dynamically moving sports, or subjects at lots of locations in an extensive space.

Check out the video below to see the camera setup in action.

According to the video, each camera is equipped with two motors that allow it to pan and tilt. The cameras also share lens data that gives them the capability to zoom and pull in unison. An Imaging Resource article broke it down further:

What's unique here is that the cameras aren't just panning and tilting in lockstep, they're rotating by varying amounts, to overcome the extreme parallax between their respective locations, so they're all "looking" at exactly the same point.

NHK Bullet Time camera setupBut, how fast is the turnaround? It seems like producing the final product of these images would be time-consuming, but an NHK rep says, "Pictures from this system can be sent out about one minute after filming is finished." In the NHK press release it states that "faster image processing and automatic editing and feeds moreover allow for multi-view images to be used in a timely manner during a live telecast or sporting event."

There are some obvious drawbacks, or perhaps only concerns regarding using a rig like this. For one: the number of cameras and gear needed to produce one specific kind of shot. Secondly: the amount of data storage needed to capture all of the footage. I'm sure that this is going to be -- expensive.

At least from a viewer's perspective, as opposed to a user's, I'm excited to see this setup in action in a game/match/pretty much any other sports competition other than curling (no offense curling fans.) So, keep an eye out for further developments of this rig. I know I'll be waiting with bated breath every time I watch the NBA or FIFA -- and how great would it be to see a Matrix-style replay of a bicycle kick or unwarranted response to an injury? Until then -- I will dream.

What do you think of NHK's bullet time camera system? Where else would you like to see it be used other than in live sports broadcasting?

Links:

Your Comment

20 Comments

NHK always impress me

June 15, 2013

0
Reply
Ash

They invented the shotgun mic

June 15, 2013

0
Reply
Anthony Marino

Only eight cams? Some folks strung thirty of them together.

http://www.youtube.com/watch?v=-gYVEWbg-gU

June 15, 2013

0
Reply
DLD

The rig itself is amazing and I am sure it will find its way into every major sport event in no time, but what I like even better is packing a pan & tilt motor and zoom lens all into one package, I always think there could be a camera in the future that has everything to run stable, pull focus, etc. in one complete package, think of adding the above into something like a small Movi with a side handle like the C300 and some sort of joystick thumb navigation... anyway, just dreaming away here!

June 15, 2013

0
Reply

..and that is why we have nofilmschool! Thanks for sharing, love NHK

June 15, 2013

0
Reply
Archie

GoPro works great for this kind of viewing. Because of the low cost of GoPros there's lots of potential in sports. Right now the NFL uses one camera on a cable system above the field for overhead shots that can follow the play. They literally could put an array of GoPros above the field, all the way across, from sideline to sideline. Baseball could have an array that views from the pitchers mound to home plate, with the GoPros set at 720p, 120fps, so you could watch slomo replays of pitches.

A channel on YouTube with some GoPro arrays used:

http://www.youtube.com/user/DonahueMarc/videos

GoPro array, drums (there's a behind the scenes video of this one in the channel):

http://www.youtube.com/watch?v=WgXg9b507H4

June 15, 2013

0
Reply
Gene

I don't know if it's the effect or the subject but either way it's boring. I don't think an array would work well in LIVE sports. You can't compare GoPro to the cameras that are used especially the skycam during NFL games. They use a Sony HDC-P1 HD-SDI.

June 17, 2013

0
Reply
Joe

But what an awful video...

June 20, 2013

0
Reply
Myles

Always looks too jerky to me

June 15, 2013

0
Reply
Fresno Bob

I love this effect, but I think it is definitely one of those that if overused becomes distracting. 5 times a game would get silly.

June 15, 2013

0
Reply
Kevin

I know what you're thinking Ryan! ;-)

June 15, 2013

0
Reply
Darren Wolff

Why doesn't it look smooth like in the Matrix? The Matirx made it smooth and they did it like 15 years ago. Also, why don't they "move around" during motion! They kept pausing the video first.

June 15, 2013

0
Reply
Pianohero

The Matrix used a system with roughly 80 - 160 cameras, making for a much smoother move between each camera. Also, heavy post production and lots of budgets and the smartest minds on this technique back in those days = smooth as f...k footage.

June 18, 2013

0
Reply
Elias

Lame headline.

This has nothing to do with LeBron James. The closest thing was an uncoordinated Japanese kid dunking on a lowered hoop from toys r us.

I was expecting an uber coordinated 6'8" black dude throwing down a monster jam on a 10' NBA rim.

June 15, 2013

0
Reply
steve

yeah me too..

June 16, 2013

0
Reply
Fried

and me.

June 20, 2013

0
Reply
Myles

Me three! We'll do better in the future.

June 20, 2013

0
Reply
avatar
Ryan Koo
Founder
Writer/Director

Damn... I already gave away my Heat season tickets thinking he was washed up.

June 21, 2013

0
Reply
Zan Shin

So does it have to freeze frame to move between the cameras? I'd like to see the footage keep moving as it cuts down the line of cameras.

June 20, 2013

0
Reply

Interesting, but they should work more on exposure and color harmonization.

June 21, 2013

0
Reply
hellsing