March 24, 2014

Do Human Eyes 'See' Like Cameras? A Look at the Resolution & Frame Rate of Vision

VsauceCameras are the "eyes" of cinema, recognizing, capturing, and processing images at certain frame rates and resolutions. But, what about our own eyes? At what "frame rate" do we process images and at what resolution? In these excellent videos, Michael Stevens, host of everybody's favorite YouTube science channel, Vsauce, shows us how our eyes compare to cameras, not only in how well they "see", but also in how they "record" images.

In case you haven't heard, science is awesome, especially the science of cinema. We've talked before about how rolling shutter isn't just something that annoys cinematographers, but is a natural distortion that affects the way we see our universe. This time (and again with a video from Vsauce), we take a scientific look at resolution and frame rates. At what resolution do we see the world with our eyes? Do we see in "frame rates"?

First of all, our eyes and brains process images differently than lenses and cameras. In this first video from Vsauce, which explains the nature of "video", Stevens talks about the difference in how our eyes receive information and then communicate that information to our brain versus how lenses and cameras do it. A motion picture camera captures single still images that are later played back sequentially at a high enough frame rate that they appear to be moving, an effect called "beta movement". But, the way our eyes work is very different. Stevens says:

Our eyes are not cameras. Instead, they track onto objects and receive a continuous flow of photons onto the retina, sending information via a chemical reaction to the brain.

The resolution of our eyes and that of cameras is also fundamentally different. When a camera captures an object, it captures it in its entirety to produce an image. Our eyes, again, don't capture a single image, rather a flow of continuous images, but even more than that, we receive the most visual information from our central visual field (thanks to the fovea), and only there is "optimal color vision" and "20/20 acuity" possible. What does all of that mean? Well -- it means that our vision is limited, but it is also, in a way, aggregated by our brain from different sources and with varying methods in order to make sense of the world around us.

Learning this stuff isn't just good for a little entertainment -- it can actually help you see (or even develop) what could be the next revolutionary technology in the cinematic world. Considering how technologically advanced cameras and lenses have become in just a short amount of time, I wouldn't be surprised if we saw developers trying to emulate our eye's system of image processing with something like -- "non-frame rate recording" -- "∞ fps" -- "continuous photon something something capture".

What do you think about the information from the Vsauce videos? Let us know in the comments below.

[via Vsauce]

Your Comment

44 Comments

I find the eyes' color rendition a little 'plasticky' though; we really need to see a ocular vs RED comparison to know which is better

k done trolling. and drinking. night!

March 25, 2014

-1
Reply
loveit

I have always had a hard time trying to figure out what "focal length" our eyes are set at. On one hand it feels like a fish eye (your peripheral vision) but the object you're focusing on appears at more of a 40-75mm FOV. It's like we have the ability to see very wide and yet we can zoom (focus) at the same time. Bizarre thing the human body.

Also, what is the evolutionary purpose behind us not having macro focus? Our minimum focus is about 4 inches. Why?

March 25, 2014

-1
Reply

I read somewhere that we see at around a 50mm f1.0 equivalent, but as if a picture were taken in a panorama horizontally and vertically, super wide but without any distortion and with a very shallow DOF

March 25, 2014

-2
Reply
criles

Yeah, that seems about right. I would have guessed 50mm but then I started to think about "sensor" size. Lol. Guess that's relative from human to human. :)

March 25, 2014

1
Reply

I would think the eye equivalent DOF is greater than f1.0 at 50mm. my eyes at least, have deeper DOF than that

March 25, 2014

1
Reply

I wonder if he's just talking about the ability to see in low light only. Another odd thing is that at night, when our iris opens up, there doesn't seem to be any change in DOF. Wish camera lenses could figure that one out.

March 25, 2014

-1
Reply

The stack array sensors that will be part of the next generation smartphones can put several planes in focus simultaneously. This design is a lot closer to the human eye - at least, a healthy human eye - than a photo sensor has been over the last 100+ years.

March 25, 2014

1
Reply
DLD

I bet that could have big implications on dynamic range as well. I was just thinking tonight how easy it would be to do real time HDR if you could just somehow have the CPU recognize what "white point" is and use the other exposure for that part of the sensor. Almost like the tech behind "zebras" in combination with multiple exposures. Seems like it's not too far off.

March 25, 2014

1
Reply

@Luke Neumann ...when our iris opens up - Also don't forget about the biological pigment "visual purple" or Rhodopsin in low-light conditions as it is extremely sensitive to light. Yes, the human eye is an amazing mystery at times, and truly a marvel of God's design.

March 25, 2014

2
Reply
Razor

I believe that this generation - soon to be hitting the markets, purportedly - of the stack array sensors is in three layers, sort of like Sigma Foveon. Each layer can focus independently of the others. The similarity with HDR is that a photographer can use multiple exposures/shots to simultaneously focus on more than three objects with the camera video processor then putting together a final collage, as it were, to have all the selected objects in focus. I do wonder though if something can be done with the electronic shutters and ND filters. There was a company working on the e-ND filters back in the mid-00's but it seems to have gone out of business a long time ago.

March 25, 2014

0
Reply
DLD

There's a change in DOF with the eye's pupil opening up. Trust me.

Your eye rapidly refocuses on what ever occupies the center of your field of view. And when you are young the lens is flexible enough that it can easily accommodate from very near to infinity, so you think there is deep DOF, but it's really an illusion.

As you age the flexibility of the lens decreases. This is why old people like me normally wear bifocals. Because with a "normal" fixed focal length glasses we can't handle everything from seeing the road ahead of us to reading the map beside us, day and night. And you do discover that with the pupil wide open what is a difficult problem during the day becomes impossible at night.

March 25, 2014

0
Reply
Mr. Bill

Odd... my eyes seem to have macro focus XD I can literally see the many textures on the skin of my hand close up.

March 25, 2014

1
Reply
Franco

Really? Dang. I can't focus past about 4 inches.

March 25, 2014

1
Reply

check it out http://en.wikipedia.org/wiki/Myopia

Near sighted = stronger close up vision. Far sighted and you can't focus closely. I can focus almost a half inch from my eyes, but can't see diddly past 8-10 inches. That's OK because I don't need glasses with camera eyepieces

March 25, 2014

0
Reply
Zack

Amazingly I can focus in the floaters in my cornea, those small tears on the eye, since I was a teen, and can even draw them with the transparency and fine outer lines that make them look like worms, some curled on themselves. I thought everybody was seeing their own tears .

March 28, 2014

1
Reply
Agni Ortiz

tears as in Tearing, the act of breaking apart...small breakage, not the liquid tears.

March 28, 2014

0
Reply
Agni Ortiz

Human eyes cannot Macro because the convergence point between the field of view produced by both eyes, being interpreted by the brain, has limits.

March 25, 2014

1
Reply
Steve JL

Might be more than you care to know, but here's the lowdown on the eye vs. a camera from years of my own research:

The actual true focal length of the human eye is somewhere between 17-24mm (depending on genetics), with a maximum pupil (aperture) size of around 8mm, making for an average maximum speed of f/2 for most people. A "normal" lens is one in which the focal length is equal to the diagonal measurement of the sensor size, so for 35mm "full-frame" cameras, a 50mm is not actually an accurate simulation of human vision, but rather a 43mm lens... and in Super35 or APS-C sensor cameras, a 24-28mm lens provides the same field of view.

In regards to focus, while most camera lenses change focus by changing the distance between the lens elements and the sensor, the eye actually changes the focal length of the lens itself by flattening or expanding it through muscles attached to its circumference. The flexibility of the lens (which changes with age, nutrition, etc.) determines how close the eye can focus, but given the extent to which the muscles can flex and the flexibility of a child's lens, the healthy human eye is designed to be capable of focusing from 250mm to infinity.

Due to the eye's lens being composed of transparent cross-linked protein fibers, rather than a single continuous mass like a glass lens element, some diffraction results causing points of light to radiate many little "spikes" (look at a streetlight in the distance at night to see the effect best), and for the eye's bokeh to have patterns within it rather than being an evenly smooth blur. In addition, due to reflections off the retina (seen in the infamous red-eye effect in photos), the eye has fairly severe veiling glare with a pronounced red cast. On the upside, lens flare is practically non-existent, and when the conditions are severe enough for it to show, the most anyone will ever notice is a single tiny greenish circle... not a series of multicolored and often polygonal ones as in a camera lens.

Dynamic range is tricky due to automatic pupil size changes in response to light, making the effective DR of a human eye somewhere around 24 stops, as well as the fact that the retina will adapt to bright or dark conditions, so if you walk outside from a dark room your DR might be 12 stops, but within about a minute it will level out to around 18, with extremely smooth and organic highlight roll-offs.

The eye has no grain or noise at any retinal "ISO" level, however the photocells can sometimes fire randomly in response to pressure or certain chemicals (aka seeing stars after getting hit in the head, pushing on your eyeballs, when light-headed, buzzed on caffeine, etc); look up phosphenes for more info. However, this effect is a random dancing chroma noise more akin to digital chroma noise than anything else, but again, it is no more noticeable under a moonlit forest than in a midday desert.

March 25, 2014

0
Reply
Darren

Thanks for sharing this!

March 26, 2014

-2
Reply
Dan Findlay

But the eye loses the ability to see color in low light conditions right? Because of the cones requering more light to work properly.

March 26, 2014

3
Reply
Jonas

Is that because of the eye or just the fact that color needs light? Down at a certain depth in the ocean color is completely absent until you have some form of light to bring it out. Obviously the color is still there but I wonder if it's our eyes that need the light to produce the colors or the fact that color cannot exist without light.

Confusing myself now. I think this is one of those "if a tree falls and no one is there" scenarios.

March 28, 2014

1
Reply

"what is the evolutionary purpose behind us not having macro focus?"
Evolution doesn't really work like that. You have to consider what the evolutionary advantage of having macro focus would be. It wasn't a trait that would have increased the chance of survival and reproduction in our ancestors so it was never really developed.

May 31, 2014

1
Reply
Cameron Ganjani

I think it's cool that there is a "shutter speed" element to our vision. We see motion blur pretty regularly. And yes science is so awesome!

March 25, 2014

0
Reply
Peter Staubs

Infinite fps is an interesting concept in terms of the brains interpretation of the visual input.
I was recently hit by a car while riding my bike and the very second I was hit felt like several seconds..... Like a speed ramp down which was then replaced with a ramp up to normal speed again.
It felt like my brain was running super speed.... Maybe to prepare my body for the imminent clash with the ground. Intersting feeling, but one thing I did notice was that the one second was far from lossless.... I wonder if camera tech could then be lossless if utilizing the same concept.

March 25, 2014

0
Reply

I found in sports that if you got to a certain skill level you could do this at will in games. The people that get really good at sports can do it more often than others (LeBron for instance, that guy probably plays the whole game in slow mo). It's pretty crazy!

March 25, 2014

0
Reply

There was a prolific (West) German soccer forward/striker of the 60's and the 70's named Gerd Müller. Nicknamed "Der Bomber", he once said, "If you have to think, you're too late". So I once brought this idea up with a prominent NFL writer who said that, when he covered the beat, Dan Marino and Barry Sanders were two players that had that type of vision/brain processing.

March 25, 2014

-1
Reply
DLD

?? Er, WTF?
lol

March 25, 2014

0
Reply
Larry

I think it's true! Even for Quarterbacks. They aren't actually thinking but relying on muscle memory and are reacting.

March 25, 2014

-1
Reply

It's also true of the fighter pilots, especially of the WWII era. Many of them took quite a long time to get used to the speed of an aerial dogfight ... if they had lived through their early careers to begin with. Planes were capable of going as fast as 350 Mph - faster in a dive - early in the war and about 100 Mph faster near the end. Some engagements were head-to-head, where the combined speed was above 700 Mph. Then one had to figure out the shooting/deflection angles too.

March 27, 2014

-1
Reply
DLD

Modern Fighter jets come with individual personalized helmets that control the plane and weapons by the pilots eyesight. It is like thought control.

March 28, 2014

-2
Reply
Agni Ortiz

Infinite fps is an interesting concept in terms of the brains interpretation of the visual input.
I was recently hit by a car while riding my bike and the very second I was hit felt like several seconds. Yes, but what does any of this have to do with 4k?

March 25, 2014

0
Reply
Gene

I was in a brutal motorcycle crash when I was 18 and I also remember the stop motion effect you described as if it just happened, few minutes ago. Those are the moments of true awareness. Life is permanently like that, but we live in the Matrix now, a constant illusion of subjective perceptions and pseudo- understandings. Just like a movie.

March 28, 2014

0
Reply
Agni Ortiz

I assume it's pretty close to what happens when your life flashes before your eyes as well. Same feeling of timelessness but even more intense.

March 28, 2014

0
Reply

Isn't that intriguing? As a group humans have not reached all the levels of perception or a complete understanding on what Consciousness means. I hear that "Life flashing" effect happens just when we are about to die, so I guess not many volunteer to complete that study, eh?

March 28, 2014

0
Reply
Agni Ortiz

I guess because the eye is an organic structure, it probably has a combined range of lens abilities, but is also prone to breakdowns too (recently I need to wear glasses for some tasks).
Now if Metabones could design a tiny speedbooster implant for me with an EF mount in the optic nerve, that might help.

March 25, 2014

0
Reply
Saied

There is what people call contact lenses for that...

March 25, 2014

-1
Reply
Raul

What I still don't quite understand is this: if a TV displays an image at 1,000fps through some shitty interpolation or whatever (for arguments sake lets just pretend there is actually something shot and delivered in 1,000fps), why would this give someone a headache? I'm not doubting that it does just saying that it makes no sense to me that this affects the way our brains interpret the image as it's not like it makes our eyes work any differently... what I'm saying is, if there is a tennis ball on the TV being delivered in a crazy high frame rate, it's not like your brain stops adding motion blur, that's just the way the brain works right? Does he mean that it's the frequency that our brains don't like as opposed to the speed we see something travelling on our TV's? Again... not doubting anything, just trying to understand.

March 25, 2014

0
Reply
Kraig

It doesn't make sense to me either. In real life, a real tennis ball does NOT have any motion blur, and there is no optical (or physiological) process that automatically adds that motion blur before our eyes get to see it, yet we don't get a headache when we watch that tennis ball in real life. The higher the frame rate, the less motion blur is necessary to eliminate the "Private Ryan" effect of fast motion on slower frame rates. I'd like to hear the source to this "headache" claim.

March 27, 2014

0
Reply
Predrag

Well, actually, you can totally see motion blur with your eyes. If you hold your head and eyes very still as something passes right in front of you very quickly, you'll see a blurring that looks very like camera motion blur. Try it on a subway platform (at your own risk!) or by having someone throw a ball past your face as you try really hard not to flinch. Usually, we blink and our eye pre-moves with the object, so we don't see the motion blur. But it's totally there.

Also getting very drunk will help you see the motion blur. All the lovely blurs. Sake is especially good for Bokeh.
-Olaf

March 27, 2014

-1
Reply

Our brains do process what a camera might think of as frames per second and shutter speeds. But it is dynamic, constantly variable, and many things can affect it. On average under regular conditions, we see things at about 50 to 60 frames per second. However, things like adrenaline, heart rate can all affect this. If you look at the creation of film projectors and old monitors, this is why inventors all settled on projecting images at this rate. And why we traditionally record things in 24 and 30 fps, as each frame is projected twice. Which is very important, I'll get back to explaining why a bit later.

At some point in time, technology thought it could do better. And thus began the higher frame and refresh rates. But the thing is, we aren't used to seeing things without motion blur. Now when we see things move, we track it with our eyes. Our active brain doesn't want to (or if it's moving very fast, it simply can't) process every individual 'frame' or instance of where the ball is. Rather our subconscious filters out and selects key points for our active brain to track and blurs the rest together. The slower an object is, the less noticeable the blur is. This is how we are used to defining motion in our brain, it's how we perceive the world. Even if you don't actually recognize it, or don't see it blur, it is always there ever so slightly.

Filming things at 24 and 30 fps allows the shutter to record frames that are somewhat blurry for motion. Our brains are a-ok with that. But when you film something at a higher frame rate have to adjust the the shutter to match the frames per second (in order to have realistic motion blur), but therein lies the dilemma as the higher the fps, the higher the shutter and by default a faster the shutter speed results the less motion blur. At a certain point when the shutter gets too high, it has a negative effect from the lack of motion blur almost as bad as having too little frames per second. This is why things shot at high shutter seem jittery in motion Like "Saving Private Ryan"

So things shot with faster shutter speeds have no motion blur to connect one frame to another in terms of succession. Now back to the problem, when we project high frame rate images we do so at a refresh rate so fast our brains cant comprehend it all. We are well beyond 50 or 60. You think oh well my brain will fill in the spots just like when Im tracking fast moving objects in real life right? Not necessarily. High shutter speeds fundamentally are captured with the wrong motion cadence. The lack of motion blur coupled with the high refresh/projection rates makes for jittery motion or worse our brains seeing multiple frames ontop of each other as it cant process key points/frames fast enough. Every frame is in focus and sharp, yet theres no blur to tie it all together. This is why newer tv's often have a liquidy feel when things are moving, or worse look like soap operas on crack. This is why film was traditionally 24fps film was projected each frame twice, because people brains would often compile two separate frames into one. Thus making our brain think temporally that objects are in two places at once. This while trying to compensate for motion makes for a less than stellar experience.

March 27, 2014

1
Reply

Thinking of a technology that will make cameras process images like human eyes is a wrong idea, WHY? because you will get a two times image processing the first from the camera and the second from your eyes which leads to a destruction of the image information.

March 28, 2014

-1
Reply
nadir

TV-signals broadcast at 1000 fps?! What? I basically wrote this guy off right there...

What he basically has done is a common mistake. He has confused Shutter Speed and FrameRate. Those are VERY different things. And yet. I have seen people confuse these concepts on a regular basis and my gut reaction is to demote my opinion of these people to the level of those that think that inside cameras and televisions, tiny little leprechauns are using magic pixie-dust to produce full color images when you feed them IR cookies with your remote... ok... probably a bit harsh. And I apologize for my snarky and snobbish attitude about this. But that is kind of how far away from reality the 1000 fps statement is...

Let me clarify:

Shutter-speed (as I said) is what he is actually describing. Shutter-speed is the amount of time that the sensor or film-emulsion will get to sample the incoming photons on a still image. In sports-events they are frequently in the realm of 1/1000 sec. That is. 1000th of a second. Since the object that they record doesn't get to move that far in a thousandth of a second it will mostly be recording as if it was standing perfectly still. In normal dramatic production we tend to use a vastly slower shutter-speed like 1/50th sec just so we can let the sensor or film collect as much light as possible and be able to produce the bright and colorful image we might want. Remember, less time for photons mean less photons get to the sensor or film, making it less exposed to light and therefore darker. You can go as high as you like and get more and more crisp and motion-blur-free images at the cost of light-sensitivity. And you can go as far low as you like to collect more light but instead get more motion-blur since the recorded object can more easily move during a 1/50 exposure compared to a 1/1000. In moving images there's a limit though on the low end if you don't want to start lowering the framerate since you can't (for the most time at least) be still exposing a frame when you need to start exposing the next frame. And during those hundred years we basically got used to the film look that wouldn't allow exposures that took more than half the time of the framerate because in film cameras, you actually need to move the film to get a new fresh un-exposed frame ready. And the time allotted for that movement was more often than not determined by a spinning disk shutter that was half open and half closed, which is why cinematographers using film use degrees of a circle instead of fractions of seconds. And the only cameras that could do slower shutter-speeds than half the frame-rate was video-cameras, since their reset-speed is often negligible in the grand scheme of things. So these shutter-speeds are more associated with cheap soap-operas and news and documentary work than more costly (often film-based) productions.

Frame-Rate on the other hand... is how often we do these exposures in order to produce the illusion of movement. You can decrease it to save on bandwith (or film-consumption) but introduce stuttering. You can increase it to make movements smoother but you also increase the amount of film it consumes and have to store and safely process and project without breakage (main reason why Showscan never took off probably more than the video-y looking image). And historically this has been always limited by economics. Limited to the lowest possible rate. in the silent era it was accepted that 15-20 fps was just enough to give smooth movement. But in order to get intelligible sound from systems that relied on sound printed on the film (again, a more economical and reliable way instead of hand-synching every showing to a disc) they increased it to a higher (but still the lowest possible) 24 fps. Video (starting with TV-broadcasting, video recording wasn't really available for a good while and most stations relied on filming their TV-sets to archive productions) needed a reliable clock to synch the CRT-line that produced the images with the broadcast. So they used the mains-line Hz as a base. But CRT-TV's at the time would go dark at the top when the beam reached the bottom, so they doubled the Hz and invented the interlacing to compensate. This is the main origin to the NTSC choice of 60i and the PAL choice of 50i. And until the new ISO of 120 fps systems takes hold of manufacturers, this means that NO-ONE IS GOING ABOVE 60 or 50 FPS IN TV-BROADCASTING, that is regardless what the sales-rep at your local TV-mart is peddling you.

What he is also touching on, but barely comprehending is that TV-manufacturers have, in the last decade or so, been trying to go where the broadcasters are too cheap to go. Higher fps by artificially generating virtual frames between the broadcast frames. The average Joe loves this because it makes it "smoother". But because of 100 years of experience we have also associated high fps as cheap-TV (as in, productions that couldn't afford to shoot on a format that was itself inherently fps-limited to 24-25 fps). I once, for fun, found a command-line command that limited the FPS of Counter-Strike to 24 fps. This made the game more filmic in my eyes. But kind of hard to play. And I watched the two released Hobbit movies in HFR-theaters and was struck by how much the talking-scenes looked like BBC-dramas and the action-scenes looked like video-game-cut-scenes.

And the whole notion of "too high frame-rate will hurt your brain" is a strange statement that makes no sense whatsoever unless one assumes that he has no idea what the difference between fps and shutter-speed actually is. Why? Well, because most eye-brain-combo's can handle infinite fps visual-input quite fine thank you very much. Flickering and stuttering high shutter-speed fast and erratic motion, on the other hand quickly becomes bothersome.

Now I'll end my nerd-rant and end with a disclaimer that I could very well be wrong in some parts of my post, but at least it's more connected to reality than "1000fps current-gen broadcasters"-statements.

And again. I appologize for the tone in this post, but the video was so far out of touch with reality that it hit a nerve in my nerd-glands... I'm deeply sorry.

March 30, 2014

1
Reply

Heh... I quickly realized that I goofed on my statement about the mains-Hz to video-fps-statement since the mains Hz actually is 60 or 50 Hz (depending on your region) to begin with. I probably will have to re-research that bit of info... but broadcasters still wouldn't even dream of broadcasting in anything higher than 60fps because of simple economics. So my statement still stands on that ground ;)

March 30, 2014

2
Reply

I am starting a new online blog directory and was wondering if I can submit your site? I'm trying to mature my directory slowly by hand so that it retains good quality. I will make sure and put your website in the correct category and I'll additionally use, "%BLOGTITLE%" as your anchor text. Please let me know if this is acceptable with you by contacting me at: %EMAIL%. Cheers
Rolex Swiss ETA 2836 black gold dial with Roman flag size http://www.widhh.ca/store/store_online.php?id=Rolex-Swiss-ETA-2836-black...

April 22, 2014

1
Reply