Suppose you are sitting in your living room and suddenly everything in front of you changed to something else, say a view of the ocean. Wouldn’t you be startled? And yet, when we watch films, a cut from one scene to another changes also the entire field of view instantaneously and yet it causes us no problems. And reports about the public viewing of the very first films suggest that this new thing did not cause viewers any problems at all. I wrote a few months ago about the research by Jeffrey M. Zacks, a professor of psychology and radiology at Washington University in St. Louis, and others about why our brains are not disoriented when we watch films with even very rapid cuts that change the entire field of view instantaneously.
He has now written an article that tries to fill in with more detail why it is that we are not confused by these rapid changes, even though on the surface it seems like nothing we experience in our everyday lives.
Movies are, for the most part, made up of short runs of continuous action, called shots, spliced together with cuts. With a cut, a filmmaker can instantaneously replace most of what is available in your visual field with completely different stuff. This is something that never happened in the 3.5 billion years or so that it took our visual systems to develop. You might think, then, that cutting might cause something of a disturbance when it first appeared. And yet nothing in contemporary reports suggests that it did.
…What is going on here? Consider that our visual systems evolved over hundreds of millions of years, while film editing has been around only for a little more than 100 years. Despite this, new audiences appear to be able to assimilate splices on more or less the first try. I think the explanation is that, although we don’t think of our visual experience as being chopped up like a Paul Greengrass fight sequence, actually it is.
Simply put, visual perception is much jerkier than we realise. First, we blink. Blinks happen every couple of seconds, and when they do we are blind for a couple of tenths of a second. Second, we move our eyes. Want to have a little fun? Take a close-up selfie video of your eyeball while you watch a minute’s worth of a movie on your computer or TV. You’ll see your eyeball jerking around two or three times every second. It turns out that most of the eye movements we make are these jerky, ballistic movements called saccades. They take a little less than a tenth of a second and, while the eye is moving, the information that it is sending to your brain is pretty much garbage. Your brain has a nifty control mechanism that turns down the gain during these saccades so that you ignore the bad information. Between blinks and saccades, we are functionally blind about a third of our waking life.
…So, the signal that our brains are getting about the visual world is not like a smooth camera-pan around the environment. It’s more like a jittery music video: a sequence of brief shots of little patches of the world, stitched together. We feel like we have a detailed, continuous permanent representation of the visual details of our world, but what our visual system really delivers is a sequence of patchy pictures. Our brains do a lot of work to fill in the gaps, which can produce some pretty striking – and entertaining – errors of perception and memory.
…So now I think we have a story about why our heads don’t explode when we watch movies. It’s not that we have learned how to deal with cuts. It’s certainly not that our brains have evolved biologically to deal with film – the timescale is way too short. Instead, film cuts work because they exploit the ways in which our visual systems evolved to work in the real world.
This model of how the brain operates explains why most of us are oblivious to continuity errors in films that, despite the best efforts of the people in charge of seeing that there are none, occur in large numbers.
The technical term for that representation is an event model, and a good event model captures the information about the scene that is important for guiding your behaviour and making predictions about what might happen next.
…Our models are optimised to represent the information that is important for our comprehension of the activity. If the current shot has stuff that is inconsistent with what was in the last shot, we tend to go with what we currently see.
That makes good evolutionary sense, doesn’t it? If your memory conflicts with what is in front of your eyeballs, the chances are it is your memory that is at fault. So, most of the time your brain is stitching together a succession of views into a coherent event model, and it can handle cuts the same way it handles disruptions such as blinks and saccades in the real world.
To me, the way our brains process information is one of the most fascinating areas of study.
“Blinks happen every couple of seconds…” Every two seconds? This poor guy must have the reddest, driest eyes in history.
Come on. You are a physicist, you are comparing 3D world to a 2D movie screen ? If there is sudden change in front of you from window to ocean, there are changes in all senses, but not in 2D movie screen. But there is that slight sensation awe in 3D movies, still it doesn’t affect all sensations.
I don’t how you could miss simple thing and fascinated by complex brain power.
I know brain process info is very complex, but do you know that our spinal cord itself process tons of unconsciousness info’s, it’s like embedded programming, just relying the final outcome to the brain ? I am not even going to the brain, I am so amazed by the grey and white mater of the spinal cord itself. Come on, Mano Singham, start reading a Human Anatomy text book.
I feel very strongly about this article. Not in a bad way and not because it is wrong, but because I’m one of those people that it does not capture in terms of the broad generalizations and the experiments that this paper discusses are directly relevant to similar experiments that show that people like me are different (Tourette’s Syndrome).
I don’t see the world in a smooth perceptual stream of consciousness like other people. I have a stream that is altered in a way where I can perceive some of the “mortar” between the bricks that perception is constructed from. Basically it’s like I’m always in a startle response and implicit reflexive signals are always going off in my head. When breaking from one object of focus to another the way that my mind deals with incoming information is somewhat “disjointed” relative to how other people might perceive things. What I was just doing “follows” me in ways that can interfere with the new thing, I sometimes have to ask people to repeat what they just started telling me because I missed the first part for example.
But I’ve discovered that there are some benefits as well. For example blended perception streams make me somewhat sensitive to things like analogies and metaphors because the sub-characteristics of things are often still in my “perceptual buffer” so over time I notice patterns. Here is a paper that discusses what they describe as “enhanced cognition” in a visual task very much like the ones that the article discusses. Specifically people like me with two co-morbid disorders that come along with TS more than 50% of the time, TS+ADHD+OCD. That enhancement is from the effort that we must put out to properly putting together the emotions that we use to judge the reality in front of us. I perceive reality fine, what I feel about reality is not like what most of us feels at all. Over time this also gives me information about what people do in general, but that is still something I’m trying to figure out how to explain well. Certainly I tend to be somewhat intuitive about how fallacious reasoning works.
I suspect this is a feature and not a bug, but am not quite sure what to do with that hypothesis.
Brony,
Interesting reaction. So when you see films, do the scene shifts bother you, at least in some vague way.
In the article that I linked to there is a section that says:
And are you more likely to notice changes that others do not? You can see some examples of what they call change blindness here.
@Mano Singham
Movies and media in general does have a strong effect on me. I’m not sure how to generalize because the existing experiments have been rather limited and all I have is personal anecdotes, though I try hard to contextualize those within what I have read in brain science literature over the last five years. Here is a paper where they literally watch children with TS watching movies. A review on stress modulation* (avoiding the spam filter) in TS summarizes,
For me personally an example that comes to mind would be that in scenes that involve social embarrassment I feel it so strongly it can be difficult to keep watching. Simple color changes and similar, I’m not sure that I would necessarily notice those things as readily but I can’t rule out any differences (it’s a matter of how perception, instinct, and emotion are shaped). In fact I have lately been thinking of TS as “Autism+externalized behaviors” ans autistic people seem to want to categorize objective things while I want to categorize subjective things (but that is also something I use as a mental hypothesis).
I’ll think about the change blindness links you posted and let you know if anything seems relevant.
*The Modulating Role of Stress in the Onset and Course of Tourette’s Syndrome: A Review.
Buse et al 2014. Behav Modif. 2014 Feb 10;38(2):184-216.
I do find it hard to watch films where the camera jiggles around and constantly changes the viewing angle. It’s nausea inducing just like being a passenger in a vehicle you don’t control yourself. Films which have been shot by professionals with stable cameras (a fashion that died out in the 90’s) don’t make me ill. It’s like sitting on a ledge watching the herds and birds constantly wheeling around, quite easy to watch.
I just realized that Mano asked about transitions and I responded in terms of general sensitivities. I need to fix that but will have to think about that a bit.
Very cool article! I knew our eyes moved a lot, but I had never heard of saccades or being functionally blind a third of waking hours! I’m intrigued. Time to dive down the Wikipedia tunnel, starting with “saccades”.
I’m what they call “sensory”, so my experience really isn’t lining up with this. My brain does NOT filter out extraneous input, and can be overloaded by sudden changes in sensory stimuli.
I’m going to try to watch how I feel about things while I watch Game of Thrones and X-files tonight. It’s a little tricky because while I have spent a lot of time becoming more aware of how my mind is structured, more awareness of “overflow” from one moment to the next has not been as much of a focus. So far the best example of overflow from transitions is when someone starts talking to me and I can’t break the mental processing related to what I was just doing. I can remember times when I would get captured by emotional elements from scenes in movies and I would have some trouble shaking the emotion as the movie progressed. I don’t think that it is a matter sensory perception itself having elements that stick around, but the emotions about perception and other higher-order processes.
This is also complicated by the fact that it’s not only TS that is shaping things, it’s the ADHD and OCD as well. TS is like having inhibitory brakes removed from felt action, reaction, and urge impulses as well as a general increase in the intensity of many felt mental phenomena. It’s associated with sensory hypersensitivity of a similar category to what autistics feel (Interestingly the sensory hypersensitivity is also what is associated with savant characteristics). There is likely overflow here.
ADHD is like a weakening of some perceptual “filters” that govern what we notice in perception and how that is stored in memory. We don’t tend to be “paperwork types” (don’t store names and numbers as well), self medicate off of adrenaline and sensation and often function better in group contexts. But there are advantages in terms of being “pattern-breakers” that make us creative types and when our attention is focused we tend to hyper-focus on the trees to a level that requires routines to keep the forest in mind (and other people trying to talk to us). There is likely overflow here.
OCD I see as how perception is shaped in terms of how attention is drawn to certain basic instinctual themes, and habits directly related to using emotions or relieving a person from emotions associated with the instinctual theme. At sub-clinical levels I think this is mostly how we tend to be different in terms of personality and what we notice. The proportion of kinds of OCD seem to be different depending on co-morbid conditions. ADHD often includes a perfectionism OCD. TS has obsessions that include aggression, sex and social rules (and compulsions in things like hoarding, symmetry and counting). Some overflow could be related to OCD intrusive thoughts, but OCD could also develop in response to overflow in terms of compulsions.
As for the change blindness studies, I’m honestly not sure. People with autism are less prone to change blindness. But as I have mentioned the things people with TS would be hyper-systematic about different things than people with autism. I tend to be perceptually obsessive with respect to how and why people do what they do, not who they are or what something is. Personally I suspect that I am still prone to change blindness, but I can’t find anything on TS and change blindness.
Richard Wiseman does a cool video illustrating change blindness on his Quirkology site:
http://www.quirkology.com/UK/Video_ColourChangingTrick.shtml
I didn’t read the articles -- sounds like blipverts