Here’s another one of those internet things like the blue/gold dress photo. [stderr] And it appears that neuroscientists have a theory for how it works.
Before we go any further, let’s just do an unscientific test. Watch this, and note to yourself what you experience.
OK, now: some people report that they hear a sound at the final frame(s) when the pylon “lands” and there is the “vibration” from its impact. Do you?
My experience is that I feel like I expect a sound and it doesn’t happen. What I would normally say is something like that my past experiences have taught me that an impact that would create a shockwave like that is also going to make a sound, so my brain is waiting for the sound and – when there is none – it registers as unusual. What I am experiencing is disconfirmation of a learned behavior. I’ll go a step further and wonder out loud if one reason my brain is relatively OK with “shockwave + no sound” is from watching a lot of military footage – I’ve seen plenty of ground shockwaves without sound so maybe I’m not too surprised to see another one.
Scientists have another theory [medex] which appears to be a different way of saying what I think I said.
A synaesthesia-like effect in which people ‘hear’ silent flashes or movement, such as in popular ‘noisy GIFs’ and memes, could be due to a reduction of inhibition of signals that travel between visual and auditory areas of the brain, according to a new study led by researchers at City, University of London.
The study is the first to provide insight into the brain mechanisms underpinning such auditory sensations also known as a ‘visually-evoked auditory response’ (aka vEAR or ‘visual ear’).
Is “reduction of inhibition of signals” a fancy way of saying “learning”?
It was also found that musicians taking part in the study were significantly more likely to report experiencing visual ear than non-musician participants. This could be because musical training may promote joint attention to both the sound of music and the sight of the coordinated movements of the conductor or other musicians.
This makes me wonder whether a trained orchestral musician would experience a sort of shock, if they saw the conductor moving inappropriately for the sounds and reactions of the rest of the orchestra. Imagine if you had footage of a conductor who begins ‘conducting’ (whatever that is) and the musicians in the orchestra just sit there immobile, but there is music. I’d expect that some of us wouldn’t even notice whereas others might find the image to be freaky since it ran contrary to their lived experience. I had similar experiences with some computer games, where the in-game physics was subtly wrong and my brain would go into shrieking warnings because gravity was apparently broken.
“We found that people with ‘visual ears’ can use both senses together to see and also ‘hear’ silent motion, while for others hearing is inhibited when watching such visual sequences.”
Some neuroscientists believe visual-ear may be a type of synaesthesia, with other examples including music, letters or numbers that can evoke perceptions of colour. However, visual ear appears to be the most prevalent, with as many as 20% of people reporting some experiences of it compared to 4.4 per cent for other types.
This study seems to be pretty cool:
To shed light on what may be going on in the brain when people view such content, the researchers applied a weak alternating current to participants’ scalps, using a technique called transcranial Alternating Current Stimulation (tACS), to explore how the visual and auditory parts of the brain interact in those who experience visual ear and those who don’t.
The first experiment of the study included 36 healthy participants, including 16 classical musicians from the London Royal College of Music. All were shown auditory and visual ‘Morse code’ sequences, while electrical simulation (tACS) was applied to either the back of the head (visual areas of the brain) or the sides (auditory areas) using ‘alpha-frequency’ tACS stimulation. Participants were then classified as visual or non-visual ear depending on whether they reported ‘hearing’ the silent flashes.
The researchers found that in non-visual ear participants, alpha-frequency stimulation to auditory areas significantly reduced auditory performance but improved visual performance, while the opposite pattern was found for the same frequency of stimulation to visual areas (poorer vision, better audition).
Almost half of their study participants were classical musicians from the LRCM; I suspect there is a bunch of strongly learned behaviors, there.
“We were also interested to find that, on average, participants with visual ear performed better on both visual and auditory tasks than those without. Perhaps their audio-visual cooperation benefits performance because more of the brain is engaged in processing visual stimuli.
“Such cooperation might also benefit musical performance, explaining why so many of the musicians we tested reported experiencing visual ear.”
Next up: can a classically trained oboist be more easily trained to drool at the sound of a glockenspiel?
Note: I did not read the original study so we should assume standard journalism reporting errors. I.e.: the study was actually about nut-counting behavior in naked mole rats.