Sound before Sight: Hearing Words before Seeing the Mouth Move

Chris McKee
By: - 10th Jul 2013
email hidden; JavaScript is required

A man has been confirmed to exhibit signs that he hears noises before he witnesses the physical action creating that sound wave.

This case is the first of its kind studied by neuroscientists and may give an insight into how the brain constructs the reality around us by processing sensory inputs from multiple sources at different times. While it may seem that information reaches a person all at once, our brains actually receive visual and auditory data at different times (because sound travels slower than light) and reconstruct it so we are under the illusion that events are seamlessly played out. How our mind does this remains a mystery.

The man, called ‘PH’ in the study published in the journal Cortex, reported hearing his own voice before feeling his jaw move and watching television where news reporters spoke ahead of moving their lips. Elliot Freeman investigated PH’s claims by performing Temporal Order Judgment tests (TOJs) on the man. Figures found that voices heard by PH had a 200 millisecond lag when he watched others speak on various video clips.

Freeman’s team carried out further TOJ’s, this time playing the visual of one syllable being spoken while hearing another. Interestingly, the results of this experiment revealed that PH was hearing a third syllable, one that was neither a part of the visual or audio clips. This is known as the McGurk illusion. From witnessing this effect on PH, Freeman has suggested that the brain is processing sight before sound.

When 34 other people were tested under the same parameters as PH, Freeman’s team reported that none showed signs of being under the McGurk illusion, further indicating that the brain processes different sensory inputs at several areas in the brain, and the way we perceive events is based on an average from the speed these different mechanisms work.

For PH, one of his mechanisms has been severely slowed and therefore lowering his average temporal coherence and resynchronisation. Freeman has said he will continue to study PH’s condition to help ‘slow down’ his hearing so it can correspond to what he is seeing.

Get more awesome stories shipped directly to your inbox

Want a daily dose of awesome in your inbox? Sign up for our newsletter featuring fresh and popular content.