Our brain knows to change: merging sound and sight after hearing loss
Figure 1. Illustrations of the human and rat brain. The brain areas important for processing sounds, sights, and the combination of sound and sight are shown in red, blue and purple respectively. Human brain graphic modified from Broda-Bahm, 2013.
As humans, our brain has many extraordinary abilities such as emotion, complex arithmetic skills, and one of our greatest feats – speech. Verbal communication relies on two things: first, the speaker produces a combination of sounds to form words, and second, the listener merges both the sounds they are hearing and the mouth movements they are seeing to perceive what is being said. This ability of the listener to integrate information from both the auditory (hearing) and visual (seeing) senses is called audiovisual integration, and happens in an area of the brain called the multisensory cortex (Fig. 1 – purple). The result of this process is called audiovisual perception, as the listener can now understand the message resulting from the combination of auditory and visual cues. Although we typically think of this process in terms of human speech, it acts at a much more basic level. Indeed, audiovisual integration is working when we both hear and see a basketball hit the ground. In fact, this non-speech audiovisual integration also happens in animals with less complex brains!
But what happens when one of our senses, like hearing, becomes damaged? In humans, this phenomenon has been studied in adults who develop profound hearing loss. So far, studies in humans that looked at changes in brain activity have shown us that the brain undergoes extensive reorganization. The part of the brain that normally dealt with sound information, called the auditory cortex (Fig. 1 – red), now responds to sight information, so it becomes more like the visual cortex (Fig. 1 – blue), effectively expanding the brain’s visual processing power. In other words, the brain area that was activated upon hearing a basketball hit the ground now becomes activated upon seeing the basketball hit the ground.
Profound hearing loss is a rather extreme example that is bound to substantially alter the brain, but prompts questions about reorganization in the brains of individuals with partial hearing loss. Despite the high prevalence of partial hearing loss in adults (~1 out of 5 individuals), termed adult-onset hearing loss, much less is known about how the brain deals with integrating visual information with the residual auditory information. Do these individuals still have audiovisual perception like in normal-hearing adults? Or does the brain lose the ability to merge sight information with the poorer sound information it still receives? Because audiovisual integration also occurs in organisms with less complex brains, it means researchers can learn more about what happens in the human brain from studying other organisms, such as rodents.
Recent studies in humans and rats with partial hearing loss have shown that, much like the case of profound hearing loss, visual information begins to take over the brain, with reorganization occurring in the auditory cortex and also the multisensory cortex – the area responsible for audio-visual integration. However, despite these brain changes, audio-visual perception seems largely unchanged. This raises an important question: How is the brain maintaining audio-visual integration and perception despite visual processing taking over the brain area responsible for this ability? A recent paper published in Neural Plasticity by Dr. Schormans from Western University sought to answer this question by investigating brain cells called neurons in the auditory, multisensory, and visual brain areas in rats that either had normal hearing or partial hearing loss (see Figure. 1).
To do so, Dr. Schormans presented lights and sounds to rats that had a partial hearing loss. If a neuron in the rat brain was sensitive to the light or sound stimulus, it responded by producing an electrical signal which was measured by Dr. Schormans using specialized equipment. In this way, responses from the neurons in the multisensory cortex, called the V2L in rats, were recorded. The V2L in rats has two areas, one called the V2L multisensory zone located in the middle of the V2L and responsive to auditory and visual stimuli, and one called the V2L auditory zone, a more predominantly auditory-only area located at the bottom of the V2L. Dr. Schormans recorded the activity from neurons in both these areas so that she could identify if the neurons changed how they responded to light and/or sound stimuli (Fig. 2).
Dr. Schormans found that the brains of rats with a partial hearing loss did indeed reorganize! In Figure 2, the blue (visual) to purple (multisensory) to red (auditory) boundaries have shifted in the partial hearing loss rat brain compared to the normal hearing rat brain. The neurons in the audiovisual zone of the V2L became less responsive to sounds, and the neurons in the auditory zone of the V2L were now responsive to both sights and sounds! Therefore, although the area normally responsible for merging auditory and visual information lost its ability to do so because of a partial hearing loss in the rats, a new area took on this role. Dr. Schormans believes this means that audiovisual perception is still possible after a partial hearing loss because the audiovisual integration required for perception is now being performed by a different part of the brain, but future research is needed to confirm this hypothesis. There are many other questions Dr. Schormans and her colleagues still want to answer such as: How exactly do the neurons in the brain know to start responding to different things? And is this process the same in humans and rats?
Figure 2. Illustration of how the responses of neurons in the rat brain changed because of a partial hearing loss. Colour-coded are the brain areas where neurons responded to the auditory, multisensory (auditory + visual), or visual stimuli. Although a normal hearing rat had multisensory responses (purple) in the middle of the V2L, the multisensory responses of the rat with a partial hearing loss shifted to the bottom of the V2L and closer to the auditory cortex, so that the middle of the V2L only responded to visual stimuli. V1 = visual cortex; V2L = multisensory cortex; AuD and A1 = auditory cortex. Adapted from Schormans et al., 2019.
This research shows that the brain has a natural ability to compensate for changes in hearing ability, even when these changes are only partial. Just like in complete hearing loss, when partial hearing loss occurs, vision begins to take over the auditory and audiovisual part of the brain. What does this mean for our more complex human verbal communication? Well, even with partial hearing loss in adulthood, our brains may be able to change so that we can still combine the lip movements we see with the degraded sounds being heard to perceive what someone is saying to us.
Original research article:
Schormans, A. L. and Allman, B. L. (2019). Compensatory Plasticity in the Lateral Extrastriate Visual Cortex Preserves Audiovisual Temporal Processing following Adult-Onset Hearing Loss. Neural Plasticity. ID 7946987, https://doi.org/10.1155/2019/7946987
References:
Broda-Bahm, K. (April 8, 2013). Ban the Bullet (From Your Slides). Retrieved from Persuasive Litigator: https://www.persuasivelitigator.com/2013/04/ban-the-bullet-from-your-slides.html