Information from different senses is integrated in the human brain. The purpose of this study was to examine how the activation in the human auditory cortex changes during audio-visual stimulus. Visual speech and nonlinguistic auditory stimuli were presented simultaneously and activation in auditory cortex was measured using functional magnetic resonance imaging (fMRI). The effect of soundless vowel production in mind was studied as well.

In the experiment, two nonlinguistic auditory stimuli were presented at the middle frequencies 250 Hz (low) and 2 000 Hz (high). Duration of the tone was 100 ms and stimuli were repeated every 500 ms in 30-s periods. Low and high frequency tones and silent period were presented in random order. Visual stimuli were videos, 10 minutes in length. In the first video a woman was articulating silently four Finnish language vowels (/a/, /i/, /o/, /y/). In the second, control experiment, there was a still face image of the same woman and moving circles (directions vertical, horizontal, right, left) were superimposed on the mouth region. The third visual stimulus was a still face image and during the condition subject was instructed to soundlessly produce vowels in mind.

The results show that lip-reading activated auditory cortex more than control stimuli (moving circles), especially at lower frequencies (250 Hz). Activation was seen in both hemispheres. Furthermore, our results suggest greater activation during control experiment than during lip-reading at higher frequencies (2 000 Hz). During soundless vowel production there was no meaningful activation at either of the frequencies.

Our results are consistent with previous findings that there are multiple frequency-dependent (i.e. tonotopically organized) areas in the human auditory cortex. Activation was stronger in the left than in the right hemisphere. Further experiments at different frequency ranges should he carried out to examine tonotopical organization more precise. In the future, higher magnetic field fMRI scanners and combining several experimental study techniques to same investigation might reveal much more about tonotopic organization of the auditory cortex and activation areas during various audio-visual stimuli.