The Experts below are selected from a list of 1293 Experts worldwide ranked by ideXlab platform
Michael S Beauchamp - One of the best experts on this subject based on the ideXlab platform.
-
weak observer level correlation and strong stimulus level correlation between the McGurk Effect and audiovisual speech in noise a causal inference explanation
Cortex, 2020Co-Authors: John F Magnotti, Kristen B Dzeda, Kira Wegnerclemens, Johannes Rennig, Michael S BeauchampAbstract:Abstract The McGurk Effect is a widely used measure of multisensory integration during speech perception. Two observations have raised questions about the validity of the Effect as a tool for understanding speech perception. First, there is high variability in perception of the McGurk Effect across different stimuli and observers. Second, across observers there is low correlation between McGurk susceptibility and recognition of visual speech paired with auditory speech-in-noise, another common measure of multisensory integration. Using the framework of the causal inference of multisensory speech (CIMS) model, we explored the relationship between the McGurk Effect, syllable perception, and sentence perception in seven experiments with a total of 296 different participants. Perceptual reports revealed a relationship between the efficacy of different McGurk stimuli created from the same talker and perception of the auditory component of the McGurk stimuli presented in isolation, both with and without added noise. The CIMS model explained this strong stimulus-level correlation using the principles of noisy sensory encoding followed by optimal cue combination within a common representational space across speech types. Because the McGurk Effect (but not speech-in-noise) requires the resolution of conflicting cues between modalities, there is an additional source of individual variability that can explain the weak observer-level correlation between McGurk and noisy speech. Power calculations show that detecting this weak correlation requires studies with many more participants than those conducted to-date. Perception of the McGurk Effect and other types of speech can be explained by a common theoretical framework that includes causal inference, suggesting that the McGurk Effect is a valid and useful experimental tool.
-
a laboratory study of the McGurk Effect in 324 monozygotic and dizygotic twins
Frontiers in Neuroscience, 2019Co-Authors: Guo Feng, Michael S Beauchamp, Bin Zhou, Wen Zhou, John F MagnottiAbstract:Multisensory integration of information from the talker’s voice and the talker’s mouth facilitates human speech perception. A popular assay of audiovisual integration is the McGurk Effect, an illusion in which incongruent visual speech information categorically changes the percept of auditory speech. There is substantial interindividual variability in susceptibility to the McGurk Effect. To better understand possible sources of this variability, we examined the McGurk Effect in 324 native Mandarin speakers, consisting of 73 monozygotic (MZ) and 89 dizygotic (DZ) twin pairs. When tested with 9 different McGurk stimuli, some participants never perceived the illusion and others always perceived it. Within participants, perception was similar across time (r = 0.55 at a 2-year retest in 150 participants) suggesting that McGurk susceptibility reflects a stable trait rather than short-term perceptual fluctuations. To examine the Effects of shared genetics and prenatal environment, we compared McGurk susceptibility between MZ and DZ twins. Both twin types had significantly greater correlation than unrelated pairs (r = 0.28 for MZ twins and r = 0.21 for DZ twins) suggesting that the genes and environmental factors shared by twins contribute to individual differences in multisensory speech perception. Conversely, the existence of substantial differences within twin pairs (even MZ co-twins) and the overall low percentage of explained variance (5.5%) argues against a deterministic view of individual differences in multisensory integration.
-
a causal inference explanation for enhancement of multisensory integration by co articulation
Scientific Reports, 2018Co-Authors: John F Magnotti, Kristen B Smith, Marcelo Salinas, Jacqunae Mays, Lin L Zhu, Michael S BeauchampAbstract:The McGurk Effect is a popular assay of multisensory integration in which participants report the illusory percept of “da” when presented with incongruent auditory “ba” and visual “ga” (AbaVga). While the original publication describing the Effect found that 98% of participants perceived it, later studies reported much lower prevalence, ranging from 17% to 81%. Understanding the source of this variability is important for interpreting the panoply of studies that examine McGurk prevalence between groups, including clinical populations such as individuals with autism or schizophrenia. The original publication used stimuli consisting of multiple repetitions of a co-articulated syllable (three repetitions, AgagaVbaba). Later studies used stimuli without repetition or co-articulation (AbaVga) and used congruent syllables from the same talker as a control. In three experiments, we tested how stimulus repetition, co-articulation, and talker repetition affect McGurk prevalence. Repetition with co-articulation increased prevalence by 20%, while repetition without co-articulation and talker repetition had no Effect. A fourth experiment compared the Effect of the on-line testing used in the first three experiments with the in-person testing used in the original publication; no differences were observed. We interpret our results in the framework of causal inference: co-articulation increases the evidence that auditory and visual speech tokens arise from the same talker, increasing tolerance for content disparity and likelihood of integration. The results provide a principled explanation for how co-articulation aids multisensory integration and can explain the high prevalence of the McGurk Effect in the initial publication.
-
Published estimates of group differences in multisensory integration are inflated
2018Co-Authors: John F Magnotti, Michael S BeauchampAbstract:A common measure of multisensory integration is the McGurk Effect, an illusion in which incongruent auditory and visual speech are integrated to produce an entirely different percept. Published studies report that participants who differ in age, gender, culture, native language, or traits related to neurological or psychiatric disorders also differ in their susceptibility to the McGurk Effect. These group-level differences are used as evidence for fundamental alterations in sensory processing between populations. Using empirical data and statistical simulations tested under a range of conditions, we show that published estimates of group differences in the McGurk Effect are inflated when only statistically significant (p < 0.05) results are published. With a sample size typical of published studies, a group difference of 10% would be reported as 31%. As a consequence of this inflation, follow-up studies often fail to replicate published reports of large between-group differences. Inaccurate estimates of Effect sizes and replication failures are especially problematic in studies of clinical populations involving expensive and time-consuming interventions, such as training paradigms to improve sensory processing. Reducing Effect size inflation and increasing replicability requires increasing the number of participants by an order of magnitude compared with current practice.
-
The McGurk Effect in the scientific literature.
2018Co-Authors: John F Magnotti, Michael S BeauchampAbstract:(A) The number of citations per year of the McGurk and MacDonald Nature paper describing the illusion, since initial publication in 1976. (B) An analysis of the sample size (defined as the number of participants in an experimental group) in published papers on the McGurk Effect (119 total papers, 262 total experimental groups).
John F Magnotti - One of the best experts on this subject based on the ideXlab platform.
-
weak observer level correlation and strong stimulus level correlation between the McGurk Effect and audiovisual speech in noise a causal inference explanation
Cortex, 2020Co-Authors: John F Magnotti, Kristen B Dzeda, Kira Wegnerclemens, Johannes Rennig, Michael S BeauchampAbstract:Abstract The McGurk Effect is a widely used measure of multisensory integration during speech perception. Two observations have raised questions about the validity of the Effect as a tool for understanding speech perception. First, there is high variability in perception of the McGurk Effect across different stimuli and observers. Second, across observers there is low correlation between McGurk susceptibility and recognition of visual speech paired with auditory speech-in-noise, another common measure of multisensory integration. Using the framework of the causal inference of multisensory speech (CIMS) model, we explored the relationship between the McGurk Effect, syllable perception, and sentence perception in seven experiments with a total of 296 different participants. Perceptual reports revealed a relationship between the efficacy of different McGurk stimuli created from the same talker and perception of the auditory component of the McGurk stimuli presented in isolation, both with and without added noise. The CIMS model explained this strong stimulus-level correlation using the principles of noisy sensory encoding followed by optimal cue combination within a common representational space across speech types. Because the McGurk Effect (but not speech-in-noise) requires the resolution of conflicting cues between modalities, there is an additional source of individual variability that can explain the weak observer-level correlation between McGurk and noisy speech. Power calculations show that detecting this weak correlation requires studies with many more participants than those conducted to-date. Perception of the McGurk Effect and other types of speech can be explained by a common theoretical framework that includes causal inference, suggesting that the McGurk Effect is a valid and useful experimental tool.
-
a laboratory study of the McGurk Effect in 324 monozygotic and dizygotic twins
Frontiers in Neuroscience, 2019Co-Authors: Guo Feng, Michael S Beauchamp, Bin Zhou, Wen Zhou, John F MagnottiAbstract:Multisensory integration of information from the talker’s voice and the talker’s mouth facilitates human speech perception. A popular assay of audiovisual integration is the McGurk Effect, an illusion in which incongruent visual speech information categorically changes the percept of auditory speech. There is substantial interindividual variability in susceptibility to the McGurk Effect. To better understand possible sources of this variability, we examined the McGurk Effect in 324 native Mandarin speakers, consisting of 73 monozygotic (MZ) and 89 dizygotic (DZ) twin pairs. When tested with 9 different McGurk stimuli, some participants never perceived the illusion and others always perceived it. Within participants, perception was similar across time (r = 0.55 at a 2-year retest in 150 participants) suggesting that McGurk susceptibility reflects a stable trait rather than short-term perceptual fluctuations. To examine the Effects of shared genetics and prenatal environment, we compared McGurk susceptibility between MZ and DZ twins. Both twin types had significantly greater correlation than unrelated pairs (r = 0.28 for MZ twins and r = 0.21 for DZ twins) suggesting that the genes and environmental factors shared by twins contribute to individual differences in multisensory speech perception. Conversely, the existence of substantial differences within twin pairs (even MZ co-twins) and the overall low percentage of explained variance (5.5%) argues against a deterministic view of individual differences in multisensory integration.
-
a causal inference explanation for enhancement of multisensory integration by co articulation
Scientific Reports, 2018Co-Authors: John F Magnotti, Kristen B Smith, Marcelo Salinas, Jacqunae Mays, Lin L Zhu, Michael S BeauchampAbstract:The McGurk Effect is a popular assay of multisensory integration in which participants report the illusory percept of “da” when presented with incongruent auditory “ba” and visual “ga” (AbaVga). While the original publication describing the Effect found that 98% of participants perceived it, later studies reported much lower prevalence, ranging from 17% to 81%. Understanding the source of this variability is important for interpreting the panoply of studies that examine McGurk prevalence between groups, including clinical populations such as individuals with autism or schizophrenia. The original publication used stimuli consisting of multiple repetitions of a co-articulated syllable (three repetitions, AgagaVbaba). Later studies used stimuli without repetition or co-articulation (AbaVga) and used congruent syllables from the same talker as a control. In three experiments, we tested how stimulus repetition, co-articulation, and talker repetition affect McGurk prevalence. Repetition with co-articulation increased prevalence by 20%, while repetition without co-articulation and talker repetition had no Effect. A fourth experiment compared the Effect of the on-line testing used in the first three experiments with the in-person testing used in the original publication; no differences were observed. We interpret our results in the framework of causal inference: co-articulation increases the evidence that auditory and visual speech tokens arise from the same talker, increasing tolerance for content disparity and likelihood of integration. The results provide a principled explanation for how co-articulation aids multisensory integration and can explain the high prevalence of the McGurk Effect in the initial publication.
-
Published estimates of group differences in multisensory integration are inflated
2018Co-Authors: John F Magnotti, Michael S BeauchampAbstract:A common measure of multisensory integration is the McGurk Effect, an illusion in which incongruent auditory and visual speech are integrated to produce an entirely different percept. Published studies report that participants who differ in age, gender, culture, native language, or traits related to neurological or psychiatric disorders also differ in their susceptibility to the McGurk Effect. These group-level differences are used as evidence for fundamental alterations in sensory processing between populations. Using empirical data and statistical simulations tested under a range of conditions, we show that published estimates of group differences in the McGurk Effect are inflated when only statistically significant (p < 0.05) results are published. With a sample size typical of published studies, a group difference of 10% would be reported as 31%. As a consequence of this inflation, follow-up studies often fail to replicate published reports of large between-group differences. Inaccurate estimates of Effect sizes and replication failures are especially problematic in studies of clinical populations involving expensive and time-consuming interventions, such as training paradigms to improve sensory processing. Reducing Effect size inflation and increasing replicability requires increasing the number of participants by an order of magnitude compared with current practice.
-
The McGurk Effect in the scientific literature.
2018Co-Authors: John F Magnotti, Michael S BeauchampAbstract:(A) The number of citations per year of the McGurk and MacDonald Nature paper describing the illusion, since initial publication in 1976. (B) An analysis of the sample size (defined as the number of participants in an experimental group) in published papers on the McGurk Effect (119 total papers, 262 total experimental groups).
Mark T Wallace - One of the best experts on this subject based on the ideXlab platform.
-
the temporal binding window for audiovisual speech children are like little adults
Neuropsychologia, 2016Co-Authors: Andrea Hillockdunn, Wesley D Grantham, Mark T WallaceAbstract:Abstract During a typical communication exchange, both auditory and visual cues contribute to speech comprehension. The influence of vision on speech perception can be measured behaviorally using a task where incongruent auditory and visual speech stimuli are paired to induce perception of a novel token reflective of multisensory integration (i.e., the McGurk Effect). This Effect is temporally constrained in adults, with illusion perception decreasing as the temporal offset between the auditory and visual stimuli increases. Here, we used the McGurk Effect to investigate the development of the temporal characteristics of audiovisual speech binding in 7–24 year-olds. Surprisingly, results indicated that although older participants perceived the McGurk illusion more frequently, no age-dependent change in the temporal boundaries of audiovisual speech binding was observed.
-
the impact of multisensory integration deficits on speech perception in children with autism spectrum disorders
Frontiers in Psychology, 2014Co-Authors: Ryan A Stevenson, Magali Segers, Susanne Ferber, Morgan D Barense, Mark T WallaceAbstract:Speech perception is an inherently multisensory process. When having a face-to-face conversation, a listener not only hears what a speaker is saying, but also sees the articulatory gestures that accompany those sounds. Speech signals in visual and auditory modalities provide complementary information to the listener (Kavanagh and Mattingly, 1974), and when both are perceived in unison, behavioral gains in in speech perception are observed (Sumby and Pollack, 1954). Notably, this benefit is accentuated when speech is perceived in a noisy environment (Sumby and Pollack, 1954). To achieve a behavioral gain from multisensory processing of speech, however, the auditory and visual signals must be perceptually bound into a single, unified percept. The most commonly cited Effect that demonstrates perceptual binding in audiovisual speech perception is the McGurk Effect (McGurk and MacDonald, 1976), where a listener hears a speaker utter the syllable “ba,” and sees the speaker utter the syllable “ga.” When these two speech signals are perceptually bound, the listener perceives the speaker as having said “da” or “tha,” syllables that are not contained in either of the unisensory signals, resulting in a perceptual binding, or integration, of the speech signals (Calvert and Thesen, 2004).
-
the multisensory temporal binding window perceptual fusion training and autism
I-perception, 2011Co-Authors: Ryan A Stevenson, Mark T WallaceAbstract:The process of integrating information across sensory modalities is highly dependent upon the temporal coincidence of the inputs. Audiovisual information is integrated within a range of temporal offsets, known as the temporal binding window (TBW), which varies between individuals. Three particular findings relating to TBW have led us to a novel approach to address sensory integration impairments in children with autism. The first is that autistic children have an atypically wide TBW, as measured through manipulations of audiovisual illusions. Second, an individual's TBW is related to their ability to perceptually fuse audiovisual inputs, particularly as seen in the McGurk Effect; the narrower the right TBW, the stronger the McGurk Effect. The third finding is that the TBW is plastic. Through perceptual feedback training, our lab showed that individuals' right TBW can be narrowed. These three findings, which we will present, lead to a study of perceptual feedback training in autistic children, who may have...
Schwartz Jean-luc - One of the best experts on this subject based on the ideXlab platform.
-
Audiovisual Binding for Speech Perception in Noise and in Aging
'Wiley', 2018Co-Authors: Chandrashekara Ganesh Attigodu, Berthommier Frédéric, Schwartz Jean-lucAbstract:International audienceSpeech Perception involves fusion of multiple sensory input and it doesn’t fuse automatically, perhaps it depends on numerous external/internal factors (e.g. attention, noise or age). In this paper, we exploit a specific paradigm in which a short audiovisual context made of coherent or incoherent speech material is displayed before an incongruent audiovisual target likely to provide fusion (McGurk Effect, McGurk & MacDonald, 1976). We confirm that incoherent context leads to unbinding, that is a reduction in the amount of fusion. Importantly, adding acoustic noise in the context though not in the target increases fusion. This suggests that listeners systematically evaluate the reliability of their sensory channels and weight them accordingly in the fusion process. We also show that older subjects display more unbinding, and discuss the potential consequences concerning their ability to understand speech in adverse conditions. We relate all these data to a “Binding-and-Fusion” model of audiovisual speech perception
-
Audio Visual Integration with Competing Sources in the Framework of Audio Visual Speech Scene Analysis
'Springer Science and Business Media LLC', 2016Co-Authors: Ganesh Attigodu, Berthommier Frédéric, Schwartz Jean-lucAbstract:International audienceWe introduce “Audio-Visual Speech Scene Analysis” (AVSSA) as an extension of the two-stage Auditory Scene Analysis model towards audiovisual scenes made of mixtures of speakers. AVSSA assumes that a coherence index between the auditory and the visual input is computed prior to audiovisual fusion, enabling to determine whether the sensory inputs should be bound together. Previous experiments on the modulation of the McGurk Effect by audiovisual coherent vs. incoherent contexts presented before the McGurk target have provided experimental evidence supporting AVSSA. Indeed, incoherent contexts appear to decrease the McGurk Effect, suggesting that they produce lower audiovisual coherence hence less audiovisual fusion. The present experiments extend the AVSSA paradigm by creating contexts made of competing audiovisual sources and measuring their Effect on McGurk targets. The competing audiovisual sources have respectively a high and a low audiovisual coherence (that is, large vs. small audiovisual comodulations in time). The first experiment involves contexts made of two auditory sources and one video source associated to either the first or the second audio source. It appears that the McGurk Effect is smaller after the context made of the visual source associated to the auditory source with less audiovisual coherence. In the second experiment with the same stimuli, the participants are asked to attend to either one or the other source. The data show that the modulation of fusion depends on the attentional focus. Altogether, these two experiments shed light on audiovisual binding, the AVSSA process and the role of attention
-
Audio-visual speech scene analysis: Characterization of the dynamics of unbinding and rebinding the McGurk Effect
'Acoustical Society of America (ASA)', 2015Co-Authors: Nahorna Olha, Berthommier Frédéric, Schwartz Jean-lucAbstract:International audienceWhile audiovisual interactions in speech perception have long been considered as automatic, recentdata suggest that this is not the case. In a previous study, Nahorna et al. [(2012). J. Acoust. Soc.Am. 132, 1061–1077] showed that the McGurk Effect is reduced by a previous incoherentaudiovisual context. This was interpreted as showing the existence of an audiovisual binding stagecontrolling the fusion process. Incoherence would produce unbinding and decrease the weight ofthe visual input in fusion. The present paper explores the audiovisual binding system to characterizeits dynamics. A first experiment assesses the dynamics of unbinding, and shows that it is rapid: Anincoherent context less than 0.5 s long (typically one syllable) suffices to produce a maximalreduction in the McGurk Effect. A second experiment tests the rebinding process, by presenting ashort period of either coherent material or silence after the incoherent unbinding context.Coherence provides rebinding, with a recovery of the McGurk Effect, while silence providesno rebinding and hence freezes the unbinding process. These experiments are interpreted in theframework of an audiovisual speech scene analysis process assessing the perceptual organization ofan audiovisual speech input before decision takes place at a higher processing stage
-
Multisensory and sensorimotor interactions in speech perception
Frontiers in Psychology, 2014Co-Authors: Tiippana Kaisa, Schwartz Jean-luc, Möttönen RiikkaAbstract:International audienceSpeech is multisensory since it is perceived through several senses. Audition is the most important one as speech is mostly heard. The role of vision has long been acknowledged since many articulatory gestures can be seen on the talker's face. Sometimes speech can even be felt by touching the face. The best-known multisensory illusion is the McGurk Effect, where incongruent visual articulation changes the auditory percept. The interest in the McGurk Effect arises from a major general question in multisensory research: How is information from different senses combined? Despite decades of research, a conclusive explanation for the illusion remains elusive. This is a good demostration of the challenges in the study of multisensory integration. Speech is special in many ways. It is the main means of human communication, and a manifestation of a unique language system. It is a signal with which all humans have a lot of experience. We are exposed to it from birth, and learn it through development in face-to-face contact with others. It is a signal that we can both perceive and produce. The role of the motor system in speech perception has been debated for a long time. Despite very active current research, it is still unclear to which extent, and in which role, the motor system is involved in speech perception. Recent evidence shows that brain areas involved in speech production are activated during listening to speech and watching a talker's articulatory gestures. Speaking involves coordination of articulatory movements and monitoring their auditory and somatosensory consequences. How do auditory, visual, somatosensory, and motor brain areas interact during speech perception? How do these sensorimotor interactions contribute to speech perception? It is surprising that despite a vast amount of research, the secrets of speech perception have not yet been solved. The multisensory and sensorimotor approaches provide new opportunities in solving them. Contributions to the research topic are encouraged for a wide spectrum of research on speech perception in multisensory and sensorimotor contexts, including novel experimental findings ranging from psychophysics to brain imaging, theories and models, reviews and opinions
Mikko Sams - One of the best experts on this subject based on the ideXlab platform.
-
Visual Processing Affects the Neural Basis of Auditory Discrimination
2016Co-Authors: Impact Factor, Doi Source Pubmed, Daniel Kislyuk, Mikko SamsAbstract:& The interaction between auditory and visual speech streams is a seamless and surprisingly Effective process. An intriguing example is the ‘‘McGurk Effect’’: The acoustic syllable /ba/ presented simultaneously with a mouth articulating /ga / is typically heard as /da / [McGurk, H., & MacDonald, J. Hearing lips and seeing voices. Nature, 264, 746–748, 1976]. Previous studies have demonstrated the interaction of auditory and visual streams at the auditory cortex level, but the importance of these interactions for the qualitative perception change remained unclear because the change could result from in-teractions at higher processing levels as well. In our electro-encephalogram experiment, we combined the McGurk Effect with mismatch negativity (MMN), a response that is elicited in the auditory cortex at a latency of 100–250 msec by an
-
ISCA Archive INTEGRATION OF DICHOTICALLY AND VISUALLY PRESENTED SPEECH STIMULI
2014Co-Authors: Mikko Sams, Sari RusanenAbstract:In dichotic listening, two competing messages are delivered to the left and right ear, respectively. Right-handed subjects tend to report hearing more frequently the message input to the right ear. This is called Right Ear Advantage (REA). When intensities and other properties of the messages are properly adjusted, subjects may have a single perception localized to the center of the head. Frequently the two stimuli ‘fuse ’ and the resultant perception corresponds to neither of the presented stimuli. In a similar vein, in audiovisual speech perception discordant auditory and visual components of the stimulus may fuse (the McGurk Effect). In the present study, we investigated the relationship between dichotic listening and audiovisual speech perception. Our results demonstrated REA and dominance of acoustical /ta / stimulus. The influence of visual speech on the perception was clearly stronger than REA. When visual information was concordant with the auditory input to one ear, the perception of that syllable increased strongly, irrespective of the ear of stimulation. Interestingly, REA appeared even when visible speech modified perception. 1
-
Visual Processing Affects the Neural Basis of Auditory Discrimination
Journal of Cognitive Neuroscience, 2008Co-Authors: Daniel Kislyuk, Riikka Möttönen, Mikko SamsAbstract:The interaction between auditory and visual speech streams is a seamless and surprisingly Effective process. An intriguing example is the “McGurk Effect”: The acoustic syllable /ba/ presented simultaneously with a mouth articulating /ga/ is typically heard as /da/ [McGurk, H., & MacDonald, J. Hearing lips and seeing voices. Nature, 264, 746–748, 1976]. Previous studies have demonstrated the interaction of auditory and visual streams at the auditory cortex level, but the importance of these interactions for the qualitative perception change remained unclear because the change could result from interactions at higher processing levels as well. In our electroencephalogram experiment, we combined the McGurk Effect with mismatch negativity (MMN), a response that is elicited in the auditory cortex at a latency of 100–250 msec by any above-threshold change in a sequence of repetitive sounds. An “odd-ball” sequence of acoustic stimuli consisting of frequent /va/ syllables (standards) and infrequent /ba/ syllables (deviants) was presented to 11 participants. Deviant stimuli in the unisensory acoustic stimulus sequence elicited a typical MMN, reflecting discrimination of acoustic features in the auditory cortex. When the acoustic stimuli were dubbed onto a video of a mouth constantly articulating /va/, the deviant acoustic /ba/ was heard as /va/ due to the McGurk Effect and was indistinguishable from the standards. Importantly, such deviants did not elicit MMN, indicating that the auditory cortex failed to discriminate between the acoustic stimuli. Our findings show that visual stream can qualitatively change the auditory percept at the auditory cortex level, profoundly influencing the auditory cortex mechanisms underlying early sound discrimination.
-
cortical operational synchrony during audio visual speech integration
Brain and Language, 2003Co-Authors: Andrew A Fingelkurts, Riikka Möttönen, Christina M Krause, Mikko SamsAbstract:Information from different sensory modalities is processed in different cortical regions. However, our daily perception is based on the overall impression resulting from the integration of information from multiple sensory modalities. At present it is not known how the human brain integrates information from different modalities into a unified percept. Using a robust phenomenon known as the McGurk Effect it was shown in the present study that audio-visual synthesis takes place within a distributed and dynamic cortical networks with emergent properties. Various cortical sites within these networks interact with each other by means of so-called operational synchrony (Kaplan et al., 1997). The temporal synchronization of cortical operations processing unimodal stimuli at different cortical sites reveals the importance of the temporal features of auditory and visual stimuli for audio-visual speech integration.