Top-Down Predictions of Familiarity and Congruency in Audio-Visual Speech Perception at Neural Level
Kolozsvári, O. B., Xu, W., Leppänen, P. H. T., & Hämäläinen, J. A. (2019). Top-Down Predictions of Familiarity and Congruency in Audio-Visual Speech Perception at Neural Level. Frontiers in Human Neuroscience, 13, Article 243. https://doi.org/10.3389/fnhum.2019.00243
Published inFrontiers in Human Neuroscience
DisciplinePsykologiaMonitieteinen aivotutkimuskeskusPsychologyCentre for Interdisciplinary Brain Research
© The Authors, 2019.
During speech perception, listeners rely on multimodal input and make use of both auditory and visual information. When presented with speech, for example syllables, the differences in brain responses to distinct stimuli are not, however, caused merely by the acoustic or visual features of the stimuli. The congruency of the auditory and visual information and the familiarity of a syllable, that is, whether it appears in the listener's native language or not, also modulates brain responses. We investigated how the congruency and familiarity of the presented stimuli affect brain responses to audio-visual (AV) speech in 12 adult Finnish native speakers and 12 adult Chinese native speakers. They watched videos of a Chinese speaker pronouncing syllables (/pa/, /pha/, /ta/, /tha/, /fa/) during a magnetoencephalography (MEG) measurement where only /pa/ and /ta/ were part of Finnish phonology while all the stimuli were part of Chinese phonology. The stimuli were presented in audio-visual (congruent or incongruent), audio only, or visual only conditions. The brain responses were examined in five time-windows: 75-125, 150-200, 200-300, 300-400, and 400-600 ms. We found significant differences for the congruency comparison in the fourth time-window (300-400 ms) in both sensor and source level analysis. Larger responses were observed for the incongruent stimuli than for the congruent stimuli. For the familiarity comparisons no significant differences were found. The results are in line with earlier studies reporting on the modulation of brain responses for audio-visual congruency around 250-500 ms. This suggests a much stronger process for the general detection of a mismatch between predictions based on lip movements and the auditory signal than for the top-down modulation of brain responses based on phonological information. ...
Publication in research information system
MetadataShow full item record
Related funder(s)European Commission; Academy of Finland
Funding program(s)Research profiles, AoF
The content of the publication reflects only the author’s view. The funder is not responsible for any use that may be made of the information it contains.
Additional information about fundingThis work was supported by the European Union Projects ChildBrain (Marie Curie Innovative Training Networks, # 641652), Predictable (Marie Curie Innovative Training Networks, # 641858), and the Academy of Finland (MultiLeTe #292 466).
Showing items with similar title or keywords.
Neural generators of the frequency-following response elicited to stimuli of low and high frequency : a magnetoencephalographic (MEG) study Gorina-Careta, Natàlia; Kurkela, Jari L.O.; Hämäläinen, Jarmo; Astikainen, Piia; Escera, Carles (Elsevier, 2021)The frequency-following response (FFR) to periodic complex sounds has gained recent interest in auditory cognitive neuroscience as it captures with great fidelity the tracking accuracy of the periodic sound features in the ...
Magnetoencephalography Responses to Unpredictable and Predictable Rare Somatosensory Stimuli in Healthy Adult Humans Xu, Qianru; Ye, Chaoxiong; Hämäläinen, Jarmo A.; Ruohonen, Elisa M.; Li, Xueqiao; Astikainen, Piia (Frontiers Media SA, 2021)Mismatch brain responses to unpredicted rare stimuli are suggested to be a neural indicator of prediction error, but this has rarely been studied in the somatosensory modality. Here, we investigated how the brain responds ...
Dasovich-Wilson, Johanna N.; Thompson, Marc; Saarikallio, Suvi (SAGE Publications, 2022)Streaming music videos on the internet is an increasingly popular music listening activity that has remained virtually unexplored within music psychology. Studies of the role of music in film, as well as empirical research ...
Brain Source Correlates of Speech Perception and Reading Processes in Children With and Without Reading Difficulties Azaiez, Najla; Loberg, Otto; Hämäläinen, Jarmo A.; Leppänen, Paavo H. T. (Frontiers Research Foundation, 2022)Neural correlates in reading and speech processing have been addressed extensively in the literature. While reading skills and speech perception have been shown to be associated with each other, their relationship remains ...
Xu, Weiyong; Kolozsvari, Orsolya; Oostenveld, Robert; Leppänen, Paavo H.T.; Hämäläinen, Jarmo (Frontiers Research Foundation, 2019)Learning to associate written letters/characters with speech sounds is crucial for reading acquisition. Most previous studies have focused on audiovisual integration in alphabetic languages. Less is known about logographic ...