Brain activity changes related to learning of audiovisual associations in reading

Abstract
Learning to connect letters or characters of written scripts to their corresponding sounds is crucial for reading acquisition. In alphabetic languages, the letter– speech sound integration has been shown to have a protracted developmental trajectory, and failure to reach an automatic level of audiovisual integration was correlated with reading difficulties. This dissertation aims to systematically investigate the audiovisual integration process in learning to read using magnetoencephalography, by extending the previous findings on alphabetic language to logographic language, and furthermore, by examining the learning of grapheme–phoneme association during initial learning stages. Study I aimed to investigate the audiovisual integration process in a logographic language (Chinese). This audiovisual integration involved the left superior temporal cortex in Chinese, which is similar to findings in alphabetic languages. In addition, it also activated the left inferior frontal regions, which are related to the processing of additional semantic information embedded in Chinese characters. Study II correlated various brain indices of audiovisual processing with reading-related cognitive measures in children at varying stages of reading. It demonstrated that the auditory late component is closely related to rapid automatized naming and phonological processing skills. Moreover, the multisensory interaction effect was observed mainly in temporoparietal regions, and brain responses in some of these regions were further associated with children’s reading and writing abilities. Study III simulated the initial learning of grapheme–phoneme associations in adults. The results from Study III highlighted the dynamic characteristics of audiovisual learning and provided a more refined model of grapheme–phoneme learning in reading acquisition. Overall, the findings from this dissertation showed evidence that audiovisual processing is dynamic during initial learning and memory consolidation of cross-modal associations. Furthermore, audiovisual processing is less automatic in children and is linked to their reading-related cognitive skills. Finally, there are some universal audiovisual processing brain regions and mechanisms across languages that are complemented by additional regions related to processes of distinct linguistic features in different types of scripts. Keywords: audiovisual integration, language learning, child brain, magnetoencephalography, reading
Main Author
Format
Theses Doctoral thesis
Published
2020
Series
ISBN
978-951-39-8220-1
Publisher
Jyväskylän yliopisto
The permanent address of the publication
https://urn.fi/URN:ISBN:978-951-39-8220-1Use this for linking
ISSN
2489-9003
Language
English
Published in
JYU Dissertations
Contains publications
  • Artikkeli I: Xu, W., Kolozsvári, O. B., Oostenveld, R., Leppänen, P. H. T., & Hämäläinen, J. A. (2019). Audiovisual processing of chinese characters elicits suppression and congruency effects in MEG. Frontiers in human neuroscience, 13,18. DOI: 10.3389/fnhum.2019.00018
  • Artikkeli II: Xu, W., Kolozsvari, O. B., Monto, S. P., & Hämäläinen, J. A. (2018). Brain responses to letters and speech sounds and their correlations with cognitive skills related to reading in children. Frontiers in human neuroscience, 12, 304. DOI: 10.3389/fnhum.2018.00304
  • Artikkeli III: Xu, W., Kolozsvari, O. B., Oostenveld, R., & Hämäläinen, J. A. (2020). Rapid changes in brain activity during learning of grapheme-phoneme associations in adults. NeuroImage, 117058. DOI: 10.1016/j.neuroimage.2020.117058
License
In CopyrightOpen Access
Copyright© The Author & University of Jyväskylä

Share