Brain activity changes related to learning of audiovisual associations in reading
Abstract
Learning to connect letters or characters of written scripts to their corresponding
sounds is crucial for reading acquisition. In alphabetic languages, the letter–
speech sound integration has been shown to have a protracted developmental
trajectory, and failure to reach an automatic level of audiovisual integration was
correlated with reading difficulties. This dissertation aims to systematically
investigate the audiovisual integration process in learning to read using
magnetoencephalography, by extending the previous findings on alphabetic
language to logographic language, and furthermore, by examining the learning
of grapheme–phoneme association during initial learning stages. Study I aimed
to investigate the audiovisual integration process in a logographic language
(Chinese). This audiovisual integration involved the left superior temporal cortex
in Chinese, which is similar to findings in alphabetic languages. In addition, it
also activated the left inferior frontal regions, which are related to the processing
of additional semantic information embedded in Chinese characters. Study II
correlated various brain indices of audiovisual processing with reading-related
cognitive measures in children at varying stages of reading. It demonstrated that
the auditory late component is closely related to rapid automatized naming and
phonological processing skills. Moreover, the multisensory interaction effect was
observed mainly in temporoparietal regions, and brain responses in some of
these regions were further associated with children’s reading and writing
abilities. Study III simulated the initial learning of grapheme–phoneme
associations in adults. The results from Study III highlighted the dynamic
characteristics of audiovisual learning and provided a more refined model of
grapheme–phoneme learning in reading acquisition. Overall, the findings from
this dissertation showed evidence that audiovisual processing is dynamic during
initial learning and memory consolidation of cross-modal associations.
Furthermore, audiovisual processing is less automatic in children and is linked
to their reading-related cognitive skills. Finally, there are some universal
audiovisual processing brain regions and mechanisms across languages that are
complemented by additional regions related to processes of distinct linguistic
features in different types of scripts.
Keywords: audiovisual integration, language learning, child brain,
magnetoencephalography, reading
Main Author
Format
Theses
Doctoral thesis
Published
2020
Series
ISBN
978-951-39-8220-1
Publisher
Jyväskylän yliopisto
The permanent address of the publication
https://urn.fi/URN:ISBN:978-951-39-8220-1Use this for linking
ISSN
2489-9003
Language
English
Published in
JYU Dissertations
Contains publications
- Artikkeli I: Xu, W., Kolozsvári, O. B., Oostenveld, R., Leppänen, P. H. T., & Hämäläinen, J. A. (2019). Audiovisual processing of chinese characters elicits suppression and congruency effects in MEG. Frontiers in human neuroscience, 13,18. DOI: 10.3389/fnhum.2019.00018
- Artikkeli II: Xu, W., Kolozsvari, O. B., Monto, S. P., & Hämäläinen, J. A. (2018). Brain responses to letters and speech sounds and their correlations with cognitive skills related to reading in children. Frontiers in human neuroscience, 12, 304. DOI: 10.3389/fnhum.2018.00304
- Artikkeli III: Xu, W., Kolozsvari, O. B., Oostenveld, R., & Hämäläinen, J. A. (2020). Rapid changes in brain activity during learning of grapheme-phoneme associations in adults. NeuroImage, 117058. DOI: 10.1016/j.neuroimage.2020.117058
Copyright© The Author & University of Jyväskylä