What Comes First : Combining Motion Capture and Eye Tracking Data to Study the Order of Articulators in Constructed Action in Sign Language Narratives
Jantunen, T., Puupponen, A., & Burger, B. (2020). What Comes First : Combining Motion Capture and Eye Tracking Data to Study the Order of Articulators in Constructed Action in Sign Language Narratives. In N. Calzolari, F. Béchet, P. Blache, K. Choukri, C. Cieri, T. Declerck, S. Goggi, H. Isahara, B. Maegaard, J. Mariani, H. Mazo, A. Moreno, J. Odijk, & S. Piperidis (Eds.), LREC 2020 : Proceedings of the 12th Conference on Language Resources and Evaluation (pp. 6003-6007). European Language Resources Association. LREC proceedings. https://www.aclweb.org/anthology/2020.lrec-1.735.pdf
Published inLREC proceedings
© European Language Resources Association (ELRA)
We use synchronized 120 fps motion capture and 50 fps eye tracking data from two native signers to investigate the temporal order in which the dominant hand, the head, the chest and the eyes start producing overt constructed action from regular narration in seven short Finnish Sign Language stories. From the material, we derive a sample of ten instances of regular narration to overt constructed action transfers in ELAN which we then further process and analyze in Matlab. The results indicate that the temporal order of articulators shows both contextual and individual variation but that there are also repeated patterns which are similar across all the analyzed sequences and signers. Most notably, when the discourse strategy changes from regular narration to overt constructed action, the head and the eyes tend to take the leading role, and the chest and the dominant hand tend to start acting last. Consequences of the findings are discussed.
PublisherEuropean Language Resources Association
Parent publication ISBN979-10-95546-34-4
ConferenceInternational Conference on Language Resources and Evaluation
Is part of publicationLREC 2020 : Proceedings of the 12th Conference on Language Resources and Evaluation
Publication in research information system
MetadataShow full item record
Related funder(s)Academy of Finland
Funding program(s)Research costs of Academy Research Fellow, AoF; Academy Research Fellow, AoF; Postdoctoral Researcher, AoF
Additional information about fundingThe study was financed by the Academy of Finland under grants 269089 & 304034 (TJ) and 299067 (BB).
Showing items with similar title or keywords.
The more you move, the more action you construct : a motion capture study on head and upper-torso movements in constructed action in Finnish Sign Language narratives Jantunen, Tommi; De Weerdt, Danny; Burger, Birgitta; Puupponen, Anna (John Benjamins Publishing Company, 2021)This paper investigates, with the help of motion capture data processed on corpus principles, the characteristics of head and upper-torso movements in constructed action and regular narration (i.e., signing without constructed ...
All Eyes on Me : Behaving as Soloist in Duo Performances Leads to Increased Body Movements and Attracts Observers’ Visual Attention Küssner, Mats B.; Van Dyck, Edith; Burger, Birgitta; Moelants, Dirk; Vansteenkiste, Pieter (University of California Press, 2020)Duo musicians exhibit a broad variety of bodily gestures, but it is unclear how soloists’ and accompanists’ movements differ and to what extent they attract observers’ visual attention. In Experiment 1, seven musical duos’ ...
Burger, Birgitta; Puupponen, Anna; Jantunen, Tommi (European Group for Eye Movement Research, 2018)Both eye tracking and motion capture technologies are nowadays frequently used in human sciences, although both technologies are usually used separately. However, measuring both eye and body movements simultaneously would ...
The focus and timing of gaze matters : Investigating collaborative knowledge construction in a simulation-based environment by combined video and eye tracking Lämsä, Joni; Kotkajuuri, Jimi; Lehtinen, Antti; Koskinen, Pekka; Mäntylä, Terhi; Kilpeläinen, Jasmin; Hämäläinen, Raija (Frontiers Media SA, 2022)Although eye tracking has been successfully used in science education research, exploiting its potential in collaborative knowledge construction has remained sporadic. This article presents a novel approach for studying ...
Jantunen, Tommi (De Gruyter Open, 2017)This paper investigates the interplay of constructed action and the clause in Finnish Sign Language (FinSL). Constructed action is a form of gestural enactment in which the signers use their hands, face and other parts of ...