Musical Instruments, Body Movement, Space, and Motion Data: Music as an Emergent Multimodal Choreography
Abstract
Music is a complex multimodal medium experienced not only via sounds but also
through body movement. Musical instruments can be seen as technological objects coupled
with a repertoire of gestures. We present technical and conceptual issues related to the
digital representation and mediation of body movement in musical performance. The paper
reports on a case study of a musical performance where motion sensor technologies tracked
the movements of the musicians while they played their instruments. Motion data were used
to control the electronic elements of the piece in real time. It is suggested that computable
motion descriptors and machine learning techniques are useful tools for interpreting motion
data in a meaningful manner. However, qualitative insights regarding how human body
movement is understood and experienced are necessary to inform further development of
motion-capture technologies for expressive purposes. Thus, musical performances provide
an effective test bed for new modalities of human–computer interaction.
Main Authors
Format
Articles
Journal article
Published
2017
Series
Subjects
Publisher
University of Jyväskylä, Agora Center
The permanent address of the publication
https://urn.fi/URN:NBN:fi:jyu-201705272518Use this for linking
DOI
https://doi.org/10.17011/ht/urn.201705272518
Review status
Peer reviewed
ISSN
1795-6889
Language
English
Published in
Human Technology: An Interdisciplinary Journal on Humans in ICT Environments
Citation
- Visi, F., Coorevits, E., Schramm, R., & Miranda, E. R. (2017). Musical Instruments, Body Movement, Space, and Motion Data: Music as an Emergent Multimodal Choreography. Human Technology, 13 (1), 58-81. doi:10.17011/ht/urn.201705272518
Copyright© the Authors & the Agora Center, University of Jyväskylä, 2017. This is an open access article distributed under a Creative Commons Attribution-NonCommercial 4.0 International License.