Understanding musical expressiveness using interactive multimedia platforms
Leman, M. & Camurri, A. (2005). Understanding musical expressiveness using interactive multimedia platforms. Musicae Scientiae, Special issue 2005-2006, 209-233.
Background in musicology. Understanding the gesture-based foundations of musical involvement opens a number of new perspectives for musicology, with the likely effect of a change in approach. Giving justice to the role of gesture in music calls for an integrated view of perception and action, which implies a shift from auditory-based cognition towards embodied cognition, an approach that includes the whole human body as mediator between mental processes and physical energy. Background in computing. With the help of new measurement tools, new computational models of multi-modal interaction, and new interactive multimedia platforms, it becomes easier to study music from a multi-modal point of view, both in terms of scientific and artistic perspectives. Music also offers an appropriate domain for the testing and developing of interactive systems and computational models for handling non-verbal multi-modal communication. Aims. In studying musical gesture, both musicology and engineering can profit from a joint collaboration. While technology provides new tools for the measurement and modelling of musical involvement, knowledge of its corporeal foundation may largely contribute to the development of technology that in turn contributes to measurement and modelling. Furthermore, this knowledge may be exploited in artistic applications. As such, the interaction between musicology and music engineering may lead to interesting developments in the domain of embodied cognition and interactive multimedia platforms. Main contribution. An overview of recent work on the development of an interactive multimedia platform for the measurement and modelling of musical expressiveness is presented. The paper introduces the notion of corporeal imitation and mirroring behavior as a core concept fur understanding musical expressiveness. Implications. The approach advocated in this paper implies a shift of paradigm in systematic musicology, away from an all too narrow focus on auditory perception and sound structure, towards a broader view that encompasses multi-modal musical involvement. This approach is grounded in multi-sensory integration, the coupling of perception and action, the study of motor imitation, and issues that relate to affect, emotions and subjectivity. The advantage of using a interactive multimedia platform for music research, rather than ad hoc solutions, pertains to the use of modular functionalities, which allow rapid prototyping of experimental setup, flexible modelling, and the subsequent use of results in artistic applications. ...
MetadataShow full item record
Showing items with similar title or keywords.
Postcards and Supasigns : Extending Integrationist Theory Through the Creation of Interactive Digital Artworks Pryor, Sally (University of Jyväskylä, Agora Center, 2007)Integrationism is a post-structuralist theory of language and communication. The theory has been applied to a groundbreaking analysis of writing as a form of communication where writing is teased apart from speech and ...
Platformed Interactions : How Social Media Platforms Relate to Candidate–Constituent Interaction During Finnish 2015 Election Campaigning Nelimarkka, Matti; Laaksonen, Salla-Maaria; Tuokko, Mari; Valkonen, Tarja (Sage Publications, 2020)Interaction between candidates and constituents via social media is a well-studied domain. The article takes this research further through a synthesis with platform studies, emerging scholarship that applies a critical ...
How Do Explicitly Expressed Emotions Influence Interpersonal Communication and Information Dissemination? : A Field Study of Emoji's Effects on Commenting and Retweeting on a Microblog Platform Zhang, Zhixing; Zhang, Yixin (Association for Information Systems, 2016)
“Do you understand (me)?” negotiating mutual understanding by using gaze and environmentally coupled gestures between two deaf signing participants Sivunen, Nina; Tapio, Elina (De Gruyter Mouton, 2022)In this paper we explore the use of multimodal and multilingual semiotic resources in interactions between two deaf signing participants, a researcher and an asylum seeker. The focus is on the use of gaze and environmentally ...
AnatomySketch : An Extensible Open-Source Software Platform for Medical Image Analysis Algorithm Development Zhuang, Mingrui; Chen, Zhonghua; Wang, Hongkai; Tang, Hong; He, Jiang; Qin, Bobo; Yang, Yuxin; Jin, Xiaoxian; Yu, Mengzhu; Jin, Baitao; Li, Taijing; Kettunen, Lauri (Springer, 2022)The development of medical image analysis algorithm is a complex process including the multiple sub-steps of model training, data visualization, human–computer interaction and graphical user interface (GUI) construction. ...