Sonification of Emotion : Strategies for Continuous Display of Arousal and Valence

Abstract

Sonification is an interdisciplinary field of research broadly interested in the use of sound to convey information. A fundamental attribute of sound is its ability to evoke emotion, but the display of emotion as a continuous data type has not yet received adequate attention. This paper motivates the use of sonification for display of emotion in affective computing, and as a means of targeting mechanisms of emotion elicitation in music. Environmental sound and music are presented as two possible sources for non-verbal auditory emotion elicitation, each with specific determinants and available features. The review concludes that the auditory-cognitive mechanisms of brain-stem reflex and emotional contagion provide the most advantageous framework for development. A sonification model is presented that implements cues that target these mechanisms. Computationally based strategies for evaluation are presented drawing upon the music emotion recognition literature. Additional aesthetic considerations are discussed that benefit usability and attractiveness of the display.