Modelling and prediction of perceptual segmentation

Abstract

While listening to music, we somehow make sense of a multiplicity of auditory events; for example, in popular music we are often able to recognize whether the current section is a verse or a chorus, and to identify the boundaries between these segments. This organization occurs at multiple levels, since we can discern motifs, phrases, sections and other groupings. In this work, we understand segment boundaries as instants of significant change. Several studies on music perception and cognition have strived to understand what types of changes are associated with perceptual structure. However, effects of musical training, possible differences between real-time and non real-time segmentation, and the relative importance of different musical dimensions on perception and prediction of segmentation are still unsolved problems. Investigating these issues can lead to a better understanding of mechanisms used by different types of listeners in different contexts, and to gain knowledge of the relationship between perceptual structure and underlying acoustic changes in the music. In this work, we collected segmentation responses from musical pieces in two listening experiments, a real-time task and a non real-time task. Boundary data was obtained from 18 non-musicians in the real-time task and from 18 musicians in both tasks. We used kernel density estimation to aggregate boundary responses from multiple participants into a perceptual segment density curve, and novelty detection to obtain computational models based on audio musical features extracted from the musical stimuli. Overall, our findings provide evidence for an effect of experimental task on perceptual segmentation and its prediction, and clarify the contribution of local and global musical characteristics. However, the findings do not resolve discrepancies in the literature regarding musicianship. Furthermore, this investigation highlights the role of local musical change between homogeneous regions in boundary perception, the impact of boundary indication delays on segmentation, and the problem of segmentation time scales on modelling.