Minimal Learning Machine : Theoretical Results and Clustering-Based Reference Point Selection

Abstract

The Minimal Learning Machine (MLM) is a nonlinear, supervised approach based on learning linear mapping between distance matrices computed in input and output data spaces, where distances are calculated using a subset of points called reference points. Its simple formulation has attracted several recent works on extensions and applications. In this paper, we aim to address some open questions related to the MLM. First, we detail the theoretical aspects that assure the MLM's interpolation and universal approximation capabilities, which had previously only been empirically verified. Second, we identify the major importance of the task of selecting reference points for the MLM's generalization capability. Several clustering-based methods for reference point selection in regression scenarios are then proposed and analyzed. Based on an extensive empirical evaluation, we conclude that the evaluated methods are both scalable and useful. Specifically, for a small number of reference points, the clustering-based methods outperform the standard random selection of the original MLM formulation.