On the Role of Taylor’s Formula in Machine Learning

Abstract

The classical Taylor’s formula is an elementary tool in mathematical analysis and function approximation. Its role in the optimization theory, whose data-driven variants have a central role in machine learning training algorithms, is well-known. However, utilization of Taylor’s formula in the derivation of new machine learning methods is not common and the purpose of this article is to introduce such use cases. Both a feedforward neural network and a recently introduced distance-based method are used as data-driven models. We demonstrate and assess the proposed techniques empirically both in unsupervised and supervised learning scenarios.