The Neural Support Vector Machine

This paper describes a new machine learning algorithm for regression and dimensionality reduction tasks. The Neural Support Vector Machine (NSVM) is a hybrid learning algorithm consisting of neural networks and support vector machines (SVMs). The output of the NSVM is given by SVMs that take a central feature layer as their input. The feature-layer representation is the output of a number of neural networks that are trained to minimize the dual objectives of the SVMs. Because the NSVM uses a shared feature layer, the learning architecture is able to handle multiple outputs and therefore it can also be used as a dimensionality reduction method. The results on 7 regression datasets show that the NSVM in general outperforms a standard SVM and a multi-layer perceptron. Furthermore, experiments on eye images show that the NSVM autoencoder outperforms state-of-the-art dimensionality reduction methods.