Investigating the feature extraction capabilities of non-negative matrix factorisation algorithms for black-and-white images

Department of Mathematical and Actuarial Sciences, Universiti Tunku Abdul Rahman (Sungai Long Campus), Jalan Sungai Long, Bandar Sungai Long, 43000 Kajang, Selangor Darul Ehsan, Malaysia

^* e-mail: liewhh@utar.edu.my
^** e-mail: ngws@utar.edu.my
^*** e-mail: chenhv@utar.edu.my

Abstract

Nonnegative matrix factorisation (NMF) is a class of matrix factorisation methods to approximate a nonnegative matrix as a product of two nonnegative matrices. To derive NMF algorithms, the optimisation problems for NMF are developed and the divergence used in the optimisation problems can have many forms. The β-divergence is the most popular and is used in this research. The NMF algorithms derived from the β-divergence have a few hyperparameters including the rank and the initial conditions. This paper surveyed on the software implementations of the NMF algorithms and then applied the open source software implementations of Frobenius norm based NMF algorithm, KL divergence based NMF algorithm and binary matrix factorisation (BMF) with fixed ranks to three classes of black-and-white images. For black-and-white images with a lot of common features (like MNIST), KL divergence NMF with appropriate initial guess is empirically found to be best NMF algorithm for black-and-white image feature extraction compare to other NMF algorithms. All NMF algorithms for data with little to no common features are useful in generating feature images which can be used to inspire art design as well as in the realm of computer vision.