Evolutionarily Tuned Generalized Pseudo-Inverse in Linear Discriminant Analysis

Abstract

Linear Discriminant Analysis (LDA) and the related Fisher's linear discriminant are very important techniques used for classification and for dimensionality reduction. A certain complication occurs in applying these methods to real data. We have to estimate the class means and common covariance matrix, which are not known. A problem arises if the number of features exceeds the number of observations. In this case the estimate of the covariance matrix does not have full rank, and so cannot be inverted. There are a number of ways to deal with this problem. In our previous paper, we proposed improving LDA in this area, and we presented a new approach which uses a generalization of the Moore-Penrose (MP) pseudo-inverse to remove this weakness. However, for data sets with a larger number of features, our method was computationally too slow to achieve good results. Now we propose a model selection method with a genetic algorithm to solve this problem. Experimental results on different data sets demonstrate that the improvement is efficient.