Eigenface
≈ 10 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
Cambridge An eigenface ( ) is the name given to a set of eigenvectors when used in the computer vision problem of human face recognition. The approach of using eigenfaces for recognition was developed by Sirovich and Kirby and used by Matthew Turk and Alex Pentland in face classification. The eigenvectors are derived from the covariance matrix of the probability distribution over the high-dimensional vector space of face images. The eigenfaces themselves form a basis set of all images used to construct the covariance matrix. This produces dimension reduction by allowing the smaller set of basis images to represent the original training images. Classification can be achieved by comparing how faces are represented by the basis set.
Cambridge An eigenface ( ) is the name given to a set of eigenvectors when used in the computer vision problem of human face recognition. The approach of using eigenfaces for recognition was developed by Sirovich and Kirby and used by Matthew Turk and Alex Pentland in face classification. The eigenvectors are derived from the covariance matrix of the probability distribution over the high-dimensional vector space of face images. The eigenfaces themselves form a basis set of all images used to construct the covariance matrix. This produces dimension reduction by allowing the smaller set of basis images to represent the original training images. Classification can be achieved by comparing how faces are represented by the basis set. The eigenface approach began with a search for a low-dimensional representation of face images. Sirovich and Kirby showed that principal component analysis could be used on a collection of face images to form a set of basis features. These basis images, known as eigenpictures, could be linearly combined to reconstruct images in the original training set. If the training set consists of M images, principal component analysis could form a basis set of N images, where N 1 = (23% of E 1 ) + (2% of E 2 ) + (51% of E 3 ) + ... + (1% E n ). In 1991 M. Turk and A. Pentland expanded these results and presented the eigenface method of face recognition. In addition to designing a system for automated face recognition using eigenfaces, they showed a way of calculating the eigenvectors of a covariance matrix such that computers of the time could perform eigen-decomposition on a large number of face images. Face images usually occupy a high-dimensional space and conventional principal component analysis was intractable on such data sets. Turk and Pentland's paper demonstrated ways to extract the eigenvectors based on matrices sized by the number of images rather than the number of pixels. Once established, the eigenface method was expanded to include methods of preprocessing to improve accuracy. Multiple manifold approaches were also used to build sets of eigenfaces for different subjects and different features, such as the eyes. Choose the principal components. Sort the eigenvalues in descending order and arrange eigenvectors accordingly. The number of principal components k is determined arbitrarily by setting a threshold ε on the total variance. Total variance , = number of components, and \lambda represents component eigenvalue. k is the smallest number that satisfies \frac{(\lambda_{1}+\lambda_{2}+...+\lambda_{k})}{v} > \epsilon These eigenfaces can now be used to represent both existing and new faces: we can project a new (mean-subtracted) image on the eigenfaces and thereby record how that new face differs from the mean face. The eigenvalues associated with each eigenface represent how much the images in the training set vary from the mean image in that direction. Information is lost by projecting the image on a subset of the eigenvectors, but losses are minimized by keeping those eigenfaces with the largest eigenvalues. For instance, working with a 100 × 100 image will produce 10,000 eigenvectors. In practical applications, most faces can typically be identified using a projection on between 100 and 150 eigenfaces, so that most of the 10,000 eigenvectors can be discarded. Here is an example of calculating eigenfaces with Extended Yale Face Database B. To evade computational and storage bottleneck, the face images are sampled down by a factor 4×4=16. clear all; close all; load yalefaces [h, w, n] = size(yalefaces); d = h * w; % vectorize images x = reshape(yalefaces, [d n]); x = double(x); % subtract mean mean_matrix = mean(x, 2); x = bsxfun(@minus, x, mean_matrix); % calculate covariance s = cov(x'); % obtain eigenvalue & eigenvector [V, D] = eig(s); eigval = diag(D); % sort eigenvalues in descending order eigval = eigval(end: - 1:1); V = fliplr(V); % show mean and 1st through 15th principal eigenvectors figure, subplot(4, 4, 1) imagesc(reshape(mean_matrix, [h, w])) colormap gray for i = 1:15 subplot(4, 4, i + 1) imagesc(reshape(V(:, i), h, w)) end Note that although the covariance matrix S generates many eigenfaces, only a fraction of those are needed to represent the majority of the faces. For example, to represent 95% of the total variation of all face images, only the first 43 eigenfaces are needed. To calculate this result, implement the following code: % evaluate the number of principal components needed to represent 95% Total variance. eigsum = sum(eigval); csum = 0; for i = 1:d csum = csum + eigval(i); tv = csum / eigsum; if tv > 0.95 k95 = i; break end; end; Performing PCA directly on the covariance matrix of the images is often computationally infeasible. If small images are used, say 100 × 100 pixels, each image is a point in a 10,000-dimensional space and the covariance matrix S is a matrix of 10,000 × 10,000 = 10 8 elements. However the rank of the covariance matrix is limited by the number of training examples: if there are N training examples, there will be at most N − 1 eigenvectors with non-zero eigenvalues. If the number of training examples is smaller than the dimensionality of the images, the principal components can be computed more easily as follows. Let T be the matrix of preprocessed training examples, where each column contains one mean-subtracted image. The covariance matrix can then be computed as S = TT T and the eigenvector decomposition of S is given by \mathbf{Sv}_i = \mathbf{T}\mathbf{T}^T\mathbf{v}_i = \lambda_i \mathbf{v}_i However TT T is a large matrix, and if instead we take the eigenvalue decomposition of \mathbf{T}^T\mathbf{T}\mathbf{u}_i = \lambda_i \mathbf{u}_i then we notice that by pre-multiplying both sides of the equation with T, we obtain \mathbf{T}\mathbf{T}^T\mathbf{T}\mathbf{u}_i = \lambda_i \mathbf{T}\mathbf{u}_i Meaning that, if u i is an eigenvector of T T T, then v i = Tu i is an eigenvector of S. If we have a training set of 300 images of 100 × 100 pixels, the matrix T T T is a 300 × 300 matrix, which is much more manageable than the 10,000 × 10,000 covariance matrix. Notice however that the resulting vectors v i are not normalised; if normalisation is required it should be applied as an extra step.
Connection with SVD
Let denote the data matrix with column as the image vector with mean subtracted. Then, \mathrm{covariance}(X) = \frac{XX^T}{n} Let the singular value decomposition (SVD) of be: X= U{\Sigma}V^T Then the eigenvalue decomposition for XX^T is: XX^T = U{\Sigma}{{\Sigma}^T}U^T= U{\Lambda}U^T , where Λ=diag (eigenvalues of XX^T ) Thus we can see easily that: The eigenfaces = the first k ( k\leq n ) columns of U associated with the nonzero singular values. The ith eigenvalue of XX^T = \frac{1}{n}( ith singular value of X)^2 (i.e., \lambda_i=\frac{1}{n}\sigma_i^2 ) Using SVD on data matrix , it is unnecessary to calculate the actual covariance matrix to get eigenfaces. Given input image vector U\in\Re^n , the mean image vector from the database M , calculate the weight of the k-th eigenface as: w_k = V_k^T(U-M) Then form a weight vector W = [w_1, w_2,..., w_k,..., w_n] Compare W with weight vectors W_m of images in the database. Find the Euclidean distance. d = \| W-W_m \|^2 If d , then the mth entry in the database is a candidate of recognition. If \epsilon_1 , then may be an unknown face and can be added to the database. If d > \epsilon_2 , U is not a face image. The weights of each gallery image only convey information describing that image, not that subject. An image of one subject under frontal lighting may have very different weights to those of the same subject under strong left lighting. This limits the application of such a system. Experiments in the original Eigenface paper presented the following results: an average of 96% with light variation, 85% with orientation variation, and 64% with size variation. Various extensions have been made to the eigenface method. The eigenfeatures method combines facial metrics (measuring distance between facial features) with the eigenface representation. Fisherface uses linear discriminant analysis and is less sensitive to variation in lighting and pose of the face. Fisherface uses labelled data to retain more of the class-specific information during the dimension reduction stage. A further alternative to eigenfaces and Fisherfaces is the active appearance model. This approach uses an active shape model to describe the outline of a face. By collecting many face outlines, principal component analysis can be used to form a basis set of models that encapsulate the variation of different faces. Many modern approaches still use principal component analysis as a means of dimension reduction or to form basis images for different modes of variation.
Review
Eigenface provides a relatively simple way to realize face recognition in that: Its training process is completely automatic and easy to code. Eigenface adequately reduces statistical complexity in face image representation. Once eigenfaces of a database are calculated, face recognition can be achieved in real time. Eigenface can handle large databases. However, the deficiencies of the eigenface method are also obvious: It is very sensitive to lighting, scale and translation, and requires a highly controlled environment. Eigenface has difficulty capturing expression changes. The most significant eigenfaces are mainly about illumination encoding and do not provide useful information regarding the actual face. To cope with illumination distraction in practice, the eigenface method usually discards the first three eigenfaces from the dataset. Since illumination is usually the cause behind the largest variations in face images, the first three eigenfaces will mainly capture the information of 3-dimensional lighting changes, which has little contribution to face recognition. By discarding those three eigenfaces, there will be a decent amount of boost in accuracy of face recognition, but other methods such as fisherface and linear space still have the advantage. A. Pentland, B. Moghaddam, T. Starner, O. Oliyide, and M. Turk. (1993). "View-based and modular Eigenspaces for face recognition". Technical Report 245, M.I.T Media Lab. T. Heseltine, N. Pears, J. Austin, Z. Chen (2003). "Face Recognition: A Comparison of Appearance-Based Approaches". Proc. VIIth Digital Image Computing: Techniques and Applications, vol 1. 59–68. Delac, K., Grgic, M., Liatsis, P. (2005). "Appearance-based Statistical Methods for Face Recognition". Proceedings of the 47th International Symposium ELMAR-2005 focused on Multimedia Systems and Applications, Zadar, Croatia, 08-10 June 2005, pp. 151–158 OpenCV + C++Builder6 implementation of PCA Java applet demonstration of eigenfaces Introduction to eigenfaces Face Recognition Function in OpenCV Eigenface-based Facial Expression Recognition in Matlab
Text derived from https://en.wikipedia.org/w/index.php?oldid=1373894082 under CC BY-SA 4.0; modifications have been made.