How to label the training projections obtained by PCA to use for training SVM for classification? MATLAB

Question

I have a "training set" of images. I have formed the 'Eigenspace'. Now i need to label the projections to train the SVM. The projections of "face 1" to the Eigenspace has to be labelled +1 and the projections of all the other faces to the Eigenspace has to be labelled -1.

I don't know how to do this.Any suggestions would be really helpful!

I formed the eigenspace using the following :

    function [signals,V] = pca2(data)
    [M,N] = size(data); 
    data = reshape(data, M*N,1); % subtract off the mean for each dimension 
    mn = mean(data,2); 
    data = bsxfun(@minus, data, mean(data,1)); 
    % construct the matrix Y 
    Y = data'*data / (M*N-1); 
    [V D] = eigs(Y, 10); % reduce to 10 dimension 
    % project the original data 
    signals = data * V; 

Noncombatant

I formed the eigenspace using the following : function [signals,V] = pca2(data) [M,N] = size(data); data = reshape(data, MN,1); % subtract off the mean for each dimension mn = mean(data,2); data = bsxfun(@minus, data, mean(data,1)); % construct the matrix Y Y = data'*data / (MN-1); [V D] = eigs(Y, 10); % reduce to 10 dimension % project the original data signals = data * V; – Noncombatant

Answer 1

If you are trying to recognize more than one person, you have to create one separate data file for each person, and one sepparate SVM for each person. This is because SVM are focused on two-class separation.

This is an example using libsvm for Matlab (here is the full code), supposing you have the data in a file:

[person1_label, person1_inst] = libsvmread('../person1');
[person2_label, person2_inst] = libsvmread('../person2');
[person3_label, person3_inst] = libsvmread('../person3');

model1 = svmtrain(person1_label, person1_inst, '-c 1 -g 0.07 -b 1');
model2 = svmtrain(person2_label, person2_inst, '-c 1 -g 0.07 -b 1');
model3 = svmtrain(person3_label, person3_inst, '-c 1 -g 0.07 -b 1');

To test one face, you need to apply all the models and get the max output (when using svmpredict you have to use '-b 1' to obtain the probability estimates.

Additionally, in Matlab you don't need to use svmread or svmwrite, you can pass directly the data:

training_data = [];%Your matrix that contains 4 feature vectors
person1_label =[1,1,-1,-1];
person2_label = [-1,-1,1,-1];
person3_label = [-1,-1,-1,1];

model1 = svmtrain(person1_label, person_inst, '-c 1 -g 0.07 -b 1');
model2 = svmtrain(person2_label, person_inst, '-c 1 -g 0.07 -b 1');
model3 = svmtrain(person3_label, person_inst, '-c 1 -g 0.07 -b 1');

Caban

I am trying to recognize only one face for now. The thing is, I have used PCA and projected the 'training data' onto the eigenspace.My training data has images from person 1(the one to be identified) as well as images from other people. I want to now label the PCA obtained from person 1 as +1 and PCA obtained from all the other people as -1, so that when i give the test image as person 1, the svm should be able to predict the label as -1. I am not able to do this cause PCA requires projections and after projections i can't differentiate which ones are from person 1 and the others. – Noncombatant

OK, I don't know exactly how to solve it. In this example you can see your same application (but using Python). Maybe this helps you: scikit-learn.org/stable/auto_examples/applications/… – Caban

I looked through the code you mentioned but they are using an inbuilt function to "split" the data. I need to know how to do it manually. But thanks for your effort. – Noncombatant

Answer 2

label = ones(N,1);% N samples in total, +1 represents face 1
for i=1:N 
    % For each face image, you run
    [signals,V] = pca2(data); % ith data
    if ....  % other faces than face 1
        label(i) = -1;
    end
    face(i,:) = reshape(signals,1,[]);
end
model = svmtrain(label,face);

Pictorial

Answer 3

It seems that you cannot train the SVM... This is an example on how you can train a Matlab SVM:

%Generate 100 positive points (the data is a circle)
r = sqrt(rand(100,1)); % radius
t = 2*pi*rand(100,1); % angle
dataP = [r.*cos(t), r.*sin(t)]; % points

%Generate 100 negative points (the data is a circle)
r2 = sqrt(3*rand(100,1)+1); % radius
t2 = 2*pi*rand(100,1); % angle
dataN = [r2.*cos(t2), r2.*sin(t2)]; % points

data3 = [dataN;dataP];
theclass = ones(200,1);
theclass(1:100) = -1; %First 100 points are negative

%Train the SVM
cl = svmtrain(data3,theclass,'Kernel_Function','rbf');

Caban

I am using the open source library 'LIBSVM' for implementing SVM. I need to write the data using 'svmwrite' then train the model using 'svmtrain' and then predict label of test data using 'svmpredict'. The problem is my training file had projected signals from +1 class and -1 class so i don't know how to label the data appropriately. – Noncombatant

@Noncombatant you should reformulate the question because you didn't mention libsvm (you only tag as it)... And I don't understand the problem with your classes... – Caban