Features selection with sequentialfs with libsvm

Question

I want to use matlab toolbox to do feature selection. there is one good function there called sequentialfs that does a good job. However, I could not integrate it with LibSVM function to perform features selection. It works fine with KnnClassify, can somebody help me please. here is the code for KnnClassify:

fun1 = @(XT,yT,Xt,yt)...

    (sum((yt ~= knnclassify(Xt,XT,yT,5))));

[fs,history] = sequentialfs(fun1,data,label,'cv',c,'options',opts,'direction','forward');

Yirinec

Answer 1

You'll need to wrap the libsvm functions to train and test an SVM on a particular featureset. I'd suggest writing things in a separate .m file (though in priciple I think it could go in an anonymous function). Something like:

function err = svmwrapper(xTrain, yTrain, xTest, yTest)
  model = svmtrain(yTrain, xTrain, <svm parameters>);
  err = sum(svmpredict(yTest, xTest, model) ~= yTest);
end

and then you can call sequentialfs with:

[fs history] = sequentialfs(@svmwrapper, ...);

(You may need to check the order of the arguments to svmtrain, I can never remember which way round they should be).

The idea is that svmwrapper will train an SVM and return its error on the test set.

The anonymous equivalent would be:

svmwrapper = @(xTrain, yTrain, xTest, yTest)sum(svmpredict(yTest, xTest, svmtrain(yTrain, xTrain, <svm parameters>) ~= yTest);

which doesn't look very nice.

Trichiasis

Thanks man this is of a great help. This is what I wrote: svmwrapper = @(TrainLbl, TrainData, TestLbl, TestData)sum(svmpredict(TestLbl, TestData, svmtrain(TrainLbl, TrainData, '-t 2 -c 8') ~= yTest)); [fs,history] = sequentialfs(svmwrapper,train_label,train_data_w,test_label,test_data_w,'options',opts,'direction','forward'); – Yirinec

and I am getting this error: Error using sequentialfs (line 196) Data arguments X,Y,... must have the same numbers of rows. – Yirinec

Answer 2

I don't know if this question is still open, but i got the function working using the following handle:

% The order of the inputs is important as svmpredict and svmtrain for libsvm take input as
% ytrain, xtrain but sequentialfs sends data as xtrain, ytrain, xtest, ytest

svfun = @(xtrain,ytrain,xtest,ytest)sum(svmpredict(ytest,xtest,svmtrain(ytrain,xtrain,<svm options>)) ~= ytest);

[fs history] = sequentialfs(svfun,x,y)

Larisa