LIBSVM training data format (x values in svm_node for svm_problem)

Question

I am using LIBSVM to do a simple XOR classification programmatically, trying to understand how the functions work. I have set up the problem following the instructions in the Readme as close as possible. Still i get the wrong output when using svm_predict (always 1 or -1).

In a related question somebody suggested that the problem might arise when using very few training examples. I tried increasing the number of examples to 20 but this did not help.

I suspect that the problem is somewhere in the definition of prob.x and/or prob.y but can't understand where. Could you help clarify how to define prob.x and prob.y using svm_node?

I hade searched thoroughly but cannot find an answer... E.g. Here, here, here, here, and here.

Thanks in advance!

Here is my code:

//Parameters
svm_parameter param;
param.svm_type = C_SVC;
param.kernel_type = RBF;
param.degree = 3;
param.gamma = 0;
param.coef0 = 0;
param.nu = 0.5;
param.cache_size = 100;
param.C = 0.4;
param.eps = 1e-3;
param.p = 0.1;
param.shrinking = 1;
param.probability = 0;
param.nr_weight = 0;
param.weight_label = NULL;
param.weight = NULL;



//Problem definition
svm_problem prob;


//Length
prob.l = 4;                             //number of training examples


//x values

svm_node** x = new svm_node *[prob.l];  //Array of pointers to pointers to arrays

svm_node* x_space1 = new svm_node[3];   //Fist training example
svm_node* x_space2 = new svm_node[3];   //Second training example
svm_node* x_space3 = new svm_node[3];   //Third training example
svm_node* x_space4 = new svm_node[3];   //Fourth training example

x_space1[0].index = 1;                  //Fist training example
x_space1[0].value = 1;
x_space1[1].index = 2;
x_space1[1].value = 1;
x_space1[2].index = -1;

x_space2[0].index = 1;                  //Second training example
x_space2[0].value = 1;
x_space2[1].index = 2;
x_space2[1].value = 0;
x_space2[2].index = -1;

x_space3[0].index = 1;                  //Third training example
x_space3[0].value = 0;
x_space3[1].index = 2;
x_space3[1].value = 1;
x_space3[2].index = -1;

x_space4[0].index = 1;                  //Fourth training example
x_space4[0].value = 0;
x_space4[1].index = 2;
x_space4[1].value = 0;
x_space4[2].index = -1;

x[0] = x_space1;                        //Set each training example to x
x[1] = x_space2;
x[2] = x_space3;
x[3] = x_space4;

prob.x = x;                             //Assign x to the struct field prob.x


//yvalues
prob.y = new double[prob.l];
prob.y[0] = -1;
prob.y[1] = 1;
prob.y[2] = 1;
prob.y[3] = -1;


//Train model
svm_model *model = svm_train(&prob,&param);


//Test model
svm_node* testnode = new svm_node[3];
testnode[0].index = 1;
testnode[0].value = 1;
testnode[1].index = 2;
testnode[1].value = 0;
testnode[2].index = -1;

double retval = svm_predict(model,testnode);
qDebug()<<retval;                               //Should return +1 but returns -1

Huertas

Answer 1

It seems a problem with your parameters. For instance, param.gamma shouldn't be zero if you are using a RBF kernel.

Watchcase

Thanks Pedrom! That solved the problem. I'll do some more reading on C and gamma. – Huertas

@JCR No problem! :) I'd recommend also to read the README file too, because at least the gamma value is different (the inverse) to other libraries. github.com/cjlin1/libsvm/blob/master/README – Watchcase

Answer 2

1. Why is your XOR problem 3-dimensional? You do not need the third dimension in each point (in fact you define it but not use it, I am not sure what will libSVM do, but for sure it will affect the chosen gamma, as the libSVM heuristics chooses 1/number_of_dimensions)
2. Your C parameter looks suspucious ( 0.4 may be way to low, try 1000)

Oviduct

Hi lejlot and thanks for responding. According to the Readme for libsvm each training example should be ended with a -1 (to indicate that there are no more attributes for that example). Or are you referring to the dimensions of svm_node** x? I tried increasing C to 1000 but it did not help... – Huertas