libsvm java implementation

Question

I am trying to use the java bindings for libsvm:

http://www.csie.ntu.edu.tw/~cjlin/libsvm/

I have implemented a 'trivial' example which is easily linearly separable in y. The data is defined as:

double[][] train = new double[1000][]; 
double[][] test = new double[10][];

for (int i = 0; i < train.length; i++){
    if (i+1 > (train.length/2)){        // 50% positive
        double[] vals = {1,0,i+i};
        train[i] = vals;
    } else {
        double[] vals = {0,0,i-i-i-2}; // 50% negative
        train[i] = vals;
    }           
}

Where the first 'feature' is the class and the training set is similarly defined.

To train the model:

private svm_model svmTrain() {
    svm_problem prob = new svm_problem();
    int dataCount = train.length;
    prob.y = new double[dataCount];
    prob.l = dataCount;
    prob.x = new svm_node[dataCount][];     

    for (int i = 0; i < dataCount; i++){            
        double[] features = train[i];
        prob.x[i] = new svm_node[features.length-1];
        for (int j = 1; j < features.length; j++){
            svm_node node = new svm_node();
            node.index = j;
            node.value = features[j];
            prob.x[i][j-1] = node;
        }           
        prob.y[i] = features[0];
    }               

    svm_parameter param = new svm_parameter();
    param.probability = 1;
    param.gamma = 0.5;
    param.nu = 0.5;
    param.C = 1;
    param.svm_type = svm_parameter.C_SVC;
    param.kernel_type = svm_parameter.LINEAR;       
    param.cache_size = 20000;
    param.eps = 0.001;      

    svm_model model = svm.svm_train(prob, param);

    return model;
}

Then to evaluate the model I use:

public int evaluate(double[] features) {
    svm_node node = new svm_node();
    for (int i = 1; i < features.length; i++){
        node.index = i;
        node.value = features[i];
    }
    svm_node[] nodes = new svm_node[1];
    nodes[0] = node;

    int totalClasses = 2;       
    int[] labels = new int[totalClasses];
    svm.svm_get_labels(_model,labels);

    double[] prob_estimates = new double[totalClasses];
    double v = svm.svm_predict_probability(_model, nodes, prob_estimates);

    for (int i = 0; i < totalClasses; i++){
        System.out.print("(" + labels[i] + ":" + prob_estimates[i] + ")");
    }
    System.out.println("(Actual:" + features[0] + " Prediction:" + v + ")");            

    return (int)v;
}

Where the passed array is a point from the testing set.

The results are always returning class 0. With the exact results being:

(0:0.9882998314585194)(1:0.011700168541480586)(Actual:0.0 Prediction:0.0)
(0:0.9883952943701599)(1:0.011604705629839989)(Actual:0.0 Prediction:0.0)
(0:0.9884899803606306)(1:0.011510019639369528)(Actual:0.0 Prediction:0.0)
(0:0.9885838957058696)(1:0.011416104294130458)(Actual:0.0 Prediction:0.0)
(0:0.9886770466322342)(1:0.011322953367765776)(Actual:0.0 Prediction:0.0)
(0:0.9870913229268679)(1:0.012908677073132284)(Actual:1.0 Prediction:0.0)
(0:0.9868781382588805)(1:0.013121861741119505)(Actual:1.0 Prediction:0.0)
(0:0.986661444476744)(1:0.013338555523255982)(Actual:1.0 Prediction:0.0)
(0:0.9864411843906802)(1:0.013558815609319848)(Actual:1.0 Prediction:0.0)
(0:0.9862172999068877)(1:0.013782700093112332)(Actual:1.0 Prediction:0.0)

Can someone explain why this classifier is not working? Is there a step I have messed up, or a step I am missing?

Thanks

Lainelainey

Answer 1

it seems to me that your evaluate method is wrong. Should be something like this:

public double evaluate(double[] features, svm_model model) 
{
    svm_node[] nodes = new svm_node[features.length-1];
    for (int i = 1; i < features.length; i++)
    {
        svm_node node = new svm_node();
        node.index = i;
        node.value = features[i];

        nodes[i-1] = node;
    }

    int totalClasses = 2;       
    int[] labels = new int[totalClasses];
    svm.svm_get_labels(model,labels);

    double[] prob_estimates = new double[totalClasses];
    double v = svm.svm_predict_probability(model, nodes, prob_estimates);

    for (int i = 0; i < totalClasses; i++){
        System.out.print("(" + labels[i] + ":" + prob_estimates[i] + ")");
    }
    System.out.println("(Actual:" + features[0] + " Prediction:" + v + ")");            

    return v;
}

Theran

Could you explain what is the mistake in the question code? I am having problem in spotting the bug! :( – Sematic

Answer 2

Here is a rework of the above example that I've tested using data from the following R code: http://cbio.ensmp.fr/~jvert/svn/tutorials/practical/svmbasic/svmbasic_notes.pdf

import libsvm.*;

public class libsvmTest {

  public static void main(String [] args) {

      double[][] xtrain = ...
      double[][] xtest = ...
      double[][] ytrain = ...
      double[][] ytest = ...

      svm_model m = svmTrain(xtrain,ytrain);

      double[] ypred = svmPredict(xtest, m); 

      for (int i = 0; i < xtest.length; i++){
          System.out.println("(Actual:" + ytest[i][0] + " Prediction:" + ypred[i] + ")"); 
      }  

  }

  static svm_model svmTrain(double[][] xtrain, double[][] ytrain) {
        svm_problem prob = new svm_problem();
        int recordCount = xtrain.length;
        int featureCount = xtrain[0].length;
        prob.y = new double[recordCount];
        prob.l = recordCount;
        prob.x = new svm_node[recordCount][featureCount];     

        for (int i = 0; i < recordCount; i++){            
            double[] features = xtrain[i];
            prob.x[i] = new svm_node[features.length];
            for (int j = 0; j < features.length; j++){
                svm_node node = new svm_node();
                node.index = j;
                node.value = features[j];
                prob.x[i][j] = node;
            }           
            prob.y[i] = ytrain[i][0];
        }               

        svm_parameter param = new svm_parameter();
        param.probability = 1;
        param.gamma = 0.5;
        param.nu = 0.5;
        param.C = 100;
        param.svm_type = svm_parameter.C_SVC;
        param.kernel_type = svm_parameter.LINEAR;       
        param.cache_size = 20000;
        param.eps = 0.001;      

        svm_model model = svm.svm_train(prob, param);

        return model;
    }  

  static double[] svmPredict(double[][] xtest, svm_model model) 
  {

      double[] yPred = new double[xtest.length];

      for(int k = 0; k < xtest.length; k++){

        double[] fVector = xtest[k];

        svm_node[] nodes = new svm_node[fVector.length];
        for (int i = 0; i < fVector.length; i++)
        {
            svm_node node = new svm_node();
            node.index = i;
            node.value = fVector[i];
            nodes[i] = node;
        }

        int totalClasses = 2;       
        int[] labels = new int[totalClasses];
        svm.svm_get_labels(model,labels);

        double[] prob_estimates = new double[totalClasses];
        yPred[k] = svm.svm_predict_probability(model, nodes, prob_estimates);

      }

      return yPred;
  } 


}

Here is the output:

(Actual:1.0 Prediction:1.0)
(Actual:1.0 Prediction:1.0)
(Actual:1.0 Prediction:1.0)
(Actual:1.0 Prediction:1.0)
(Actual:1.0 Prediction:1.0)
(Actual:1.0 Prediction:1.0)
(Actual:1.0 Prediction:1.0)
(Actual:1.0 Prediction:1.0)
(Actual:1.0 Prediction:1.0)
(Actual:1.0 Prediction:1.0)
(Actual:1.0 Prediction:1.0)
(Actual:1.0 Prediction:1.0)
(Actual:1.0 Prediction:1.0)
(Actual:1.0 Prediction:1.0)
(Actual:1.0 Prediction:1.0)
(Actual:-1.0 Prediction:-1.0)
(Actual:-1.0 Prediction:-1.0)
(Actual:-1.0 Prediction:-1.0)
(Actual:-1.0 Prediction:-1.0)
(Actual:-1.0 Prediction:-1.0)
(Actual:-1.0 Prediction:-1.0)
(Actual:-1.0 Prediction:-1.0)
(Actual:-1.0 Prediction:-1.0)
(Actual:-1.0 Prediction:1.0)
(Actual:-1.0 Prediction:-1.0)
(Actual:-1.0 Prediction:-1.0)
(Actual:-1.0 Prediction:-1.0)
(Actual:-1.0 Prediction:-1.0)
(Actual:-1.0 Prediction:-1.0)
(Actual:-1.0 Prediction:-1.0)

Headachy

Thank you a lot for the helpful code. Why did you use param.probability = 1;? And second, do you know how one can set the weight if one have unbalanced classes? I mean the weight with which the C parameter is weighted. – Devinna

Doesn't prob_estimates lose scope when you call svm.svm_predict_probability()? – Alvord

This is simply a post to help getting started with LIBSVM; from there, its up the user to to determine what works in accordance to the problem. For questions regarding this, I suggest you visit the site of the maintaners of this package: csie.ntu.edu.tw/~cjlin/libsvm/faq.html#/… – Headachy

Answer 3

I made a slightly refactored version of LibSVM's java implementation which you may find easier to use: https://github.com/syeedibnfaiz/libsvm-java-kernel. Have a look at the Demo.java class to see how to use it.

Weeks