Multiple ROC curves in one plot ROCR

Question

Is it possible to plot the roc curve for diffrent classifiers in the same plot using the ROCR package? I've tried:

>plot(perf.neuralNet, colorize=TRUE)
>lines(perf.randomForest)

But I get:

Error en as.double(y) : cannot coerce type 'S4' to vector of type 'double'

Thank you!

Canonicals

Answer 1

The problem with your lines-approach is that there is no generic S4 lines function for an object of class performance defined in the ROCR package. But you can use the generic plot function as you did with an additional add = TRUE argument. For example this is partly from the example page of ?plot.performance:

library(ROCR)
data(ROCR.simple)
pred <- prediction( ROCR.simple$predictions, ROCR.simple$labels )
pred2 <- prediction(abs(ROCR.simple$predictions + 
                        rnorm(length(ROCR.simple$predictions), 0, 0.1)), 
        ROCR.simple$labels)
perf <- performance( pred, "tpr", "fpr" )
perf2 <- performance(pred2, "tpr", "fpr")
plot( perf, colorize = TRUE)
plot(perf2, add = TRUE, colorize = TRUE)

OR, you can store all your predictions in a matrix and do all the subsequent steps in one:

preds <- cbind(p1 = ROCR.simple$predictions, 
                p2 = abs(ROCR.simple$predictions + 
                rnorm(length(ROCR.simple$predictions), 0, 0.1)))

pred.mat <- prediction(preds, labels = matrix(ROCR.simple$labels, 
                nrow = length(ROCR.simple$labels), ncol = 2) )

perf.mat <- performance(pred.mat, "tpr", "fpr")
plot(perf.mat, colorize = TRUE)

Btw, if you for some reason really wanted to use lines to plot consecutive ROC curves you would have to do sth. like this:

plot(perf) 
lines([email protected][[1]], [email protected][[1]], col = 2)

Kapor

Answer 2

Echoing @adibender, and adding a comment: the example doesn't cover how to set separate colors for each individual curve using the second (plot all at once) approach. In this case, pass col as a list:

library(ROCR)
data(ROCR.hiv)
x   <- prediction(ROCR.hiv$hiv.nn$predictions, ROCR.hiv$hiv.nn$labels)
ROC <- performance(x, "tpr", "fpr")
plot(ROC, col = as.list(1:10))

Gertrudis

Answer 3

R has functions for draw several plots in one window. And if package doesn't support several plots in one window,you can solve problem with standard instruments of R. Other way: Example of several ROCs Article with this script:An example of ROC curves plotting with ROCR

Dorcy

Thanks! The only problem is that this solutions seems to be for cross-validation data, so it's not possible for example to add a legend to distinguish which curve belong to which classifier. And which are the instruments for drawing several plots in the same window? – Canonicals

You can use function subplot or see this document: [link] statmethods.net/advgraphs/layout.html – Dorcy