How do I get a predictions list from running svm in e1071 package

Question

Hi

Q1: I have been trying to get the AUC value for a classification problem and have been trying to use e1071 and ROCR packages in R for this. ROCR has a nice example "ROCR.simple" which has prediction values and label values.

library(ROCR)
data(ROCR.simple)
pred<-prediction(ROCR.simpe$predictions, ROCR.simple$labels)
auc<-performance(pred,"auc")

This gives the AUC value, no problem. MY PROBLEM is: How do I get the type of data given by ROCR.simple$predictions in the above example? I run my analysis like

library(e1071)
data(iris)
y<-Species
x<-iris[,1:2]
model<-svm(x,y)
pred<-predict(model,x)

Upto here I'm ok. Then how do I get the kind of predictions that ROCR.simpe$predictions give?

Q2:

there is a nice example involving ROCR.xvals. This is a problem with 10 cross validations.

They run

pred<-prediction(ROCR.xval$predictions,ROCR.xval$labels)
auc<-performance(pred,"auc")

This gives results for all 10 cross validations.

My problem is:

How do I use

model<-svm(x,y,cross=10)     # where x and y are as given in Q1

and get all 10 results of predictions and labels into a list as given in ROCR.xvals?

Answer 1

Q1. You could use

pred<-prediction(as.numeric(pred), as.numeric(iris$Species))
auc<-performance(pred,"auc")

BUT. number of classes is not equal to 2. ROCR currently supports only evaluation of binary classification tasks (according to the error I got)

Q2. I don't think that the second can be done the way you want. I can only think to perform cross validations manualy i.e.

Get resample.indices (from package peperr)

cv.ind <- resample.indices(nrow(iris), sample.n = 10, method = c("cv"))
x <- lapply(cv.ind$sample.index,function(x){iris[x,1:2]})
y <- lapply(cv.ind$sample.index,function(x){iris[x,5]})

then generate models and predictions for each cv sample

model1<-svm(x[[1]],y[[1]])
pred1<-predict(model1,x[[1]])

etc. Then you could manualy construct a list like ROCR.xval