beginner question on investigating on samples in Weka

Question

Hello there,

I've just used Weka to train my SVM classifier under "Classify" tag. Now I want to further investigate which data samples are mis-classified,I need to study their pattern,but I don't know where to look at this from Weka. Could anyone give me some help please? Thanks in advance.

Answer 1

You can enable the option from:

You will get the following instance predictions:

=== Predictions on test split ===

 inst#     actual   predicted  error prediction
   1   2:Iris-ver  2:Iris-ver         0.667 
  ...
  16   3:Iris-vir  2:Iris-ver   +     0.667 

---

EDIT

As I explained in the comments, you can use the StratifiedRemoveFolds filter to manually split the data and create the 10-folds of the cross-validation.

This Primer from the Weka wiki has some examples of how to invoke Weka from the command line. Here's a sample bash script:

#!/bin/bash

# I assume weka.jar is on the CLASSPATH

# 10-folds CV
for f in $(seq 1 10); do
    echo -n "."

    # create train/test set for fold=f
    java weka.filters.supervised.instance.StratifiedRemoveFolds -i iris.arff \
        -o iris-f$f-train.arff -c last -N 10 -F $f -V
    java weka.filters.supervised.instance.StratifiedRemoveFolds -i iris.arff \
        -o iris-f$f-test.arff -c last -N 10 -F $f

    # classify using SVM and store predictions of test set
    java weka.classifiers.functions.SMO -C 1.0 \
        -K "weka.classifiers.functions.supportVector.RBFKernel -G 0.01" \
        -t iris-f$f-train.arff -T iris-f$f-test.arff \
        -p 0 > f$f-pred.txt
        #-i > f$f-perf.txt
done
echo

For each fold, this will create two datasets (train/test) and store the predictions in a text file as well. That way you can match each index with the actual instance in the test set.

Of course the same can be done in the GUI if you prefer (only a bit more tedious!)