Neural Network Training

Question

I try to recognize pictures of every digit. I have removed everything else than the digit so that there is almost no noise other than the digit is placed differently on the pictures. I use Neuroph's image recognizing gui and have some questions about training.

It seems that the larger resolution I use for the pictures the worser the training becomes. Why is this? I have 100 pictures in my training set. 10 of each digit. Is that maybe too little? Why is every training no matter what i do just converging to some number usually between 2-3 in total network error.

Hope that you can help.

EDIT:

Here is a picture of one of the trainings

It doesn't learn much

Answer 1

The information increase is exponential!

When you provide images with higher resolution, you end up giving extra information which has to be evaluated.

If your resolution is 10x10px, that would result in 100 pixels per image and you have 10 images per digit, that's 1000 pixels per digit.

Now if you double the resolution to 20x20px it would result in 400 pixels per image or 4000 pixels per digit.

In other words: increasing the resolution results in an exponential increase for the number of pixels you have to evaluate per each digit.

Adding more (unnecessary) pixels increases the chance of errors:

Usually, in machine learning the pictures are stripped down to the bare minimum (both in size and color) in order to provide as little room for error as possible. When you have more pixels, the algorithm may learn something about those pixels that is completely irrelevant to the actual form in the image.

Answer 2

For one, I have to second Lirik's comment that increasing the resolution is not helpful here.

Aside from that, 10 samples of each digit is most likely not enough data for your neural net to train with. 10 samples of a digit might not even be enough training data if you were training a human to recognize digits in some wierd language that they don't understand. See if you get better results by increasing the size of your training data to around 250-300 of each digit. Exactly how much you will need is hard to say without knowing anything about your particular net.

My guess is that you will see better results with more training data and lower resolution images.