Neural Networks in NeuronDotNet

Question

hello i am using neurondotnet library for ANN programing, my input layer vector is 400 and my output layer vector is 5 and my sample for learning is only 28. my problem is that i don't have any idea of how to choose how many hidden layer i should have and the size of their vectors. can anyone put me on the right direction ? thank you.

Answer 1

Check out their help guide http://neurondotnet.freehostia.com/manual/design.html

From the guide

Number of hidden layers (in Backpropagation Networks)

A Backpropagation network with no hidden layers cannot perform non-linear classification. (It cannot be used as a XOR function approximator). So, one hidden layer is a must for a backpropagation network. Further, it has been mathematically proved that a backpropagation network with single hidden layer when appropriately trained, can be used to approximate any function. So single hidden layer is the best choice in most cases.

Having multiple hidden layers speeds up the learning process and the trained network fits exactly with the training samples but fails to perform well on the test data. This effect is called overtraining where the trained network tends to memorize the training samples instead of learning them.

Answer 2

I found this FAQ which suggests some guidelines on determining what to do with the hidden layer. They note, in particular,

In most situations, there is no way to determine the best number of hidden units without training several networks and estimating the generalization error of each.

Read more: http://www.faqs.org/faqs/ai-faq/neural-nets/part3/section-10.html#ixzz0b0VxJHV1