How can I apply multithreading to the backpropagation neural network training?

Question

For my university project I am creating a neural network that can classify the likelihood that a credit card transaction is fraudulent or not. I am training with backpropagation. I am writing this in Java. I would like to apply multithreading, because my computer is a quad-core i7. It bugs me to spend hours training and see most of my cores idle.

But how would I apply multithreading to backpropagation? Backprop works by adjusting the errors backwards through the network. One layer must be done before the other can continue. Is there any way that I can modify my program to do multicore backdrop?

Answer 1

First of all don't use backpropagation. There are many other options out there. I would suggest trying RPROP (resilient propagation). It won't be that big of modification to your backpropagation algorithm. You do not need to specify learning rate or momentum. Its really almost as if you have an individual, variable, learning rate for every connection in the neural network.

As to applying multithreading to backpropagation. I just wrote an article on this topic.

http://www.heatonresearch.com/encog/mprop/compare.html

Basically I create a number of threads and divide up the training data so each thread has a near equal amount. Then at regular intervals I merge the weights back together from the independent threads. I provide some of the performance results at the above link. It does really speed things up!

Answer 2

For using more CPU cores with Java you can just try more options for JVM without changing the code:

• -server
• -d64
• -XX:-UseParallelGC

and other options at http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp

Answer 3

use paralel processing aplicable methos like Genetic Algorith so use a genetic algorith for the training ANN