I've been trying to learn about Neural Networks for a while now, and I can understand some basic tutorials online. Now i want to develop online handwritten recognition using Neural Network. So i haven't any idea where to start? And i need a very good instruction. In finally i'm java programmer.
What do you suggest I do?
Have a look at some of the project floating around the net:
To list just the three first links Google spits out for "java handwriting recognition"
Some hints to get you started:
If you can, work with vectors instead of bitmap graphics. Ideally, you should have the speed and direction of each stroke. It's often more simple to recognize a letter by the way in which a curve bends plus the speed at which it was drawn instead of the form.
Attack the problem with several approaches. Use neural networks, shape recognition, size, previous and next letter, dictionaries. All of them will give you different results with different error levels. This can help greatly to improve the results.
Good luck!
Start simple with character recognition on the Unipen database.
You will need to extract pertinent features out of raw trajectory data in order to form what's commonly called a "feature vector". For instance you could resample the data using an interpolation scheme to end up with n tuples, each tuple containing information such as:
Once you have a fixed size feature vector, you use it as the input to your neural network. Try MLP networks for a start.
You will have to experiment in order to decide which features are best.
If you need to get started on extracting features from Ink data, have a look at HP's Lipi Toolkit (note that their recognizers don't use neural networks though).
You can also have a look at this 15 Steps to Implement a Neural Network tutorial.
Neural network [if i am not mistaken] would work on interpreting patterns, You provide inputs to your program and the program searches that pattern in a set of stored patterns and based on the match provides a possible match.
In your case, the dimensions for the patterns could be data such as speed + direction or only direction, etc
I made a small simulation of a neural bot that chats on my site in a similar fashion.
The more patterns the program 'Learns', the more accurate answers it provides.
Introduction To Neural Networks for Java is a good introductory book and includes a handwriting recognition example.
Peter Norvig's Artificial Intelligence: A Modern Approach is a good book on general AI and explains a lot about the basics, and there is a section on Back Propagation neural networks.
To train your neural network you'll need datasets.
There's THE MNIST DATABASE of handwritten digits, or the Pen-Based Recognition of Handwritten Digits Data Set at the UCI Machine Learning Repository
The UCI ML repository has lots of great datasets, many of which would be good to train neural networks. Even if you don't know what they're about you can grab some and see if your ML system can do the classification tasks. Look at Classification tasks with a large number of attributes and instances, although you can try smaller ones too when you start out.
By the way, there are a lot more techniques besides neural networks, including Support Vector Machines, which are popular.
Neural nets need a lot of soak time. The concepts are easy enough, but they can be overwhelming to the beginner.
Take a look at what Jochen Fröhlich has done with neural networks in Java. It sounds like an ideal starting point for a Java programmer like yourself.
Hi,
the books: Character Recognition Systems: A Guide for Students and Practitioners and Recognition of Whiteboard Notes: Online, Offline and Combination are good points of starts.
Bear in mind that if your aim is to actually recognize these characters, your performance will stand and fall based on the quality and selection of the input features.
It is absolutely critical to choose the right features, and to preprocess (i.e. get rid of noise features, extraneous data, duplicate or strongly correlated features) as much as you can. In my experience, you'll get much better performance from the most boring and plain nearest neighbour implementation with good features than from a cutting edge algorithm with less well selected features.
For you, that means delaying reading the neural net literature for now (just take some off the shelf black-box implementation first) and reading up on what kind of image processing etc. real systems use. If your data can include pressure and speed info, all the better. Something like an LDA heatmap can be illustrative to initially see which features matter, and which don't.
For basic classification, there are tons of decent algorithms. Most work just fine and will work just fine for you. The difficult part isn't in picking or tweaking the algorithm, it's in avoiding the garbage-in-garbage-out scenario.
If you are looking for concepts, I suggest BrainNet,
Neural Networks - Part I: A simple handwriting recognition system in .NET
http://amazedsaint.blogspot.com/2008/01/neural-networks-part-i-simple.html
BrainNet will help you to
- Obtain a fair understanding regarding Neurons and neural networks
- Gain a good concept regarding intelligent systems
- Learn how to play with this neural network library to use it in your projects.
- Understand how to develop some cool neural network programs