I want to create a simple search engine for learning purpose.
I want to know how to index a simple blog site.
A blog site has many pages and in every page there is a blogpost. But in every page there are other stuff in common as well ( header, footer, category block and other stuff ).
In your opinion, How can I index this blog ?
The program language doesn't matter obviously.