You should probably start with a little theory and simple examples such as the midpoint displacement algorithm. You should also learn a little about Perlin Noise if you are interested in generating graphics. I used this to get me started with my final year project on procedural generation.
Fractals are closely related to procedural generation.
Terragen and SpeedTree will show you some amazing possibilities of procedural generation.
Procedural generation is a technique that can be used in any language (it is definitely not restricted to procedural languages such as C, as it can be used in OO languages such as Java, and Logic languages such as Prolog). A good understanding of recursion in any language will strengthen your grasp of Procedural Generation.
As for 'serious' or non-game code, procedural generation techniques have been used to:
- simulate the growth of cities in order to plan for traffic management
- to simulate the growth of blood vessels
- SpeedTree is used in movies and architectural presentations