Which family should I start to learn? (Never did any programming on microcontroller)
My boss picked up the basics using AVR within a week without prior experience.
My vote goes to PIC for the extreme variety of devices availables. But I must say that when I started to use PICs, they was almost nothing else. Maybe now things are changed.
I started on a Motorola M68HC11, it was simple enough. I think you'll get about the same experience with any 8Bit controllers.
My first experience with microcontrollers was with an OOPic-R. It allowed me to make simple robotic experiments without worrying too much about the code. The object oriented programming flow makes everything work fast and is easy to program.
Recently, I tried another variety of PIC's, the dirt cheap PICAXE. The included programming interface is a breeze to work with. Also, to physically interface the PICAXE, you only need an RS-232 port to program it and two pins on the chip (no need to do level shifting). I've embedded the PICAXE in very small containers (SMD and DIP chips available) and it has worked quite well.
I have no experience with programming microcontrollers in assembly. If you want to try that, the AVR might be more suitable because of it's bigger user community.
As far as I know, the cheapest way to program an AVR using ATMEL's tools is the ATMEL Dragon. But that will set you back at least 80$.
Today AVR and PIC are probably the most common microcontrollers among hobbyists. Both have a very wide range of device variants and both can be used to achieve similar results.
For a beginner I would suggest AVR due to various reasons:
AVR family (tiny, mega) is coherent and easy to understand. The architecture is powerful and modern, and is especially suitable for C compilers. AVRs can of course be programmed in assembly too.
Due to its C-friendly architecture, there are quality C compilers available, both commercial and free. The ubiquitous GCC is ported to AVR and called avr-gcc.
For getting started all you really need is a handful of basic components, the AVR chip itself and a breadboard. Even the programming cable between PC and AVR can be built essentially for free (a so called wiggler). However, several commercial development kits are available, most notably Atmel's own STK500. A commercial development kit is more expensive way for getting started, but doesn't require practically any prior knowledge about electronics. Some development kits contain for example LCD displays so it's easy to get interesting stuff done.
It has a rich hobbyist community.
PIC is notorious for its peculiar architecture. Many love PIC for this, some hate it. AVR is more straightforward and doesn't seem to cause as much extreme and polar opinions.
Both AVR and PIC are used in many serious commercial applications. However, they are not the only options of course. My personal favorite microcontroller for both hobby and commercial work is Silicon Laboratories' C8051 family, most notably C8051F530. There is an excellent free C compiler and assembler for the C8051 family called SDCC.
Summary: There are lots of options, but please don't let that overwhelm you. Just pick one and start learning with it. Microcontrollers are, really, surprisingly easy to master once you just decide to get going!
I've done some PIC programming - mostly because I liked the idea the chip were only a dollar or two. However, for a beginner, making a decision solely on price is premature optimization.
Programming in assembler is an experience. You basically have to learn about 100 concepts before you can blink an LED. (Watchdog timer, reset pins, 8-bit counters/overflows, delay loops, hex, binary, bit-masking, interrupts, interrupt service requests, IO ports, etc.) It's all very educational - and a great feeling to get so close to the machine - but being able to code something in C will hide some of this complexity so you can focus on results. For this reason I would say go with the AVR. (And I believe the prices are now closer to PICs.)
Also: If you're interested in getting things done (and don't mind spending ~$30) check out the arduino. A guy selling them at my local electronics shop was saying he's selling tons of them to art students. (It uses the IDE from the Processing project, and compiles code with avr-gcc.)
Update: Fixed comment that Arduino runs interpreted code. Also updated the approx Arduino price.
I would suggest AVR. It has far surpassed PIC as the microcontroller platform of choice for general hobbyist projects. Most notably, consider the Arduino (and other *duino) platform, which provides a high end AVR in an easy to interface and popular form factor.
If you just want to know what is MCU programming, start from Arduino is a good idea. It's cheap, with a novice-friendly IDE (based on processing programming language, which has a similar syntax with C).
But this did not answer your quesiton, beacuse altough Arduino is based on AVR, you cannot feel there is an AVR MCU behind that modern IDE. :)
I vote for TI's MSP430 series. I've used PICs extensively (also Atmel chips a little) and by far the most important thing to me is a good debugging IDE. TI has done a pretty good job on this, and their C++ compiler works really well. You can get going with an eval board for less than $100 including an IDE + USB-debugger. The PICs have better & more diverse hardware peripherals, but MPLAB is a piece of crap and the only C++ IDE for PICs is one by IAR which is rather expensive. (more than $2K)
I very much prefer the AVR over PIC, whose architecture I find a bit messy. This may be just me, and it won't trouble you if you can write in a high level language, most likely (some dialect of) C.
Since you're new to microcontrollers I presume performance will not be the issue, so instead I would look for availability of development tools: prototyping boards, IDE and simulation/debugging tools. Personally I liked AVR Studio (Atmel's free development environment) a lot.
Jason mentions the TI's MSP430, which is an excellent controller indeed, especially if you're in very low power applications. But I wouldn't recommend it for a newbie, since configuration is a bit cumbersome. (I recall that the description of the oscillator covered 20+ pages in the user manual.)
I had much more success with PIC while I was just getting started. I tried to get a simple starter kit from Arduino and just couldn't get a good basic kit without spending more than $100-200 nickel and diming the setup together. Got a great little starter kit from PIC for about $40 and it has everything: IDE, programmer, starter board with built in circuitry for demos and tutorials. One purchase. Also, the PIC environment was very easy to get set up and working. I was playing with it within an hour.
I/we chose PIC mostly because there is more peripheral hardware for the same price. And more importantly, you can't even find comparable AVRs. I did choose one of the legacy free versions though (started with PIC18, migrated to dspic33)
The IDE is free, the (C) compiler is free in the student version (that disables optimization after the first month). Entry level programmers are fairly cheap too. If you have heaps of interrupts, counters and timers, there is a chance you won't need optimization at all.
Note that the above remarks about AVR catering more to HLL development are slightly outdated unless you really go for the legacy architectures like PIC12 and 16.
One typically programs the more modern PIC18 (8-bit) and the 16-bit architectures (24F,30F and dspic33 which are based on the same principal core) in C. The 16-bitters even use GCC. There are also 32-bitters now, but they rival more with ARM in the audio/video processing scene. Strangely enough, the modern ones are often cheaper than the old ones. Probably they are produced on in a more modern process that has higher yields.
Another note: meanwhile Microchip/PIC bought Atmel/AVR, but I assume that for the first few years that won't affect the productlines much.
I'm really looking forward to the 60MIPs ethernet enabled 16-bitter that is going to be released this summer (afaik streetprice just above EUR 10)
On of the best features of AVR is the community in the forums at www.avrfreaks.net. You get a bunch of experienced electronics engineers hanging out, willing to help newbies to get going.
Some people commented on the strange (and C unfriendly) architecture of the PIC micro. This is true of the smaller PICs, but the 16 bit chips (PIC24F, dsPIC30, etc) have very clear architectures that work very well with C.
The PIC24F line has the ability to assign pins to functions (timers, A/D, serial I/O) on the fly, making it a bit easier to design with. The MPLAB environment for debugging and development is quite nice.
Some more dicussion about the superiority of AVR, on the other Stack Overflow: http://www.embeddedgurus.net/stack-overflow/2009/04/pic-stack-overflow.html
I would make my choice based on availability of a C cross-compiler. In the past, that would make AVR the choice. I'm not sure what the status is now.
I've programmed a PIC in assembler, and it was not much fun. C is much nicer in many ways.
I would say that i fount the 8051 microcontroller the most easiest and Atmel has come up with microcontrollers with so many inbuilt functions .... but still people are more preferably using the AVR's... my hand would go up with the 8051 family ( if found comfortable ) else the AVR's ...
I love AVR. its easy to program and resources available. there are few community like arduino works with it.
I don't understand what the big deal with arduino is, it will ruin your chance of ever understanding what is actually happening. I program with AVR's and PIC's regularly, basically there is not much difference, I can't see what the big fuss is all about. However for a beginner stay away from arduino, it may be simple, but thats the trap, it gives you no concept of hardware architecture, and no idea what is happening behind the scenes, the stuff beginners need to learn to be an effective programmer. When I was a beginner I started out with an ATmega32 a $20 USBasp programmer, AVR Studio (Free) and AVRDude (Comes with WinAVR) and followed the intro tutorials in AVR Freaks. That is all you need, Done!!!
P.S. If you want to really learn how to program micros and have the time learn the assembler for your micro and you will be 20 times the C programmer than someone who started out using arduino.