I want to be able to build my own small electronic device that just has a few buttons, and stores each keypress inside a memory.
I am a pretty experienced programmer but I have no idea where to start regarding hardware, or what to search for on google.
Please give me any info that can help me out.
I can only advice how I started:
Get a bread board and some ATMel microcontrollers (ATMega16 is perfect for beginners). You can either buy an STK500 or you use the In-System-Programming feature, and ask someone that can build you a programmer (I recommend USBasp). When you use Linux, use AVRdude and the gccavr toolchain. Under windows, you can use the WinAVR studio.
And buy some components, like resistors, capacitors, probably a quartz crystal to make simple circuits.
That's how I started anyway.
Arduino boards are inexpensive and the platform is wildly popular. I buy stuff like that from sparkfun.com in the developer area down the left side of the page. At sparkfun you will also find many other similar eval boards from various vendors. I like the lillypad over the arduino pro mini only because it has the programming pins already soldered. You will want the ftdi usb serial board thing for power and programming. I am a fan of the armmite pro, which is arm based. the mbed2 is real easy when you plug it in it shows up like a thumb drive, you copy your .bin file to it and press the button and it loads the program and runs it. The blue leds give me migraines but that was solved by replacing with green leds. I have a number of the header style olimex boards, good stuff, have never used the ones with displays and buttons though. Going to other sites the ez430 msp430 is a good starting platform but no buttons which you are interested in using, the stellaris cortex-m3 based family is good I would skip the 811 board and go for maybe the 1978 or something in that range, the 811 is too easy to brick.
Most of the ones mentioned above (not the olimex boards) have sandboxes for you to play in safely (turn key development environments), but at the same time you are not locked into those environments, you can do your own thing if you like, use different toolchains, flash programmers, etc. I personally would avoid the lpcxpresso for that reason, painfully tied to both windows and their sandbox.
the sam7-h256 for example, an olimex board, is powered by the usb, and a program called sam-ba is used to load your programs into the chip. That particular board does not have buttons but boards in that family are also programmed the same way. basically one usb cable for power and programming. And like the AVR (arduino and avr butterfly are based on the avr micro) the at91sam7s is an atmel part. Atmel seems to have that edge over competitors for better documentation and support. At least my belief is that contributes to the popularity of the avr (the avrfreaks website existed before the avr butterfly and the arduino), and it certainly makes me like their arm based products.
short answer, start with http://www.sparkfun.com along the left side click on Development Tools, and there are many solutions. You want to find something like the arduino, armmite pro, mbed, that either standalone or with a ftdi based usb thing you both power the board, and have an interface for programming. There should be links on the page to websites with development tools, compilers, etc, and tools for actually doing the loading of the program on the board. I recommend trying the arm, avr, and msp430 micros, as well as different vendors (many different arm based solutions with their own pros and cons for example).
Arduino is based on Atmel AVR, so the same Arduino or STK500 board can be programmed with AVR Studio + WinAVR combination, which is nice if you already have experience in GCC. For all additional info and tutorials take a look at http://www.avrfreaks.net