I'm new to OpenStreetMap and I've been browsing the wiki and the net and I can't seem to find a tutorial anywhere but I've seen examples on the net.
Basically I want to generate my own OpenStreetmap and plot markers taking the latitude and longitude from a MySQL database and plotting them on my map. When the user clicks on a mark I'd like to have a popup. Basically I want this http://code.google.com/apis/maps/articles/phpsqlajax.html but for OpenStreetMap and not Google-maps.
Looks like they are using openLayer for map rendering. Here are some examples and api docs.
To accomplish this, you need to figure out the javascript for presenting markers on a "slippy map" interface.
OpenLayers is the name of a popular javascript library. It's free and open source. It's used to display a slippy map on the OpenStreetMap.org homepage, and various other sites around the web. It's often confused with OpenStreetMap itself, or people mistakenly get the impression that you must use OpenLayers if you want to embed OpenStreetMap maps on your site. Not true. There's lots of alternative javascript libraries for displaying a slippy map
...including the google maps API! You can set up a google maps display, but show OpenStreetMap "tile" images instead of google tiles. See Google Maps Example. This has the advantage of code compatibility, meaning you could follow through google maps tutorial you've linked there, but then drop in a cheeky bit of code to specify OpenStreetMap as the tile layer.
This has the disadvantage of showing an big evil google logo, and requiring an even more evil google maps API key, so all the cool kids are using OpenLayers.
There's various examples of using OpenLayers on the OpenStreetMap wiki. The "OpenLayers Dynamic POI" example shows the use of database for markers (although that example is a bit convoluted). Another popups example on my site
Hope that helps