I believe there's a large productivity boost when going from one monitor to two, and pretty much every piece of research on the matter agrees. However, I also believe that there are diminishing returns when going from two to three (or more). Not sure what the research says on that.
To me, the key to using multiple monitors depends on how many "modes" you typically work in. For example, most developers have modes for coding, communication, research, etc. Depending on the type of work you do, you might also have a "design" mode or want things like media/music players always accessible.
I use two large monitors and prefer to dedicate a primary monitor to my current mode - typically dev-related tasks. I use the secondary monitor for communication, so it's typically got Outlook, Skype, and Trillian on it. There's some extra space over there for reference material - I'll tear off a Chrome tab and place it over there with things like documentation, samples, etc. When I switch out of dev mode, it's typically for a longer period of time (like working on doc, dealing with spreadsheets, etc), so I just use all of those apps on the primary instead of dev apps.
Occasionally, I find it helpful to use "virtual desktops" when I am going to switch modes more frequently - but that's not typical. The main thing is recognizing that there's only one "active" mode at a time. Someone else answered that two is good because you can't look at more than two things at a time -- I'd argue you can't look at more than ONE thing at a time.
So for me, having a primary/main workspace and managing that is key, and then I use the secondary workspace for things that are intermittent and more transient in nature (media, communications, etc). If you have a lot of more transient tasks/apps that you use (or prefer to also have Twitter, Facebook, or video windows open as well), and you have the desk space for it, then I could see how a third monitor would be helpful -- I just wouldn't look for it to be as mind-blowing an upgrade as going to a second monitor was.