I'm pretty sure no one has ever written an application to analyze financial data (sarcasm).
Regardless, I'm considering writing one for fun and need a way to access (1) large amounts of historical data and (2) real-time fluctuations in stock prices etc... (my finance jargon is weak).
Is there an API (free or pay) that I can hook into to pull this sort of data from? The Google financial API doesn't seem to provide anything but data for specific portfolios.
Thanks!
Tom
For the historical side, you can pull free daily price data from Yahoo Finance, the download links for each symbol spit out a CVS file. There are software wrappers around that such as Finance::Quote that automate collecting a bunch of data from them.
As for intraday history, the recently closed OpenTick used to be the way to go, so there's a bit of a void left behind after their announcement. I used to download a lot of intraday stuff from the quote.com site with a paid subscription; that works, but was a very manual process rather than a straight API pull last time I used them.
It's possible to get both real-time and historical data of decent quality if you open an account at Interactive Brokers and talk to their API. I wrote a whole trading system that executed orders and everything with that a few years ago, worked fine. If you search specifically for people using the IB API, you'll discover a pretty large community working on software like what you're considering.
On the free side, Yahoo has historical data back to 1962 available in CSV format. It's not comprehensive in terms of stats (just the basics) or range (mostly focused on U.S. exchanges) but pretty decent -- most crucially, it supports "adjusted close" prices that take dividends & splits into account. Some of the other consumer friendly finance websites (MSN, BB, Morningstar, Google, etc) are ok too but I've come to prefer Yahoo.
I doubt you will find real-time quotes in a scraper-friendly format for free. It's only in the last few years that discount online brokers have started offering "free" real-time quotes when trading inside your account. If you already have such an account, see if your brokerage offers an API -- many do. If not, Interactive Brokers is a common recommendation for automation minded traders.
IMO, any kind of "real time" data is useless without Level II quotes. There are a handful of websites like this one that are open to the public, but they're clearly geared to avoid bots. Once your needs reach this level you'll need registration at minimum, and probably a small paid subscription / account maintenance fee.
When it comes to $$ services the major players are Bloomberg, Factset, and Thompson/Reuters. You can get just about any info imaginable...for the right price. Note: many of their services are oriented toward running your own data mining algorithms on their servers and returning only the result set, though usually you have the option of doing either. (Depending, of course, on what you're willing to pay.)
There are tons of smaller companies as well that specialize in different niches. As you can imagine, analysts & hedge fund managers & the like are willing to throw buckets of cash at anything that might give them an edge, so the competition and variety are large.
One issue you'll find as you expand your range is that different data providers use different identifiers for the same security. U.S. stocks are mostly ok; the problem arises with foreign exchanges, ETFs, indexes, and so on. There exist standards like ISIN, but unfortunately that's generally not what companies expect you type into their web forms or POST into their API.
For close to realtime stock data see this question. The yahoo method works well, index's like the Dow are realtime, companys such as Microsoft are delayed by 10 to 15 minutes.
I use the data from these guys. Its 1 min data (ASCII). Since most of my trading is done on 3, 5 and 10 min it works great.
For real-time streaming data, try using your broker's API. My broker, TD Ameritrade, offers a free access to their real-time feed if you open an account with them. You can find ActiveX and .NET examples at their web site.
You should also take a look at XIgnite. They have a comprehensive suite of APIs exposed via SOAP for all sorts of market data (realtime and historical) and fairly clear pricing.
Take a look at http://www.mergent.com/servius - lots of in-depth financial data, historical stock quotes, etc. in an easy-to-use REST API form.