Financial Data/Formula Calculation (Storage / Performance)

Question

Hi. I am currently in the analysis phase of developing some sort of Locale-based Stock Screener ( please see Google's' for similar work) and I would appreciate advice from the SO Experts. Firstly the Stock Screener would obviously need to store the formulas required to perform Calculations. My initial conclusion would that the formulae would need to be stored in the Database Layer. What are your ideas on this? Could I improve speed( very important) by storing formulas in a flat file(XML/TXT)?

Secondly, I would also like to ask advice on the internal execution of formulae by the Application. Currently I am leaning towards executing formulae on parameters AT RUN TIME as against running the formulae on parameters whenever these parameters are provided to the system and storing the execution results in the DB for simple retrieval later( My Local Stock Exchange currently does NOT support Real Time Stock Price updates). While I am quite certain that the initial plan ( executing at run time) is better initially , the application could potentially handle a wide variety of formulae as well as work on a wide variety of input parameters. What are your thoughts on this?

I have also gone through SO to find information on how to store formulae in a DB but wanted to enquire the possible ways one could resolve recursive formulae i.e. formaulae which require the results of other formulae to perform calculations? I wouldn't mind pointers to other questions or fora at all.

[EDIT] [This page]2 provides a lot of infromation as to what I am trying to achieve but what is different is the fact that I need to design some formulae with SPECIAL tokens such as SP which would represent Stock Price for the current day and SP(-1) would represent price for the previous day. These special token would require the Application to perform some sort of DB access to retrieve the values which they are replaced with.

An example formula would be:

(SP/SP(-1)) / 100

which calculates Price Change for Securities and my idea is to replace the SP tokens with the values for the securities when Requested by the user and THEN perform the calculation and send the result to the user.

Thanks a lot for all your assistance.

Answer 1

Kris, I don't mean to presume that I have a better understanding of your requirements than you, but by coincidence I read this article this afternoon after I posted my earlier comment;

http://thedailywtf.com/Articles/Soft_Coding.aspx

Are you absolutely sure that the "convenience" of updating formulae without recompiling code is worth the maintenance head ache that such a solution may possibly become down the line?

I would strongly recommend that you hard code your logic unless you want someone without access to the source to be updating formulae on a fairly regular basis.

And I can't see this happening too often anyway, given that the particular domain here, stock prices, has a well established set of formulae for calculating the various relevant metrics.

I think your effort will be much better spent in making a solid and easily extensible "stock price" framework, or even searching for some openly available one with a proven track record.

Anyway, your project sounds very interesting, I hope it works out well whatever approach you decide to take. Good luck!