views:

281

answers:

11

This is essentially a design patterns question:

I was expecting to query a database to get a list of stocks(shares/securites whatever) that are most highly correlated for a given stock.

Instead I thought maybe I should create an object which has a static HashMap and store my data in there. Then "query" it every time I need to.

Would there be anything wrong with this approach, as I believe it would improve performance significantly over querying a database for the same data. The amount of data is relatively small and doesn't grow so that won't cause a problem. Might there be any issues that will bite me later?

+4  A: 

I would still use the database for backup reasons, but on the client use a caching api like oscache to store the data on the local filesystem for quick access, then if the system goes down restore the cache from the database and the carry on using the cache in your code

ssmithstone
+1 database is all about management. It is not called RDBMS - Related database MANAGEMENT SYSTEM for nothing.But if you don't have read/write transactions all the time, loading the database into a memory structure can improve speeds vastly. I did this once in a company where the db was only updated once a week.
Marco van de Voort
+1  A: 

There is nothing wrong with this approach in general. Just make sure you don't access your static class directly, because then you create a "bad singleton". (Like a global variable)

Instead introduce some kind of registering mechanism to register this class at all objects which will query it. Than you'll have a "good singleton" and there will be no later issues.

public class MyDataClass {
    private MyDataClass() { }
    public static MyDataClass getInstance() {
        if (instance == null) instance = new MyDataClass();
        return instance;
    }
    private static MyDataClass instance = null;
}

public class MyDataProcessor {
    public void registerData(MyDataClass data) { 
       this.data = data;
    }
    public void process() {
        assert(data != null);
        data.getData(...);
    }
    private MyDataClass data;
}

Of course this approach might have some issues after all on an architectural layer when it comes to persistence, robustness, ... Perhaps it would be better to use a database with a caching layer, but that highly depends on the actual needs of the users.

DR
When you say don't access the static class directly - do you mean don't access the HashMap directly. I have created the equivalent of get and set methods. Is that what you are referring to?
Ankur
No that's not what I meant. I added an example.
DR
+1  A: 

The problem with your approach is not having a separation between code and data.

That results in some disadvantages:

  • you can't do backups
  • you have to recompile/redeploy your application to change data

As you do not seem to need a relational database for that, I suggest you to use an embedded key-value-database like BerkeleyDB, which is also very fast.

Markus Lux
+1  A: 

The issues that will get you are if the data does change and you need to update the hash and recompile every time.

That being said, you know the data, you can tell how much that is going to affect you.

One option could be that you keep the data in a database, but then cache it to a HashMap.

Robin Day
+1  A: 

Normally what comes back and bites is the assumption that something will never change. So if you later get requirements that involve updating the values, adding new ones and having multiple clients accessing the data if then you already are using a database you will have less work. Instead do some caching on the client side to get the performance boost. Just my 2c.

Anders K.
A: 

Store the data in a HashMap if that's easier.

You can then serialise a copy of this using XStream. This will create an XML version of the HashMap object.

So your application can write this to disk as required, and you can edit it by hand and reload it into the application. This means you have your HashMap, and a configurable representation on the disk that you can change as required.

Brian Agnew
A: 

Static caching can work.

Keep in mind that if you have multiple applications accessing the same database you will need some sort of messaging bus and a distributed cache

Sam Saffron
+3  A: 

If a read-only cache that does not get updated automatically is all you need for your application, then there's nothing wrong with this approacht.

There is one caveat though: If the amount of data (inlcuding the overhead of Java objects and the HashMap) becomes too big to be held entirely in RAM, performance will decrease quickly and drastically (by a factor of 10,000 or more). Database systems are designed for efficient access to on-disk data; a HashMap isn't.

Michael Borgwardt
That last part is very useful to keep in mind
Ankur
A: 

Should you use a cache? Yes, if it's a lot faster and either you never want to update the data, or that it's not important if the cached data isn't up-to-date with the real data.

You should put this access behind an API, and hide the caching behind the abstraction. Then you can play with different caching strategies, or discard the idea of caching completely without having to rewrite the rest of your code each time.

Paul Hankin
+1  A: 

Your idea is not that bad, however I would personally go for a caching solution.

Ignore the people who say that you would need to recompile/redistribute each time the data changes, as I don't think they understand what you mean by static. i.e. they think you are going to have hard coded the values in your source.

kingmunkyman
A: 

Again, depending on what you do, this could be fine. As long as you had some method of persisting what's in the HashMap or the like in case of server maintainence/shutdown.

All you need is a serialization method. I saw that XStream was mentioned above.

There is also a library called Prevayler. This serializes changes on the fly but lets you keep everything in memory. So in the even of sudden power failure where you don't have time to shutdown properly, this may be of some use.

Experiment with different methods to see what works for you and extract the details away behind a DAO.

Clinton