What's the best scalable modern architecture for a high volume website (Java)

views:

111

answers:

+6 Q:

What's the best scalable modern architecture for a high volume website (Java)

We have a website that is getting over 1m hits a day.

The site is running off an old architecture (Struts Tomcat Hibenate MySQl) with no real scope for scalability.

Now that we are expanding further I am looking for a modern (not too bleeding edge please) web architecture.

My first thoughts were to use Google App Engine - but I'm not sure that is open enough.

Our developers are all Java so it would need to be something they could pick up easily.

Ideally I'd like to run the same way as DIGG, Twitter, Amazon etc.

If you know how they do it - or have a suggestion I'd be very interested to know.

I'd exhaust all possibility for "no real scope for scalability" before I rewrote my app. It might just mean that you can only scale horizontally by throwing more hardware and clustering at the problem.

Google App Engine will mean JPA instead of Hibernate and MySQL, because I believe they use BigTable in the back.

duffymo 2010-09-15 12:02:25

Appreciate that thought - the system is quite messy and we really do need a rewrite at this point - just want to make sure that we go down the right path.

Dan 2010-09-15 12:04:14

refactor instead of rewrite...

Thorbjørn Ravn Andersen 2010-09-15 12:25:59

Have a look if you can get clustering cheaply with Terracotta.

For a rewrite: Have a look at what is already present in Java EE 6. You might find it fits your needs nicely. Glassfish 3.1 will support clustering before you are done rewriting.

Thorbjørn Ravn Andersen 2010-09-15 12:05:22

I've seen Terracotta but I'm looking for a whole web stack. Where would that fit in - do you mean under servlets?

Dan 2010-09-15 12:08:18

Transparently scaling to more machines. Have a look...

Thorbjørn Ravn Andersen 2010-09-15 12:23:25

Hire a team or even better a company which has an experience in building scalable solutions. Promote current developers in Business Analysts and make them help the scalable solutions specialists with the business logic.

Boris Pavlović 2010-09-15 12:07:08

Is that really necessary? Surely a team of experienced developers should be able to deal with this assuming they are pointed in the right direction.

Dan 2010-09-15 12:10:30

+2 A:

Unless you've got major bottlenecks baked into the design, then hardware is almost always going to be the best way to scale.

By 'best' I mean

cheapest
quickest
best performance increase
guaranteed results

It sounds like your current setup is having problems? Can you give any information on what kind of problems you have?

Edit: Can you give info on your current hardware setup? How many web servers (presumably running Apache?), how many app servers, how many DB servers?

Qwerky 2010-09-15 12:07:41

The system gets slow and there are timeouts we have transaction problem sometimes as well.

Dan 2010-09-15 12:35:57

We have 2 web servers and one DB server.

Dan 2010-09-15 12:36:17

@Dan Are you measuring how hard your servers are working for eg CPU, disk and memory?

Qwerky 2010-09-15 13:48:02

@Dan If your app servers are running at high CPU then maybe add seperate http servers in front of them, or simply buy more servers and balance the load between them. There is a wealth of information around about MySQL tuning, but quick wins might be buying more memory (MySQL uses memory to cache results, resulting in less hits to the disk), buying faster disks, buying more disks (and arranging your data on them accordingly), regularly optimizing the table.

Qwerky 2010-09-15 13:59:36

ansaurus

tags:

views:

answers:

What's the best scalable modern architecture for a high volume website (Java)

related questions