scalability

Dynamic / auto scaling

Hi, I was curious if anyone has experimented with auto-scaling web or db tier in ec2 or other cloud computing infrastructure? It seems theoretically possible but I am curious what the practical limitations are/maybe. Thanks, manish ...

Books on Web services scalability

Can anyone point me to some books on Web services scalability? EDIT Thank you for your answers. What I was looking for was a bit more specific. Basically scalability in an SOA architecture. I was able to find a couple of books that helped: Web Service Contract Design and Versioning for SOA Service-Oriented Architecture (SOA): Con...

Why are some websites spread across www2, www3 sub-domains whilst others manage scaling without it?

I know it's to do with having a variety of load balancing servers, but why do some sites make use of differently named "www" sub domains (www2.somesite.com, www3.somesite.com etc) where as other can be perfectly massive without doing this - ie all traffic is to www.hugesite.com. Does it indicate certain architectural decisions / have a ...

How do I speed up deletes from a large database table?

Here's the problem I am trying to solve: I have recently completed a data layer re-design that allows me to load-balance my database across multiple shards. In order to keep shards balanced, I need to be able to migrate data from one shard to another, which involves copying from shard A to shard B, and then deleting the records from sha...

Is CodeIgniter a wise choice for large applications?

Hi, I keep on reading how great codeigniter is from a development standpoint. And I am sure that using the framework will make the development process quicker. But the question I ask myself is, will there be a difference to a individually made framework, which caters to your needs? Is CI, despite the advertised small footprint, going ...

Connecting thousands of clients to a Jabber server through a single connection

We are using Openfire (Jabber) to enable chat and presence capabilities to our MMORPG. In our server architecture clients only open a single connection with the game server, and upon login, the game server creates a new connection to Jabber for this new client. The problem is, we don't want to open a new connection to Jabber for every c...

Running a Asp.net website with MS SQL server - When should i worry about scalability?

Hello All, I run a medium sized website on an ASP.net platform and using MS SQL server to store the data. My current site stats are: ~ 6000 Page Views a day ~ 10 tables in the SQL server with around 1000 rows per table ~ 4 queries per page served The hosting machine has 1GB RAM I expect by the end of 2009 to hit around: ~ 20,000...

APC Enabled but Apache Still Opening Files?

Hey guys, I'm working on a high-traffic webserver farm serving dynamic PHP code which includes around 100 files on most requests. APC opcode cache is enabled, include_once_override is enabled, and I have allocated 64MB ram to the cache, yet when I strace an apache process I still see it open()ing and stat()ing all of these includes for ...

Most efficient way to match a certain number of items in a db.Model ListProperty

In reference to this different but not unrelated question I will borrow the example models. class Foo(db.Model): bars = db.ListProperty(db.Key) class Bar(db.Model): pass If I have a certain Foo entity and I want to get all of the other foo entities also containing a certain bar Key in its bars ListProperty, I would use the following...

Scalable http session management (java, linux)

Is there a best-practice for scalable http session management? Problem space: Shopping cart kind of use case. User shops around the site, eventually checking out; session must be preserved. Multiple data centers Multiple web servers in each data center Java, linux I know there are tons of ways doing that, and I can always come up wi...

Storing Very Granular User Preferences

I've seen a few questions on here about storing user preferences, but they seem to be referring mostly to a fairly minimal set of preferences. I'm currently working on a highly customizable web app, which will need to store a great number of preferences, and I'm struggling with how to store them. The kind of preferences I'll be storing ...

Which is faster: Many rows or many columns?

In MySQL, is it generally faster/more efficient/scalable to return 100 rows with 3 columns, or 1 row with 100 columns? In other words, when storing many key => value pairs related to a record, is it better to store each key => value pair in a separate row with with the record_id as a key, or to have one row per record_id with a column f...

Good blogs/articles/ books on building highly scalable systems?

Any good blogs/articles/ books on designing highly scalable systems? Good source code for studying would help a lot too. ...

php security and scalablity

hi is there is any book on php site security and on scalability ...

Is there a fast and scalable solution to save data?

I'm developing a service that needs to be scalable in Windows platform. Initially it will receive aproximately 50 connections by second (each connection will send proximately 5kb data), but it needs to be scalable to receive more than 500 future. It's impracticable (I guess) to save the received data to a common database like Microsoft...

C++ Socket Server - Unable to saturate CPU

I've developed a mini HTTP server in C++, using boost::asio, and now I'm load testing it with multiple clients and I've been unable to get close to saturating the CPU. I'm running on a 4-cpu box, and getting about 50% usage of one cpu, 20% of another, and the remaining two are idle (according to htop). Details: The server fires up on...

Optimizing Kohana-based Websites for Speed and Scalability

A site I built with Kohana was slammed with an enormous amount of traffic yesterday, causing me to take a step back and evaluate some of the design. I'm curious what are some standard techniques for optimizing Kohana-based applications? I'm interested in benchmarking as well. Do I need to setup Benchmark::start() and Benchmark::stop() f...

Making decisions about the architecture of your project; what is your decision process like?

Many of us, who have designed and developed systems from scratch, have been in situations where you had to make tough decisions about the architecture of a project. Where did you, or would you, draw the line on taking the "next step" in building an architecturally sound, and scalable system? I've built a large scale web site that was r...

Are static methods good for scalability ?

Does static methods and class are good for scalability ? I think so static class/method improves scalability of application and instance methods doesn't scales much. So is it good programming practice to write static method where ever it is possible ? ...

Mutable or immutable class ?

I had read in some design book that immutable class improves scalability and its good practice to write immutable class wherever possible. But I think so immutable class increase object proliferation. So is it good of going immutable class or better go for static class (A class with all the methods static) for improve scalability ? ...