We all hear a lot about scaling issues in Rails. I was just curious what the actual costs in handling a HTTP request is in the Rails framework. Meaning, what has to happen for each and every request which comes in? Is there class parsing? Configuration? Database Connection establishment? ...
What is the best MemCache solution for ASP.NET applications running in a windows server environment? Why? ...
Here's the problem I'm having, I've got a set of logs that can grow fairly quickly. They're split into individual files every day, and the files can easily grow up to a gig in size. To help keep the size down, entries older than 30 days or so are cleared out. The problem is when I want to search these files for a certain string. Right n...
For a complex web application that includes dynamic content and personalization, what is a good response time from the server (so excluding network latency and browser rendering time)? I'm thinking about sites like Facebook, Amazon, MyYahoo, etc. A related question is what is a good response time for a backend service? ...
The ASP.NET application that I am currently responsible for at my day job has hit its limit in terms of its ability to scale inside a single server. Obviously we are working toward moving session out of process and the test and hopefully deploy date draws near. I would like to draw on the experiencies of people using the built in load ba...
New school datastore paradigms like Google BigTable and Amazon SimpleDB are specifically designed for scalability, among other things. Basically, disallowing joins and denormalization are the ways this is being accomplished. In this topic, however, the consensus seems to be that joins on large tables don't necessarilly have to be too ex...
There's a cots(commercial off-the-shelf) application that I work on customizing, where a couple of pages take an extremely long time to load for certain distributions of data. (I'm talking approximately 3 minutes for a page to load in this instance... and the time is growing exponentially). Clearly this is unacceptable but are there st...
Hi all, I'm creating a database, and prototyping and benchmarking first. I am using H2, an open-source, commercially free, embeddable, relational, java database. I am not currently indexing on any column. After the database grew to about 5GB, its batch write speed doubled (the rate of writing was slowed 2x the original rate). I was ...
Is using a handrolled POCO queue class using pseudo code T Dequeue() { lock(syncRoot) { if(queue.Empty) Thread.Wait(); } } void Enqueue(T item) { queue.Enqueue(item); Thread.Notify(); } for WCF request queueing a scalable approach? ...
I'm starting to develop a web application in PHP that I hope will become incredibly popular and make me famous and rich. :-) If that time comes, my decision whether to parse the API's data as XML with SimpleXML or to use json_decode could make a difference in the app's scalability. Does anyone know which of these approaches is more eff...
Recent conversations with colleagues have produced varying points of view on this matter. What say you, SO members? I know, even the concept of scalability can be taken in so many different ways and contexts, but that was part of the discussion when this came up. Everyone seemed to have a different take on what scalability really means....
How do large web sites which cannot be completely stateless achieve extreme scalability at the web tier? There are sites like ebay and Amazon, which cannot be completely stateless, as they have a shopping cart or something like that. It isn't feasible to encode every item in the shopping cart into the URL, nor is it feasible to encode e...
Hi all, I'm creating an app that will have to put at max 32 GB of data into my database. I am using B-tree indexing because the reads will have range queries (like from 0 < time < 1hr). At the beginning (database size = 0GB), I will get 60 and 70 writes per millisecond. After say 5GB, the three databases I've tested (H2, berkeley DB,...
I've been arguing with some friends on the university, and we can't get to a point to which is the framework with more scalability for web applications (and still very fast). One calls for jsp, the other one for ruby other for php and so on. Could i ask you to clarify us on what as more scalability potential? Tks, hope I'm not duplicat...
The goal is having a way to guarantee a certain SLA (Service Level Agreement) for some of the web services to guarantee backend database uptime. Ideally this should be achieved with no specific code in the services but by controlling the underlying infrastructure/pipeline. ...
I'm a big fan of http://highscalability.com/ and have been looking in my current development to decompose my application along functional boundaries as a route to being able to scale out the server side, specifically the database layer. What this involves is implementing different functional components of the application (we have several...
Short version If I split my users into shards, how do I offer a "user search"? Obviously, I don't want every search to hit every shard. Long version By shard, I mean have multiple databases where each contains a fraction of the total data. For (a naive) example, the databases UserA, UserB, etc. might contain users whose names begin ...
It there any article/book that defines upper bounded design limits for WS timeouts? Do you timeout at the server or recommend the client specific timeouts too? Is there a common best practice like "never design WS that can take longer than 60 seconds, use an asynchronous token pattern" I am interested in knowing what you do or your opi...
I'm working with a database schema that is running into scalability issues. One of the tables in the schema has grown to around 10 million rows, and I am exploring sharding and partitioning options to allow this schema to scale to much larger datasets (say, 1 billion to 100 billion rows). Our application must also be deployable onto se...
When starting a new ASP.NET application, with the knowledge that at some point in the future it must scale, what are the most important design decisions that will allow future scalability without wholsesale refactoring? ...