scalability

Key factors for designing scalable web based application.

Currently I am working on web based application. I want to know what are the key factors a designer should take care while designing scalable web based application ? ...

IIS scalability

I have been tasked recently to estimate number of concurent users that IIS can handle. Unfortunately, I have 0 experience with IIS, so I hope someone will guide me where to look. I hope I will not have to do my own benchmarking in this phase. I would like to know what is the number of concurrent connections that it can handle. The clie...

processing large number of files - any library or utility to help?

I'd like to process a large number of files - think video transcoding, like youtube, where you have to transform everything into FLV format or image processing where you create thumbs from large RAW files. Is there any service or library that can help on such processing? I don't mean actual transcoding, but organizing, launching the tas...

Is there a name for this architectural pattern?

Suppose you have a web site consisting of: A web server serving your various users requests A DB for persistence A separate server asynchronously doing background stuff - preparing the data on the DB, updating it according to changes, etc. - regardless of what's going on in the main server. You can easily translate this into another ...

Scalable ruby application

I am java developer and heard a lot about ruby. My real question is can we develop large scale enterprise (web) application in ruby ? Because I heard that ruby is good for small application, but like java we can't develop large scale scalable (web) application in ruby. ...

Ok to use memcache in this way? or need a system re-architecture?

Hi all, I have a "score" i need to calculate for multiple items for multiple users. Each user has many many scores unique to them, and calculating can be time/processor intensive. (the slowness isn't on the database end). To deal with this, I'm making extensive use of memcached. Without memcache some pages would take 10 seconds to lo...

How many public high traffic websites are built with ASP.NET?

Hi all, This is really a 2 part question. First of all, I just wanted to know how common is asp.net in the real world? Secondly, I just want to know what are the read world scenario regarding scaling a asp.net site? http://highscalability.com/ almost never talked about the asp.net stack. Does anyone have any reason article that ta...

How Windows Azure Platform scales instances and balances workload?

The Windows Azure Platform allows an application to be deployed to one or more instances. The fabric controller then balances your application's workload across those instances. Can the number of instances be scaled up/down based on demand or are the number of instances static? If instances can be dynamically started how much control ...

Choice of primary key type

I have a table that potentially will have high number of inserts per second, and I'm trying to choose a type of primary key I want to use. For illustrative purposes let's say, it's users table. I am trying to chose between using GUID and BIGINT as primary key and ultimately as UserID across the app. If I use GUID, I save a trip to databa...

Is it crazy to not rely on a caching system like memcached nowadays ( for dynamic sites )?

I was just reviewing one of my client's applications which uses some old outdated php framework that doesn't rely on caching at all and is pretty much completely database dependent. I figure I'll just rewrite it from scratch because it's really outdated and in this rewrite I want to implement a caching system. It'd be nice if I could ge...

Are sharding and normalization mutually exclusive?

I ran across a comment that made me wonder: If you use a sharding approach to db scalability, does that mean the database is denormalized? Can you have a normalized, sharded database? ...

Is it fine to copy the mediawiki database architecture to design your own wiki?

In the highscalability blog, Todd Hoff talks about the wiki architecture SO adopted (initially), crunches that followed and mentions the painful refactoring needed to get back on track. To quote: Stack Overflow copied a key part of the Wikipedia database design. This turned out to be a mistake which will need massive and pai...

What is a scalable process of Project Management in Small Development Firms?

We have a small company team and even smaller Development team. The current process is that each Sales Rep (SR) is the actual Project Manager of each web application sold. Developers obtain requirements, functionality and design from the SR directly. Giving the actual head of developers to visibility over the actual work load on the d...

What are some ways to support multiple websites with a single code base?

I'm writing a pretty straight forward ASP.NET MVC web app: only a couple of CRUD pages, some folders where clients can browse documents and just 3 or 4 roles. The website will be used in a B2B scenario, where every client will have their "own" website. At this point, the only thing that will change in the website, from client to client ...

Just how scalable is Grails?

I'm looking to make a website that will probably get some heavy, repetitive traffic. Is grails up to the task? ...

Reading materials for scalability analysis?

I'm looking for educational material on the subject of scalability analysis. I'm not simply looking for Big-O analysis, but for material on approaches and techniques for analysis of the scalability of large scale transactional systems. Amazon's orders and payment systems might be good examples of the sort of systems I'm referring to. I ...

How Do I Create "Watch" System? Looking For Best Practices

I have a fairly complex community site (in php/mysql) with a number of "things" that the registered users can "watch". "Things" are: Posts, comments, event updates, user status changes, etc. At least 10 different types of "things" are available and a user can watch any other user's "things" (Think facebook) "Watch" means: A user can w...

Testing How Code Scales

I am currently working on some Ajax heavy code and I am wondering how my server will scale as more and more users (hopefully) start to use my web app. It is only on my internal test server for the moment and I was wondering how I would go about simulating a few hundred or thousand users so that I can see how it handles a heavier load. ...

Graph algorithm - Looking to improve scalability

I've written an algorithm which calculates and stores all paths of a DAG, and it works nicely on small graphs - but now i'm looking to improve it's efficiency to run over larger graphs. The core logic of the algorithm is in createSF() and makePathList() below, the other methods are helpers - I can see that append is a bottleneck. Howeve...

Determining Website Capacity

Hi SO, A client of mine has a website and they need to determine how 'scalable' the site currently is. What I mean by this is the number of users browsing around the site concurrently. It's a custom e-commerce app in .net, not written by myself and the code is... well lets just say, a bit dubious. A much bigger company is looking to b...