scalability

Inspiration needed: Selecting large amounts of data for a highscore.

I need some inspiration for a solution... We are running an online game with around 80.000 active users - we are hoping to expand this and are therefore setting a target of achieving up to 1-500.000 users. The game includes a highscore for all the users, which is based on a large set of data. This data needs to be processed in code to ...

YSlow Best Practices with Django apps, How to implement them?

Hi, I have an django 1.1.1 app, actually in developement, thinking in best practices I ran the YSlow test (Grade E Ruleset applied: YSlow V2 ) it recomends: Grade F on Add Expires headers -There are 37 static components without a far-future expiration date. Grade F on Use a Content Delivery Network (CDN) ...

How many open udp or tcp/ip connections can a linux machine have?

There are limits imposed by available memory, bandwidth, CPU, and of course, the network connectivity. But those can often be scaled vertically. Are there any other limiting factors on linux? Can they be overcome without kernel modifications? I suspect that, if nothing else, the limiting factor would become the gigabit ethernet. But for ...

.NET Client/Server Scaleability & Asynchronous I/O - Excess Thread Issues

I have a pretty simple question which perhaps someone familiar with Server/Client design & the Asynchronous I/O paradigm of .NET could answer quickly... I'm writing a server-side application which is designed to run on relatively non-sophisticated hardware (read: not-so-modern, average office desktop PCs), but accommodate a reasonably l...

AS3: Faster to use EventListeners or ArrayLoops?

I am writing a game in AS3 and have, as an example, around 40 objects on screen. Let's say they are clouds. I'm wondering which of the two paths would be less a strain on system resources: a) Put an eventListener on each object and have it update itself, or b) Loop through the array and manually update each object Or is the performanc...

Create a database for scalability

How do i create a database for scalability? I am in the middle of http://www.slideshare.net/vishnu/livejournals-backend-a-history-of-scaling which i cant read ATM and need to leave. But i would like to know more about creating a database that scales well. Somethings that it mentioned and occur in my mind are Separate handles for reads ...

High-traffic, Highly-secure web API, what language?

If you were planning on building a high-traffic, very secure site what language would you use? For example, if you were planning on say building an authorize.net-scale site, that had to handle tons of credit card transactions via an API, what would you use building it from scratch? I would imagine that most sites handling similar tra...

How to store PHP sessions in APC Cache ?

Hi Storing sessions in disk very slow and painful for me. Im having very high traffic. I want to store session in Advanced PHP Cache, How can i make this ? Thanks ...

Is this way of using Excel 2007 Pivot table for BI scalable ?

Hi all, Background: We need to consolidate sales data across the country to do analysis Our Internet connection/IT expertise/IT investment is not quite strong, therefore full BI solution is out of question I tried several SaaS BI solution (GoodData, ZohoReports) and while they're good, they seem not to fully support what we need We'r...

Achievements / Badges system

I have been browsing this site for the answer but I'm still a little unsure how to plan a similar system in its datbase structure and implementation. In PHP and MySQL it would be clear that some achievements are earned immediatley (when a specialzed action is taken, in SO case: Filled out all profile fields), although I know SO updates ...

(*nix) Cloud/Cluster solutions for bulding fast & scalable web-services

I'm going to build a high-performance web service. It should use a database (or any other storage system), some processing language (either scripting or not), and a web-server daemon. The system should be distributed to a large amount of servers so the service runs fast and reliable. It should replicate data to achieve reliability and a...

What should be the maximum number of opened socket on a server?

For a web server, the socket connection are kept alive to save overhead. At which point the server should start to drop connection that doesn't have pending request and based on what premises? ...

Persistence strategy for low latency reads and writes

I am building an application that includes a feature to bulk tag millions of records, more or less interactively. The user interaction is very similar to Gmail where users can tag individual emails, or bulk tag large amounts of emails. I also need quick read access to these tag memberships as well, and where the read pattern is more or...

How to create an ASP.NET web farm?

Hi everybody, I am looking for information on how to create an ASP.NET web farm - that is, how to make an ASP.NET application (initially designed to work on a single web server) work on 2, 3, 10, etc. servers? We created a web application which works fine when, say, there are 500 users at the same time. But now we need to make it work ...

Network Load Balancing (NLB): is it suitable for "stateful" ASP.NET applications?

Hi everybody, I have posted the following question concerning ASP.NET web farms. http://stackoverflow.com/questions/1816756/how-to-create-an-asp-net-web-farm/ Guys recommended using Network Load Balancing (NLB) as a primary way of creating a web farm. However, Wikipedia says that "NLBS is intended for ... stateless applications". Our...

How is Twitter's MySQL database architecture designed? Try your best.

No one knows. And even if you work at Twitter, you probably won't let us know the secret. So, let us all post our theories on how this communication tool database is designed! It definitely can't follow the standard replication architecture, right? It's got to do some sharding, partitioning, etc, etc. Let me know what you think! ...

Relation between language and scalability

I came across the following statement in Trapexit, an Erlang community website: Erlang is a programming language used to build massively scalable soft real-time systems with requirements on high availability. Also I recall reading somewhere that Twitter switched from Ruby to Scala to address scalability problem. Hence, I won...

One ID for every database column, how to do?

I working on a food database, every food has a list of properties (fats, energy, vitamins, etc.) These props are composed by 50 different columns of proteins, fat, carbohydrates, vitamins, elements, etc.. (they are a lot) the number of columns could increase in the future, but not too much, 80 for extreme case Each column needs an indiv...

Scalability of Oracle Forms

What is your experience regarding the scalability of Oracle Forms? What's the maximum number of application users you would use Oracle Forms for: 100, 1000, 10000, 50000? I know that this question lacks many detail information for a well-founded answer. However, I am interested in the gut feeling of seasoned Forms developers. Thanks. ...

Scaling Rails with Cache-on-write

I currently have a rails app that uses the traditional caching. cache do blocks are used to cache slow-rendering partials. This works great for the most part, except for a few pages which take too long to render on the first read. I'd like to move the rendering of these partials to the write-side from the read-side, by rendering in ei...