scalability

Are there any stable and production quality nosql datastores ?

Are there are production quality nosql stores that I can use on a production system. I have looked at cassandra, tokyodb, couchdb etc but none of them seem to be ready for deployments on production like environments. I am talking thousands of requests per minute and lots of reads/writes/updates. My only concern is speed and service times...

Distributing tree structures

For example, I have a large set of bulletins which are organized in a tree (category tree). All categories inclusive. So if I watching items in category A, I'm also see all items in subcategories of this category A. And if I'm watching at root I'm see all the bulletins. I want to achieve horizontal scalability here, because of large siz...

How would a "NOSQL" database be designed for consumer apps (e.g. social bookmarking)

I've been reading up on a lot of posts about non-relational databases, the whole NOSQL movement, and there's a lot of fresh new activity around it. It seems like a very interesting approach to building highly scalable web applications but unfortunately (but also a good thing at this nascent stage) there isn't quite a clear leader/standar...

Which PHP framework to choose for a big project? Help me to choose.

Possible Duplicate: What PHP framework would you choose for a new application and why? Hello! I'm going to create a serious web project. Until this time, I've been developing projects with Drupal, but I want to switch to a PHP framework. Which framework can you suggest and why? Scalability and performance are must. I was playin...

what does O(N) mean

Possible Duplicate: What is Big O notation? Do you use it? Hi all, fairly basic scalability notation question. I recently recieved a comment on a post that my python ordered-list implimentation "but beware that your 'ordered set' implementation is O(N) for insertions" Which is great to know, but I'm not sure what this means. ...

are there books on designing highly scalable web sites?

are there books on designing highly scalable web sites? (from a programmers perspective) I read how ebay does it: Partition by Function, Split Horizontally, Avoid Distributed Transactions, Decouple Functions Asynchronously, Move Processing To Asynchronous Flows, Virtualize At All Levels, Cache Appropriately. Are these things actually...

What is Scaling?

I always get this argument against RoR that it dont scale but I never get any appropriate answer wtf it really means? So here is novice asking, what the hell is this " scaling " and how you measure it? ...

How much should I worry about scaling?

Hi, I am planning to start a new web application from scratch, but I wonder if I am not obsessing too much over scalability. Should I worry about this since the beginning or is it better to focus on the application itself first and scale later if I happen to be really successful? ...

A upgradable approach to design a web application system

Many poeple have online startups in their head that may potentially attracts millions, but most of the time you will only have minimal budget (time and resource) to start with so you want to have it delivered within a year's time. Short after launch, you are bound to perform one or a series of upgrades that may include: code refactor to ...

Will unique RIDs ever "overflow" ?

Hello, I am making an application to run on both SQL Server and PostgreSQL, so I am asking this question about both of them. What happens whenever you create a unique primary key(using a sequence or auto-increment type thing) and you hit over 4 billion records(32 bits)? I'm not saying that our table will have 4 billion records but rath...

DotNetNuke Scalability

Hi, We are about to start in a content-rich web site and we are considering Dotnetnuke. The site will be multilingual with size compared to that of the UN site (http://www.un.org/en/index.shtml ) The question is: Would DotNetNuke support such large content size? is there a limit on the size of content or the depth of the hierarchy? ...

Efficient Matching Algorithm for Set Based Triplets

I am looking for an efficient way to solve the following problem. List 1 is a list of records that are identified by a primitive triplet: X | Y | Z List 2 is a list of records that are identified by three sets. One Xs, one Ys, one Zs. The X, Y, Zs are of the same 'type' as those in list one so are directly comparable with one another...

Is make -j distcc possible to scale over 5 times?

Since distcc cannot keep states and just possible to send jobs and headers and let those servers to use only the data just sent and preprocess and compile, I think the lastest distcc has problem in scalability. In my local build environment which has appx. 10,000 c/c++ files to build, I could only make 2 times faster than not using distc...

Scalability for large applications

Hi, There are very useful documents explaining the server architectures like Linkedin, MySpace, Amazon and etc. After seeing MySpace, I really surprised as they are using 500+ database servers for their application. Would like to know how will they maintain SQL transactions, joins, look ups if data spans across multiple database serve...

WebLogic dynamic Amazon EC2 cluster aggregation

Hello, buddies! I'm currently studying WebLogic's integration capabilities, to develop a system that ensures high performance, availability and scalability; fail-over and crash recovery and 24/7 availability on a WebLogic integration scenario. I've been exploring some concepts and evaluating potential solutions to the problem I want to...

Webservice that handles 1 million user accounts

Imagine you're writing a web app that will have 1 million users (they all grow that big, right!) How would you handle user accounts? I can imagine a few scenarios: Roll your own (database tables, salted/hashed passwords stored in a user profile table) If written with ASP.NET, use the login/role provider (which falls back to the datab...

Caveats of select/poll vs. epoll reactors in Twisted

Everything I've read and experienced ( Tornado based apps ) leads me to believe that ePoll is a natural replacement for Select and Poll based networking, especially with Twisted. Which makes me paranoid, its pretty rare for a better technique or methodology not to come with a price. Reading a couple dozen comparisons between epoll and ...

severside processing vs client side processing + ajax?

looking for some general advice and/or thoughts... i'm creating what i think to be more of a web application then web page, because i intend it to be like a gmail app where you would leave the page open all day long while getting updates "pushed" to the page (for the interested i'm using the comet programming technique). i've never cr...

Implementing a job list with internal synchronisation

I'm working on a simple job threading framework which is very similar to the one described in id Tech 5 Challenges. On the most basic level, I have a set of lists of jobs, and I want to schedule these list across a bunch of CPU threads (using a standard thread pool for the actual dispatching.) However, I wonder how this signal/wait stuff...

Why doesn't my threaded .Net app scale linearly when allocating large amounts of memory?

I’ve run into something strange about the effect of large memory allocations on the scalability of the .Net runtime. In my test application I create lots of strings in a tight loop for a fixed number of cycles and spit out a rate of loop iterations per second. The weirdness comes in when I run this loop in several threads – it appears th...