In Release It!, Michael Nygard reasons that many catastrophic system failures are often caused by a chain of things going wrong. For example, two threads deadlock. There's now two less threads in the thread pool, so load increases on the other threads increasing their likelihood of deadlocks. Suddenly, the server does not respond at all,...
I have used PHP for awhile now and have used it well with CodeIgniter, which is a great framework. I am starting on a new personal project and last time I was considering what to use (PHP vs ROR) I used PHP because of the scalability problems I heard ROR had, especially after reading what the Twitter devs had to say about it. Is scalabil...
I work for a large corporation that runs a lot of x86 based servers on which we run JVMs. We have experimented successfully with VMWare ESX to get better usage out of our data center. But these still consume a lot of power per processing unit. I had a mad idea that we should resurrect mainframes, we could host either lots of JVMs or v...
We have a scenario One Main e-commerce website - currently attracting a lot of visitors. Three sub "brand specific" sites which will hang off this site - each of these sites will potentiall have the same level of traffic over time. The client requires order processing for each brand site to happen in one place (i.e. one backoffice). ...
What are the recommended steps for creating scalable web/enterprise applications in Java and .NET? I'm more interested in what it takes to go from a low volume app to a high volume one (assuming no major faults in the original architecture). For example, what are the options in each platform for clustering, load-balancing, session manage...
My question is of the chicken and egg variety -- I want to get a job working on high-traffic, highly scalable websites, but I don't have much experience with high-volume sites. I've worked on dozens of web applications in my career spanning almost a decade. Most of these projects have been smaller, intranet systems or public websites th...
I am trying to evaluate Open Source options to replace my current CMS based publication application. My current CMS has about 12000 HTML pages and about 100000 uploaded files. The size of the data is about 20 Gigabytes. Drupal, Joomla and Plone seem interesting. However, I am concerned if these are ready to take on all this data. Do you ...
We are currently running a Java integration application on a Linux box. First an overview of the application. The Java application is a standalone application (not deployed on any JEE application server like OracleAS,WebLogic,JBOSS etc). By Stand Alone I mean its NOT a DESKTOP application. However it is run from the command line from a ...
I am in the process of building a Web Service API for my application. Also, I am planning to expose the Service via both REST and SOAP. I'm interested in getting some feedback from the community as to which programming language I should choose to implement the service? (I know C#, Java and Ruby - RoR well enough to create the service). ...
Our postgres server is about hitting its capacity and we're looking into adding a second database server. Are there any scaling solutions that are particularly good for a postgres setup? ...
Hi, I'm working on a .net portal which would be having lots of concurrent users. so scalability,performance need to be addressed in the design and architecture. We plan to use load balancing in the application. Keeping this in mind,what would be the best way of communicating between IIS web server(hosting aspx,aspx.cs files) and applic...
UPDATE 2009-05-21 I've been testing the #2 method of using a single network share. It is resulting in some issues with Windows Server 2003 under load: http://support.microsoft.com/kb/810886 end update I've received a proposal for an ASP.NET website that works as follows: Hardware load-balancer -> 4 IIS6 web servers -> SQL Server DB ...
Say you have a program that currently functions the way it is supposed to. The application has very poor code behind it, eats up a lot of memory, is unscalable and would take major rewriting to implement any changes in functionality. At what point does refactoring become less logical then a total rebuild? ...
Whenever someone talks about a services-based architecture, they often mention scalability, often in the same breath. However, it seems that using services adds more overhead, rather than reducing it, since there is now a protocol, like SOAP or REST, involved. So, does a web services based architecture really add performance benefits as ...
I've been working on the solution for financial industry. The main functionality of the application is the ability to load massive input files, digest them, update state in persistent store and generate extracts from persistent store on request. Pretty straightforward. The input files are industry standard formatted XML large (more that...
I have a web application that currently has about 500,000 users, averaging about 500 page views per hour, and we are expecting to have to scale up to 20,000,000 users within the next year. Our current system cannot handle that scale, and so we need to move to one that can. I was thinking a services-based architecture would be more ro...
EDIT: To people building tagging systems. Don't read this. It is not what you are looking for. I asked this when I wasn't aware that RDBMS all have their own optimization methods, just use a simple many to many scheme. I have a posting system that has millions of posts. Each post can have an infinite number of tags associated with it. ...
We will have a source system broadcasting messages over a JMS compliant pub/sub infrastructure. For the majority of messages published the subscribers will be GUI applications updating a screen with the data in the message. One of the subscribers, however, will be a translation service. For certain messages published the translation s...
There is a good chance that we will be tech crunched in the next few days. Unfortunately, we have not gone live yet so we don't have a good estimation of how our system handles a production audience. Our production setup consists of 2 EngineYard slices each with 3 mongrel instances, using Postgres as the database server. Obviously a hu...
Hi, There seems to be a big push for key/value based databases, which I believe memcache to be. Is the value usually some sort of collection or xml file that would hold more meaningfull data? If yes, is it generally faster to deserialize data then to do traditinally JOINS and selects on tables that return a row based result set? ...