Situation: I am currently designing a feed system for a social website whereby each user has a feed of their friends' activities. I have two possible methods how to generate the feeds and I would like to ask which is best in terms of ability to scale. Events from all users are collected in one central database table, event_log. Use...
I have written a proxy server for Linux using Poco but have since been reading up on the various approaches to achieving TCP/IP server scalability. I will need the server to handle persistent connections (not HTTP traffic) with an upper limit of about 250 simultaneous connections. Each connection typically uses about 5-10Kb/sec and the b...
I am creating a web-based RESTful service and want to cloud-enable it for scalability. I don't want to get locked into one cloud provider though. I'd like to be able to switched between Go Grid or Amazon EC2, etc. as pricing and needs evolve. Is there a common API to control the launch, monitoring and shutdown of cloud resources? I'v...
The attached simple Java code should load all available cpu core when starting it with the right parameters. So for instance, you start it with java VMTest 8 int 0 and it will start 8 threads that do nothing else than looping and adding 2 to an integer. Something that runs in registers and not even allocates new memory. The prob...
I am working on a software product with an integrated log file viewer. Problem is, its slow and unstable for really large files because it reads the whole file into memory when you view a log file. I'm wanting to write a new log file viewer that addresses this problem. What are the best practices for writing viewers for large text file...
We have a brand new webapp written that runs on Tomcat. So far, only one client is using it through the day. They run about 180 unique logins a day. Not really a lot IMO. Now, we managed to sell it to a brand new client who likes and wants to roll it out to 50,000 clients. How many of them will login at the same time - no idea. But I nee...
Last week I interviewed for a position at a TripleA MMORPG game company here in NE. I didn't get the job but one of the areas that came up during the interview was the about the scalability of the code that you write and how it should be considered early on in the design of your architecture and classes. Sadly to say I've never thought ...
I am referring to the algorithm that is used to give query suggestions when a user type a search term in google. I am mainly interested in how google algorithm is able to show: 1. Most important results (most likely queries rather than anything that matches) 2. Match substrings 3. Fuzzy matches I know you could use Trie or generalized...
From all the articles I've read so far about Mochiweb, I've heard this over and over again that Mochiweb provides very good scalability. My question is, how exactly does Mochiweb get its scalability property? Is it from Erlang's inherent scalability properties or does Mochiweb have any additional code that explicitly enables it to scale ...
Does anyone have any information on the architecture of twitter? A few specific items I'm especially interested in: I know that they use message queues. But what exactly do they use queues for? Do they "duplicate" tweets? If so, how? For example, say a user has 10,000 followers and he makes a tweet "hello world". Does twitter store "...
Hi everyone, I am having some second thoughts about where to implement the caching part. Where is the most appropriate place to implement it, you think? Inside every model, or in the controller? Approach 1 (psuedo-code): // mycontroller.php MyController extends Controller_class { function index () { $data = $this->model...
I have only very basic experience with HTML/CSS and have quite a bit of experience with testing software and web apps from a consumer perspective. I'd love to launch a web application that plays nicely with Google services, similar to some of the apps you'd find on the Google Apps Marketplace, such as ManyMoon, time to note, Socialwok, e...
Anyone know where I can find useful resource for some information for this. I reckon SOA is the way to go for scalabilty, but are there down sides such as performance and security I should look at. Overall what architecture should be considered best for enterprise web applications A good , complete, up to date book would be valuable, an...
Is PHP scalable enough to grow a project into something large? (Yes, I am aware of Facebook, maybe there are other examples?) ...
I am planning to build an application that will get a large amount of traffic. (Please don't say I won't get traffic, this is for an internal network, so the traffic will be there. Just trying to avoid the 'You won't get that much traffic, don't worry about it.) As for what type of traffic I'm expecting, users will browse various dynami...
There is a web application which is in production mode for 3 years or so by now. Historically, because of different reasons there was made a decision to use database-per customer installation. Now we came across the fact that now deployments are very slow. Should we ever consider moving all the databases back to single one to reduce en...
We're currently rewriting our organizations ASP.NET MVC application which has been written twice already. (Once MVC1, once MVC2). (Thank god it wasn't production ready and too mature back then). This time, anyhow, it's going to be the real deal because we'll be implementing more and more features as time passes and the testruns with MVC...
I'm really impressed with the power of cloud computing when it comes to the possibility to scale up and down your facilities depending on your load. How can I shift my paradigm and learn to write my applications in that way? Write it once and forget(no matter of the future load) would be the best solution. How can I practice my skills ...
I have a web app hosted in a cloud environment which can be expanded to multiple web-nodes to serve higher load. What I need to do is to catch this situation when we get more and more HTTP requests (assets are stored remotely). How can I do that? The problem I see from this point of view is that if we have more requests than mongrel cl...
When you hit a roof on reading from a database, you have two choices, scale vertically by putting more hardware in the server, or scale horizontally by putting a second server to help offload the reads. Offloading reads to a second server, means that all writes will hit both servers, while read only hits one. Problem is when you hit a...