Before you answer this I have never developed anything popular enough to attain high server loads. Treat me as (sigh) an alien that has just landed on the planet, albeit one that knows PHP and a few optimisation techniques. I'm developing a tool in PHP that could attain quite a lot of users, if it works out right. However while I'm fu...
I'm developing a web service whose methods will be called from a "dynamic banner" that will show a sort of queue of messages read from a sql server table. The banner will have a heavy pressure in the home pages of high traffic sites; every time the banner will be loaded, it will call my web service, in order to obtain the new queue of m...
For a website like reddit with lots of up/down votes and lots of comments per topic what should I go with? Lighttpd/Php or Lighttpd/CherryPy/Genshi/SQLAlchemy? and for database what would scale better / be fastest MySQL ( 4.1 or 5 ? ) or PostgreSQL? ...
I have to build a web application that's similar to a "branch/store locater". A user types in his address and the web application will plot nearby stores on a map. One of the requirements is: "The web application must support 100 simultaneous users and up to 5GB/day transfer volume." Much of the transferred data will be text and GUI ...
We need to build a system capable of processing 40,000 messages per second. No messages can be lost in case of any software or hardware failures. Each message size is about 2-4Kb. Processing of a message consists of validating the message, doing some simple arithmetical calculations, saving result to database and (sometimes) sending no...
Hi guys! I'm using WPF and C#. I have a problem. I need to create a lot of bindable templated controls in scrollable area (they are all of different types). For example 1000 textboxes, 1000 drop down lists, and 1000 checkboxes. The problem is that when they are all created it works really slow. The question is - whether it is possi...
I remember finding a C++ (or maybe plain C) library which does thread pooling, socket polling and other stuff, needed to write a high-performance server on Linux. (And this library was not Boost itself but it could be that it used Boost.) As usual, I can't remember the name of the library. Any guesses? Update: that was not ACE. ...
Hi, I have a table in MySQL 5 (InnoDB) that is used as a daemon Processing Queue, thus it is being accessed very often. It is typical to have around 250 000 records inserted per day. When I select records to be processed, they are read using a FOR UPDATE query to eliminate race conditions (everything is Transaction Based). Now I am dev...
Currently I'm developing high-loaded financial portal(we use LAMP to run our project). There are great number of incoming data to be processed and stored. So optimization tasks become very important for us. Could you suggest books, articles or resources, that discover optimization questions (especially bboks). NOTE: At the moment I'm ...
I am creating a very data intensive, high volume web site. Every aspect of the website is driven by interactions with the MSSQL DB that I am using. On one page there are 10-12 different resultsets that I need to utilize in my page. So I need to know the best practice when it comes to using Linq-to-SQL and multiple results sets with a ...
Hi. How does digg or any other high-traffic website store user sessions? What do they use for storing the user sessions? File system, DB (which one?), memcache or both? Let's imagine a simple situation. Logged user has set the flag "Remember me" during login. We've set a session cookie with expiration date 1 year. For example, we are ke...
Hello, my company has recently started to get problems with the image handling for our websites. We have several websites (adult entertainment) that display images like dvd covers, snapshots and similar. We have about 100'000 movies and for each movie we have an average of 30 snapshots + covers. Almost every image has an additional vers...
Hi All, I am relatively new to working on huge applications in Java. I am working on a Java web service which is pretty heavily used by various clients. The service basically queries the database (hibernate) and then works with a lot of Lists (there are adapters to convert list returned from DB to the interface which the service publish...
I'm writing a high-loaded daemon that should be run on the FreeBSD 8.0 and on Linux as well. The main purpose of daemon is to pass files that are requested by their identifier. Identifier is converted into local filename/file size via request to db. And then I use sequential mmap() calls to pass file blocks with send(). However sometime...
Hello, Just wonder how much cost to raise a java exception (or to call native fillInStackTrace() of Throwable) compared to what it cost to log it with log4j (in a file, with production hard drive)... Asking myself, when exceptions are raised, does it worth to often log them even if they are not necessary significant... (i work in a hig...
I'm designing an application which will have a network interface for feeding out large numbers of very small metadata requests. The application code itself is very fast, basically looking up data cached in memory and sending it to the client. What's the absolute lowest latency I can get for a network application server running on a lin...
we are currently developing a high traffic facebook application. all the traffic will be within one month, where there are 500.000 to 1.000.000 expected users. after that month, the game is over and we have a winner - so the app will be archived. we are currently planning to develop the application with ruby on rails and searching for h...
Hello, Let's imagine that we have high traffic project (a tube site) which should provide sorting using this options (NOT IN REAL TIME). Number of videos is about 200K and all information about videos is stored in MySQL. Number of daily video views is about 1.5KK. As instruments we have Hard Disk Drive (text files), MySQL, Redis. Views...
Scenario is simple to describe, but might have a complex answer: Imagine a case where you have one write only mysql database. Then you have about 5 or 6 read only databases. The write database has a count for a particular inventory. You have hundreds of thousands of users banging away at this particular inventory item, but only limited ...
When you have peaks of 600 requests/second, then the memcache flushing an item due to the TTL expiring has some pretty negative effects. At almost the same time, 200 threads/processes find the cache empty and fire of a DB request to fill it up again What is the best practice to deal with these situations? p.s. what is the term for this...