scalability

How does FlockDB compare with neo4j?

Both FlockDB and neo4j are open source frameworks for keeping large graph dataset. Anyone familiar enough with both products to write a comparison? ...

How to scale php application servers horizontally?

Hello, I'm using lighttpd as webserver for php application server. The avg. load on this server is about 2-3. MySQL database is separated to another server (it's load ~0.4). How could I scale php application server? Thank you. ...

Fast data retrieval in MySQL

I have a table of users - It contains around millions of rows (user-id is the primary key). I just want to retrieve user-id and their joining date. Using SELECT user-id, joining-date FROM users requires lot of time. Is there a fast way to query/retrieve the same data from this table? ...

Architecture for database analytics

Hi, We have an architecture where we provide each customer Business Intelligence-like services for their website (internet merchant). Now, I need to analyze those data internally (for algorithmic improvement, performance tracking, etc...) and those are potentially quite heavy: we have up to millions of rows / customer / day, and I may w...

ASP.Net webapplication, keep in mind things for performance ,scalability

what are the things a developer should keep in mind or should take care when depeloping a web application. web application e.g. social networking web app. which are the things which must be implemented to get good performance and scalability. please provide any article or blogs or suggestions for this kind of question if you have -th...

Improve SQL strategy - denormalize in object-children-images case

Hi, I have a Tour object which has many Place objects. For the list of tours, I want to show some info of the tour, the number of places of the tour, and three Place's images. Right now my SQL queries are (i'm using Doctrine with Symfony on MySQL) get Tour get Tour 1 places get Tour 2 places get Tour 3 places ... get Tour n places If...

Database for Python Twisted

There's an API for Twisted apps to talk to a database in a scalable way: twisted.enterprise.dbapi The confusing thing is, which database to pick? The database will have a Twisted app that is mostly making inserts and updates and relatively few selects, and then other strictly-read-only clients that are accessing the database directly m...

Will PHP Die In Web Page Development World?

I know that PHP is still the most popular web programming language in the world. This question just want to bring some of my concerns about PHP. PHP is naturally bound to C10K problem. Since PHP (generally run in Apache) cannot be event-driven or asynchronous, each HTTP request will occupy at least one thread or process. This makes it r...

EJB3.1 Remote invocation - is it distributed automatically? is it expensive?

I'm building a JEE6 application with performance and scalability in the forefront of my mind. Business logic and JPA2-facade is held in stateless session beans (EJB3.1). As of right now, the SLSBs implement only @Remote-interfaces. When a bean needs to access another bean, it does so via RMI. My reasoning behind this is the assumption...

*Client* scalability for large numbers of remote web service calls

Hey Guys, I was wondering if you could share best practices and common mistakes when it comes to making large numbers of time-sensitive web service calls. In my case, I have a SOAP and an XML-RPC based web service to which I'm constantly making calls. I predict that this will soon become an issue as the number of calls per second will...

Is it wise to use temporary tables?

Hi guys, We have a mySQL database table for products. We are utilizing a cache layer to reduce database load, but we think that it's a good idea to minimize the actual data needed to be stored in the cache layer to speed up the application further. All the products in the database, that is visible to visitors have a price attached to t...

Scalability of Ruby on Rails versus PHP

Can anyone comment on which is more scalable between RoR and PHP? I have heard that RoR is less scalable than PHP since RoR has a little more overhead with its MVC framework while PHP is more low level and lighter. This is a bit vague - can anyone explain better? ...

JavaEE Application Server or Lightweight Container?

Let me preface this by saying this is not an actual situation of mine but I'm asking this question more for my own knowledge and to get other people's inputs here. I've used both Spring and EJB3/JBoss, and for the smaller types of applications I've built, Spring (+Tomcat when needed) has been much simpler to use. However, when scaling u...

How do you deal with denormalization / secondary indexes in database sharding?

Say I have a "message" table with 2 secondary indexes: "recipient_id" "sender_id" I want to shard the "message" table by "recipient_id". That way to retrieve all messages sent to a certain recipient I only need to query one shard. But at the same time, I want to be able to make a query that ask for all messages sent by a certain sen...

Partitioning requests in code among several servers

I have several forum servers (what they are is irrelevant) which stores posts from users and I want to be able to partition requests among these servers. I'm currently leaning towards partitioning them by geographic location. To improve the locality of data, users will be separated into regions e.g. North America, South America and so on...

Deploying and hosting scala in the cloud?

I am starting a web app considering scalability as one of the top priorities. What would be the benefits of this: cassandra scala lift vs the traditional LAMP on the cloud? Since from what I've read, please correct me, the cloud itself is scalable I have never seen anyone deploy scala on the cloud before. Is it worth the effort to lear...

database design programatically handling incrementing ID field, scalability - confused

Hey, If I am building a multi-shop e-commerce solution and want the orders table to maintain a shop based sequential ID, what is the best way of doing this? For instance imagine these order IDs in sequence: - UK0001 UK0002 UK0003 DE0001 UK0004 DE0002 etc. through grouped PK ID MySQL / MyISAM - MySQL will manage this automatical...

Commercial web application--scalable database design

I'm designing a set of web apps to track scientific laboratory data. Each laboratory has several members, each of whom will access both their own data and that of their laboratory as a whole. Many typical queries will thus be expected to return records of multiple members (e.g. my mouse, joe's mouse and sally's mouse). I think I have th...

Web services or shared database for (game) server communication?

We have 2 server clusters: the first is made up of typical web applications backed by SQL databases. The second are highly optimized multiplayer game servers which keep all data in memory. Both clusters communicate with clients via HTTP (Ajax with JSON). There are a few cases in which we need to share data between the two server type...

best practice in architecture in centralizing location of different modules (cms, webapplications, ...) -

Let's just say that i want to create a CMS and other online applications. I want to integrate them all in a central location, but they also have to be available seperately (not everyone wants more than the CMS solution). Should I create a huge central application that contains all the database, which communicates through a webserice wi...