Consider a web app in which a call to the app consists of php script running several MySQL queries, some of them memcached. The PHP does not do very complex job. it is mainly serving the mysql data with some formatting.
In the past it used to be recommended to put mysql and the app engine (PHP/Apache) on seperate boxes.
However, when the data can be divided horizontally (for example when there are 10 different customers using the service and it is possible to divide the data per customer) and when nginx +FastCGI is used instead of heavier apache, doesnt it make sense to put Nginx Memcache and MySQL on the same box? then when more customers come, add similar boxes?
Background: We are moving to Amazon Ec2. And a seperate box for mysql and app server means double EBS volumes (needed on app servers to keep the code persistant as it changes often). Also if something happens to the database box, more customers will fail.
Clarification: Currently the app is running with LAMP on a single server (before moving to EC2).