Optimizing Magento-using Websites

Question

Please be patient with my question, as this may be a bit longer.

If you look at Magento and try to compare with any other non-PHP eCommerce Shopping Cart websites, you will find that the latter are comparatively faster. I know that the following factors work as well:-

1. Hosting Server Bandwidth
2. Magento offers lots & lots of features, which others does not (& will not be able to) provide with as much flexible as it offers now.
3. Clients / Administrators can override the core behavior of the Magento functionality, using the custom module functionality, which will be a bit tough for other non-PHP websites.

Still for non=PHP websites, programmers can use the concept of stored procedures very easily, with the help of which they can load the whole database in a view & then get disconnected from the database. So whenever any server request for next / previous page occurs, the server just accesses the view to fetch the proper results, instead of going for fetching from the database directly.
Also from MySQL 5.0, it has introduced the Stored Procedure concept, but I don't know whether this is at all used in Magento. Even if it is used in Magento, whether will it be of any help for Magento, is another BIG question, because of the whole lot of features Magento provide.

There are many clients, who like their websites to load very fast as compared to Magento sites. Can Magento offer any help in this regard?

Please anybody having any idea about what & how to do will immensely help every Magento user, not just conceptually but also programmatically if possible?

If anybody else have any other solutions or any other ideas, then please share those, like William.

Answer 1

Not that long ago I launched a Magento store with over 400 thousand products, 7,000+ categories, 2,000+ attribute sets, and over 4 million attributes for those products.

We needed load times to be ~500ms, and it was to be run on one Amazon EC2 Extra Large memory instance (around $360 a month).

Our solution? Implement Solr. The search is now powered by Solr, when browsing the catalog we're using facets so users can narrow their results by our 4 million+ attributes. The issue with this was Magento doesn't allow you to narrow your results down if you're using "varchar" attribute values. So I modified the catalog page to use Solr's facets along with modifying the catalog page to not do any queries to the DB at all, except one. This query would be given a list of ID's that Solr returned and do a direct query on these. This plugin took a few days to write, mainly because I've never used Magento before.

The last change we did was modify the product view page so that attributes were a lot quicker. When you have millions of attributes the way Magento was doing the query it was really slow, a simple modification and it was fixed.

The great thing was that all of this was done via custom plugins, it didn't take long at all, and the store works great.

Edit: Also, remember to install APC (or alternative) and configure Magento use that in it's backend. It dramatically increases the speed, as not only will it speed up PHP, Magento was build to work with certain backends to speed it up. You can also do things like having Magento's cache stored in memory (tmpfs in linux). Storing static content onto a CDN or just a static content server will dramatically help as the web server processing Magento won't have to handle those requests at all. You know, the basic stuff for running an application.

You can also tell Magento to save sessions into Memcache so that your sessions are in memory, and distributed. Once you modify Magento to get rid of all those "nasty" queries, the rest is really the same as with any website. There are TONS of tools that help you "scale" out your system. Just remember we had a LOT of products, and were preparing to go to around 1,000,000 products. So if you're doing like 40,000 products, you probably won't have to go to that much trouble.