Packing, caching, JS and CSS in PHP that differentiate between development and production environment

Question

I am trying to make development easy and have highly optimized output in production.

The goals of what I am trying to do is:

• Make production pages fast! I would like that the Google Page Speed and YSlow return the best scores. This means:
  1. Combine and compress JS files and CSS and position the group in the right place (bottom or top of the page) in the HTML. For .js Google Closure seems the best choice.
  2. .JS and .CSS are smartly cached but be sure that they get reloaded when of the .JS or CSS componet is updated. 301 File not changed etc.
  3. Cache type: I think cache on disk is fine. Consider APC and Memcache or Redis if they significantly improve speed.
  4. Capable to specify and use lazy load of .JS when necessary or at least not to block the page rendering.
  5. (Optional) Compress the HTML too.
• Make website development easy:
  
  1. Use short command in the .php file when you want to include .js or .css and compress them only in the production environment
     • Use syntax like pack_js(['first.js', 'second.js' 'third.js']) and pack_css(['first.less', 'second.less' 'third.css'], true)
     • It is easy to configure develpment or production environment. Maybe just a call to SetDebug(true or false). Default production
     • Easy to set up cache folders and source folders
  2. Use of LESS to make CSS development sucks less. Automatically compile LESS files in CSS in production but use of LESS.js in development so that each time you change the .less file in development it is updated on the server.
  3. (optional) In development it includes a JS and a LESS console similar to the shell at https://www.squarefree.com/bookmarklets/webdevel.html

Note: it is OK to use Apachee modules and .htaccess files if they significantly speed up the process. But it should be able to set them up quicky, ideally with just a setup command.

Is there something that do this? Or what are the best resources to start?

I would prefer a solution that consists of a PHP script (eventually few .php files, .htaccess and compressing executables) that compresses the .JS with the Google Closure Compiler and compress/compile the CSS/LESS files stripping out useless comments and white spaces. A special cookie could be used on production server to output the development version.

I would like to have:

A php function usable like this: pack_js(['first.js', 'second.js', 'third.js']) that write something like:

On developments servers:

<script type="text/javascript" src="static/js/first.js"></script>
<script type="text/javascript" src="static/js/second.js"></script>
<script type="text/javascript" src="static/js/third.js"></script>

On production servers (if the special cookie is not present):

<script type="text/javascript" src="cache/12a42323bfe339ea9w.js"></script>

Another function: pack_css(['first.less', 'second.less', 'third.css'], true) that write something like:

On developments servers:

<link rel="stylesheet/less" href="/static/css/first.less" type="text/css" />
<link rel="stylesheet/less" href="/static/css/second.less" type="text/css" />
<link href="/static/css/third.css" type="text/css" />
<script src="http://lesscss.googlecode.com/files/less-1.0.21.min.js"&gt;&lt;/script&gt;

<script type="text/javascript" charset="utf-8">
    less.env = "development";
    less.watch();
</script>

On production servers (if the special cookie is not present):

<link href="/cache/46537a8b8e876f7a8e7.css" type="text/css" />

the second parameter specify if the less.js should be output on the development server

Obviously 12a42323bfe339ea9w.js and 46537a8b8e876f7a8e7.css are the optimized, packed and compiled version of the script. This solution should be able to detect when a source file change and recompile the scripts for production. It should be configurable to regarding locations of the script and type of caching (disk is fine). Ideally the pack_js should probably have an option to make possible lazy load of the js in production.

Every suggestion is welcomed.

Answer 1

Why don't you use a project build system to deploy your application to the production server that does just that? For PHP you might like Phing, since it would allow you to write extra plugins in PHP, that you can execute while deploying. Other options you might want to consider of you take this route are Ant and Capistrano (and there's lots of others), but these require knowledge of other languages (granted, you could start the php parser from them if you really wanted to...).

Answer 2

Great Question!

I've worked with a process with similar requirements, but we took somewhat of a different approach. I realise its maybe not exactly what you're after, but would still make some recommendations:

1. Make the compression/compilation part of the shipping process.(You may already be doing this, but it wasn't clear from the above).
2. Compress/compile it on the development servers too. It might be a hassle for debugging/testing but you want to be able ensure the production version and test versions are as similar as possible. If you have the luxury of multiple development stages, then you could possibly compress on one of them.
3. Only do the compression/compilation if it passes some sort of quality scan (e.g. jslint)
4. Don't combine the modules - keep them separate. The performance benefits you will gain will be so negligible, as to be almost pointless.
5. Don't change the HTML, just change the dependant modules content.

I realise its a bit different from what you were intending, but in my experience it leads to a more robust development promotion model.

Answer 3

Still working on exploring the best solution leveraging the resource already available.

CSS-JS-Booster and Turbine so far looks like the best starting point: http://github.com/Schepp/CSS-JS-Booster and http://turbine.peterkroener.de/

Other JS/CSS Combiners solutions and articles

• http://code.google.com/p/minify is good but seem a little to hard to configure.
• http://rakaz.nl/code/combine Let you use sintax like this: http://www.creatype.nl/javascript/prototype.js,builder.js,effects.js,dragdrop.js,slider.js (not exactly the best) I also think I saw a patch to this script but I cannot find it now. (example of usage http://brainfart.com.ua/post/code-cache-combine/)
• http://www.ejeliot.com/blog/72 and http://www.ejeliot.com/samples/combine/combine.phps Ed Eliot combine.php solution. It only let you have specify once the file to merge
• http://www.sitepoint.com/blogs/2007/04/10/faster-page-loads-bundle-your-css-and-javascript/ Suggest the use of lighttpd/mod_magnet to avoid the overhead of Apache/PHP. When a combination of files is bundled for the first time, the bundle is cached on the server so that it only needs to be regenerated if one of the source files is modified. If the browser supports compression, lighttpd/mod_compress handles the gzipping and caching of the compressed version.
• http://aciddrop.com/php-speedy/ other solution.
• http://www.artzstudio.com/2008/08/using-modconcat-to-speed-up-render-start/ Use of mod_concat and overview of other solutions
• http://wordpress.org/extend/plugins/w3-total-cache/ That is a WordPress Plugin. It is very interesting but probably to much work to make it independendt from Wordpress.
• http://bakery.cakephp.org/articles/view/combine-your-js-css-files-for-faster-loading CakePHP solution
• http://github.com/FluffyJack/CominifyCI Code Igniter that combine, minify, and timestamp CSS and JavaScript
• http://github.com/dimkalinux/fancy_js_cache CSS and JS merger for PunBB

Resource for JS compression and caching:

• http://code.google.com/p/php-closure/ PHP script that let you combine .js files ancd combile thought the Google Closure REST API. Check timestamps and cache the combined version locally.
• http://dev.juokaz.com/php/easy-javascript-packing-with-php and http://dean.edwards.name/download JS packer compressor/obfuscator. I am not sure how long the decompression take. But it was able to shrink the jQuery 1.4.2 compiled/minified with Closure to 50639 bytes from 71946 (almost 30% reduction!) with Base62 Encode on! It would be interesting to compare the gzipped versions. Regarding JS obfuscation Packer optimizer make a little more difficult to tamper in your JS code.
• http://thinkvitamin.com/dev/serving-javascript-fast/ Great discussion about gzip and caching
• http://cjohansen.no/en/ruby/juicer_a_css_and_javascript_packaging_tool Ruby Juicer JS/CSS packaging tool

LESS compilers/tutorials/tools:

• http://designshack.co.uk/articles/css/using-less-js-to-simplify-your-css3 Less.js tutorial/introduction.
• http://net.tutsplus.com/tutorials/html-css-techniques/quick-tip-you-need-to-check-out-less-js Presentation of less.js (useful comments)
• http://fadeyev.net/2010/06/19/lessjs-will-obsolete-css/ and http://github.com/cloudhead/less.js It is a LESS compiler written in JS and can also be added in the browser. Great for development
• http://lesscss.org/ What LESS is. Originally written in Ruby but there are implementation in a lot of different fanguages,
• http://leafo.net/lessphp/ Less compiler written in PHP (it does not seem updated)
• http://binarybonsai.com/code/lessgrid/ Even more easier CSS/Less developlment using grid. 960 is another way to make it things easier; http://www.webdesignerdepot.com/2010/03/fight-div-itis-and-class-itis-with-the-960-grid-system/
• http://github.com/everzet/sfLESSPlugin LESS in symphony
• http://drupal.org/project/less Drupal CSS Preprocessor
• http://github.com/sunny/csscaffold A PHP parser of something similar to LESS but not quite
• http://code.google.com/p/maven-less-plugin/ maven-less-plugin uses rhino, envjs and less.js to translate the less-code to css during the process-source phase of your maven build. In the end you get a artifact with ready-to-use css code.

At Deployment time options:

• http://phing.info/docs/guide/stable/ Phing is build/deployment system in PHP
• http://www.capify.org/index.php/Capistrano The Ruby solution