I personally recommend using AppCmd.exe (matthewthurlow's first bullet) because you don't have to count on the legacy management components being installed, or risk modifying the configuration XML manually.
If you are not comfortable with AppCmd, Mike Volodarsky has a great article on Getting Started with AppCmd.exe, and the Microsoft IIS Configuration Reference is excellent, offering UI, Code and AppCmd examples for modifying each of the configuration items (e.g. Application Pools ). The IIS7 Administration Pack also includes a Configuration Editor that allows you to generate AppCmd scripts from any existing configuration.
To integrate AppCmd into WiX, you need to create and schedule two custom actions for each command. There is general information in the WiX v3 manual documenting this procedure, and I've included a concrete example below.
First, you need to set up an immediate action to store the command line in a property:
<CustomAction
Id="CreateAppPool_Cmd"
Property="CreateAppPool"
Execute="immediate"
Value=""[WindowsFolder]system32\inetsrv\APPCMD.EXE" add apppool /name:"[APP_POOL_NAME]"" />
Next you set up a deferred action which references this property:
<CustomAction
Id="CreateAppPool"
BinaryKey="WixCA"
DllEntry="CAQuietExec"
Execute="deferred"
Return="ignore"
Impersonate="no"/>
And finally, you need to schedule these. The immediate action that sets the properties seem to work well after InstallFinalize, and the deferred action works after InstallFiles. I haven't got as far as figuring out rollback actions yet.
MapGuide Open Source does this method extensively; you can see the CA scheduling in our MapGuide.wxs file, and the CA definition in our IIS7.wxs file.