I've moved my appsettings section outside of the web.config using:
<appSettings configSource="AppSettings.config"/>
This allows me to change my appsettings without actually restarting IIS.
I know however, that IIS monitors all configuration files constantly. How can I attach to event my-appsetting-has-changed to take some custom action upon that?