The issue here is that the assembly binding information, due to its nature, is loaded with the AppDomain.
As such, to be able to change assembly binding behavior at runtime, the AppDomain would need to be restarted to reload the configuration.
Once a dll is loaded into an AppDomain, it cannot be unloaded, hence the need to restart it to load a new set of DLLs, which impersonate a different version (redirection).
You will need to isolate your application into two AppDomains, the startup AppDomain (bootstrapper), which is solely resposible for monitoring and editing the config file, and the 'worker' AppDomain, for which the modified app.config will apply, and which does all the work for your application.
If you are expecting to be able to have your application just magically change the redirection behavior without having to re-initialize any part of your application, unfortunately this is not going to be possible.
However, I have implemented the above methodology that I have described in an application recently, and it works perfectly. The caveat is that the part of the application within which these 'isolated binding redirects' apply, must be re-initialized any time a configuration change is made.