You can use the WebConfigurationManager to read specific configuration sections. This will return a ConfigurationSection object. You can use this to read/modify the ConfigurationElements in the section. Once you have updated them, you can Save the ConfigurationSection and it will update the file with your changes.
I use this to automatically encrypt the appSettings and connectionStrings on Application_Start if they aren't already encrypted. I haven't actually changed any settings this way, but it seems like you ought to be able to do so.
Saving the updated configuration file may cause the app to recycle depending on how it is built.