Whatever the defined name of your executable (not a library dll) assembly is, add a ".config" at the end...
so if your executable is to be
AcmeWidgets.EastCoast.MyApplicationName.exe
Then the app.config will get renamed to
AcmeWidgets.EastCoast.MyApplicationName.exe.config
However, I wouldn't recommend you attempt to dynamically change these settings (in the config file on disk) at runtime...
instead, can you code your app so that it instead populates and uses static variables from these config settings... and then implement the dynamic "change the value" functionality so that it changes these static variables...
This way you can still "affect" the runtime behavior dynamically, but avoid the hassle of writing to the config file, and Ops management can manage the "default" values in the config file by editing it...