app.config for DLLs will never be used.
app.config for EXEs is renamed to <Application>.exe.config, and used by the EXE
web.config is used by websites/applications.
Essentially, if Visual Studio helpfully adds an app.config file to a DLL project for you, all it's giving you is something to copy and paste into the appropriate final config file - either the app.config of the EXE, or the web.config of the web site/application.
@vikps comment: Ignoring, for a second, the issues of website subdirectories and configSource, you can not have more than one configuration file involved in the configuration of an application. And neither subdirectories or configSource will help you, in this instance.
With configSource, the application can specify that an entire configuration section should be read from an alternative configuration file. But you can only do this if the entire configuration section exists in this other file (you cannot, for instance, have a connectionStrings section in the web.config, and also load some more connection strings from another source)