Let's say I have a process "A" that loads a dynamic library "L".
Q: Is there a way to disable access to the "exec" functions to functions inside "L"?
Let's say I have a process "A" that loads a dynamic library "L".
Q: Is there a way to disable access to the "exec" functions to functions inside "L"?
There are possibly some tricks that you can do (for example, use the MMU to map the section of the C library that contains the exec() functions as non-executable) that might get the effect you want.
However - since the dynamic library is running within the same process space as you there is nothing that you can do that would permanently disable it for the library that the library couldn't undo.
The dynamic library shares the same process space as the calling application, so it's definitely not easy (and I think not possible, without also denying it to your application). If you can wrap the library in a separate application, then AppArmor or SELinux may help, but in general: why are you loading an untrusted library into your application?
You may also find that looking into how Chromium deals with sandboxing is helpful.
If you're on Linux, you can do the following:
Implement your OWN version of exec() and system() that do what you want (or don't do), and either LD_PRELOAD it, or pass RTLD_DEEPBIND to dlopen()... This will cause the linker to prefer YOUR versions of these methods over the versions provided by libc.
Use AppArmor for this. It allows to specifically reduce the operations an application can perform: Which files can it read/write, what OS functions can it call, what network services it can use.
It's a bit hard to setup but you can use a tool which records all operations that run your app needs. After a run, you can check the output, modify it a bit and then use it.