I have a particular file I want to monitor for file read attempts by all applications on OSX. I'd like to be able to interrupt the requests so I could decide which applications have permission to read the file and which don't (by querying the user, or checking a cache of user responses). Is this possible with the OSX API? If not, is it even possible to get a list of which applications or processes do read a file?
The writings of Amit Singh should come in very handy. He explored the API that provides FileSystem events a few years ago, and provided a sample tool that allows you to intercept FS events. It's open source!
If i remember his conclusion properly, their isn't an official API, but you can use apple's tools to achieve what you want.
I'm not saying there's no way to do this, but what @Jonathan is talking about isn't it.
That API is for tracking the creation, change, and destruction of files. Notably this tool is used by things like Spotlight to watch activity on the filesystem for new, interesting files.
But, wisely, reading isn't one of the events it tracks.
And even if reading WAS tracked, it is still the wrong mechanism, as it's a notification system after the fact, not in line with the call itself.
I seriously doubt what you want is possible the way you describe it.
With Access Control Lists, you can limit access at the user level (Fred can read the file, but Bob can not). This is a setting on the file itself. But there's no mechanism to allow Bobs App1 to read a file, while Bobs App2 can not, since there's really no formal mechanism of "application identity" beyond the command to executed, or whatever the program "says" its name is (both of which can be spoofed if motivated enough).
However, feel free to crawl the Darwin sources -- no doubt the answer is buried in there somewhere near the open(2) call.
EDIT, regarding comment.
What are you trying to do? What's the overall context?
Another thing that you may want to try is to use FUSE.
FUSE is a utility that let's you have "user space filesystems". People use FUSE for many purposes, like reading NTFS volumes, or mounting remote system via SSH.
They have a simple example, that gives you a skeleton that you can fill in for your purposes.
For most of the use cases, you'll simple defer to the system. However, for OPEN you will add your logic. Then you could point your FUSE utility at a directory, and "mount it". Then all of the files below that directory can use your new behavior.
I'm still not sure how you will identify Apps by name, but if it's not a real "security" issue, just for local control, I imaging you can come up with something. Activity Monitor has apps names, so they must be available, and FUSE will be running within the process space (I think), rather than through some external mechanism.
All that said, I think FUSE is your best bet, but it's probably not appropriate if you want to do this to "any file" with no preparation by the user (like not installing FUSE). If you wanted to do "any file", your FUSE system would need to be mounted at root, and then you'll simply have a full "clone" of the filesystem, with those files from the normal root "unprotected", while those from your new FUSE root will be protected. So, if someone wanted to NOT use your FUSE system, the real file is readily available to them through the actual file location.