How does the Python interpreter handle files on a shared drive?

Question

Does anyone know of a good guide for the python interpreter? I want to know details as to how it handles files it is interpreting. I have some python files on a shared drive that I want multiple processes to use but I do not know if the separate instances of the interpreter will affect one and other. Any insight would be greatly appreciated.

Answer 1

It would depend on the operating system; if reading the file locks it somehow. Two separate python processes should not interfere with each other (unless they go and access the same resource)

Answer 2

No, they will not effect each other. They are in separate processes, so they have their own view of the universe (their own global variables, locals, call stacks, etc).

That is, unless they consume/lock/modify system wide resources in such a way that they conflict with each other (files, network connections, processor mutexes, etc).

To do a very basic test of this, simply launch two terminals, and run the script in both. If they both work correctly, then you've proven that at least multiple instances of the interpreter can read the script at the same time. As for the question whether resources that those scripts consume/lock/modify will conflict with each other, it is going to entirely depend on the script.

Answer 3

Having multiple Python processes (possibly on different nodes) reading the same files is fine. But the key word is reading: if and when two or more processes (Python or otherwise!) try to write the same file at the same time, that can indeed be a problem (unless the underlying network filesystem handles such cases perfectly and seamlessly -- and I don't know of any network FS that doesn't have at least occasional, bad hiccups in such cases).

If you're on Python 2.6 or better, there's an environment variable, or equivalently a command-line flag, that you can use to tell Python to avoid writing the .pyc bytecode file as it imports the corresponding .py source file - and I quote from python2.6 -h:

-B     : don't write .py[co] files on import; also PYTHONDONTWRITEBYTECODE=x

Use this option, and all your Python processes will coexist peacefully (as least as regards the code files they're using -- of course you could still make a mess if and when you write data files from the processes under such "race conditions" to a shared drive, but that's entirely under your control;-).