Python includes, module scope issue

Question

Hi everyone,

I'm working on my first significant Python project and I'm having trouble with scope issues and executing code in included files. Previously my experience is with PHP.

What I would like to do is have one single file that sets up a number of configuration variables, which would then be used throughout the code. Also, I want to make certain functions and classes available globally. For example, the main file would include a single other file, and that file would load a bunch of commonly used functions (each in its own file) and a configuration file. Within those loaded files, I also want to be able to access the functions and configuration variables. What I don't want to do, is to have to put the entire routine at the beginning of each (included) file to include all of the rest. Also, these included files are in various sub-directories, which is making it much harder to import them (especially if I have to re-import in every single file).

Anyway I'm looking for general advice on the best way to structure the code to achieve what I want.

Thanks!

Answer 1

As far as I know program-wide global variables/functions/classes/etc. does not exist in Python, everything is "confined" in some module (namespace). So if you want some functions or classes to be used in many parts of your code one solution is creating some modules like: "globFunCl" (defining/importing from elsewhere everything you want to be "global") and "config" (containing configuration variables) and importing those everywhere you need them. If you don't like idea of using nested namespaces you can use:

from globFunCl import *

This way you'll "hide" namespaces (making names look like "globals").

I'm not sure what you mean by not wanting to "put the entire routine at the beginning of each (included) file to include all of the rest", I'm afraid you can't really escape from this. Check out the Python Packages though, they should make it easier for you.

Answer 2

In python, it is a common practice to have a bunch of modules that implement various functions and then have one single module that is the point-of-access to all the functions. This is basically the facade pattern.

An example: say you're writing a package foo, which includes the bar, baz, and moo modules.

~/project/foo
~/project/foo/__init__.py
~/project/foo/bar.py
~/project/foo/baz.py
~/project/foo/moo.py
~/project/foo/config.py

What you would usually do is write __init__.py like this:

from foo.bar import func1, func2
from foo.baz import func3, constant1
from foo.moo import func1 as moofunc1
from foo.config import *

Now, when you want to use the functions you just do

import foo
foo.func1()
print foo.constant1
# assuming config defines a config1 variable
print foo.config1

---

If you wanted, you could arrange your code so that you only need to write

import foo

At the top of every module, and then access everything through foo (which you should probably name "globals" or something to that effect). If you don't like namespaces, you could even do

from foo import *

and have everything as global, but this is really not recommended. Remember: namespaces are one honking great idea!

Answer 3

This is a two-step process:

1. In your module globals.py import the items from wherever.
2. In all of your other modules, do "from globals import *"

This brings all of those names into the current module's namespace.

Now, having told you how to do this, let me suggest that you don't. First of all, you are loading up the local namespace with a bunch of "magically defined" entities. This violates precept 2 of the Zen of Python, "Explicit is better than implicit." Instead of "from foo import *", try using "import foo" and then saying "foo.some_value". If you want to use the shorter names, use "from foo import mumble, snort". Either of these methods directly exposes the actual use of the module foo.py. Using the globals.py method is just a little too magic. The primary exception to this is in an __init__.py where you are hiding some internal aspects of a package.

Globals are also semi-evil in that it can be very difficult to figure out who is modifying (or corrupting) them. If you have well-defined routines for getting/setting globals, then debugging them can be much simpler.

I know that PHP has this "everything is one, big, happy namespace" concept, but it's really just an artifact of poor language design.

Answer 4

This depends a bit on how you want to package things up. You can either think in terms of files or modules. The latter is "more pythonic", and enables you to decide exactly which items (and they can be anything with a name: classes, functions, variables, etc.) you want to make visible.

The basic rule is that for any file or module you import, anything directly in its namespace can be accessed. So if myfile.py contains definitions def myfun(...): and class myclass(...) as well as myvar = ... then you can access them from another file by

import myfile
y = myfile.myfun(...)
x = myfile.myvar

or

from myfile import myfun, myvar, myclass

Crucially, anything at the top level of myfile is accessible, including imports. So if myfile contains from foo import bar, then myfile.bar is also available.

Answer 5

Others have talked about the right way to do this from Python.

Let me add, however, that having users change a source file to configure your application -- while common in the PHP world -- isn't very Pythonic. Consider using the ConfigParser standard library module or the ConfigObj third-party module (which supports complex datatypes such as nested dicts, lists, etc) to parse a configuration file instead.

Having a config.py module which is responsible for parsing this configuration file, and which other code in your application imports and queries for settings, is appropriate -- but having the user edit this config.py on installation really isn't.