Why doesn't import prevent NameError in a python script run with execfile()?

Question

I looked at a number of existing questions about NameError exceptions when scripts are run with exec statements or execfile() in Python, but haven't found a good explanation yet of the following behavior.

I want to make a simple game that creates script objects at runtime with execfile(). Below are 4 modules that demonstrate the problem (please bear with me, this is as simple as I could make it!). The main program just loads a script using execfile() and then calls a script manager to run the script objects:

# game.py

import script_mgr
import gamelib  # must be imported here to prevent NameError, any place else has no effect

def main():
  execfile("script.py")
  script_mgr.run()

main()

The script file just creates an object that plays a sound and then adds the object to a list in the script manager:

 script.py

import script_mgr
#import gamelib # (has no effect here)

class ScriptObject:
  def action(self):
    print("ScriptObject.action(): calling gamelib.play_sound()")
    gamelib.play_sound()

obj = ScriptObject()
script_mgr.add_script_object(obj)

The script manager just calls the action() function of each script:

# script_mgr.py

#import gamelib # (has no effect here)

script_objects = []

def add_script_object(obj):
  script_objects.append(obj)

def run():
  for obj in script_objects:
    obj.action()

The gamelib function is defined in a fourth module, which is the troublesome one to be accessed:

# gamelib.py

def play_sound():
  print("boom!")

The above code works with the following output:

mhack:exec $ python game.py
ScriptObject.action(): calling gamelib.play_sound()
boom!
mhack:exec $ 

However, if I comment-out the 'import gamelib' statement in game.py and uncomment the 'import gamelib' in script.py, I get the following error:

mhack:exec $ python game.py
ScriptObject.action(): calling gamelib.play_sound()
Traceback (most recent call last):
  File "game.py", line 10, in 
    main()
  File "game.py", line 8, in main
    script_mgr.run()
  File "/Users/williamknight/proj/test/python/exec/script_mgr.py", line 12, in run
    obj.action()
  File "script.py", line 9, in action
    gamelib.play_sound()
NameError: global name 'gamelib' is not defined

My question is: 1) Why is the import needed in the 'game.py' module, the one that execs the script? 2) Why doesn't it work to import 'gamelib' from the module where it is referenced (script.py) or the module where it is called (script_mgr.py)?

This happens on Python 2.5.1

Answer 1

From the Python documentation for execfile:

execfile(filename[, globals[, locals]])

If the locals dictionary is omitted it defaults to the globals dictionary. If both dictionaries are omitted, the expression is executed in the environment where execfile() is called.

There are two optional arguments for execfile. Since you omit them both, your script is being executed in the environment where execfile is called. Hence the reason the import in game.py changes the behaviour.

In addition, I concluded the following behaviour of import in game.py and script.py:

• In game.py import gamelib imports the gamelib module into both globals and locals. This is the environment passed to script.py which is why gamelib is accessible in the ScriptObject action method (accessed from globals).

• In script.py import gamelib imports the gamelib module into locals only (not sure of the reason). So when trying to access gamelib from the ScriptObject action method from globals you have the NameError. It will work if you move the import into the scope of the action method as follows (gamelib will be accessed from locals):

  class ScriptObject:
      def action(self):
          import gamelib
          print("ScriptObject.action(): calling gamelib.play_sound()")
          gamelib.play_sound()
  

Answer 2

The reason the 'import gamelib' in script.py has no effect is because it imports into the local scope of game.py main(), because this is the scope in which the import is executed. This scope is not a visible scope for ScriptObject.action() when it executes.

Adding debug code to print out the changes in the globals() and locals() reveals what is going on in the following modified version of the program:

# game.py

import script_mgr
import gamelib  # puts gamelib into globals() of game.py

# a debug global variable 
_game_global = "BEF main()" 

def report_dict(d):
  s = ""
  keys = d.keys()
  keys.sort() 
  for i, k in enumerate(keys):
    ln = "%04d %s: %s\n" % (i, k, d[k])
    s += ln
  return s

def main():
  print("--- game(): BEF exec: globals:\n%s" % (report_dict(globals())))
  print("--- game(): BEF exec: locals:\n%s" % (report_dict(locals())))
  global _game_global 
  _game_global = "in main(), BEF execfile()"
  execfile("script.py")
  _game_global = "in main(), AFT execfile()"
  print("--- game(): AFT exec: globals:\n%s" % (report_dict(globals())))
  print("--- game(): AFT exec: locals:\n%s" % (report_dict(locals())))
  script_mgr.run()

main()

# script.py 

import script_mgr
import gamelib  # puts gamelib into the local scope of game.py main()
import pdb # a test import that only shows up in the local scope of game.py main(). It will _not_ show up in any visible scope of ScriptObject.action()!

class ScriptObject:
  def action(self):
    def report_dict(d):
      s = ""
      keys = d.keys()
      keys.sort()
      for i, k in enumerate(keys):
        ln = "%04d %s: %s\n" % (i, k, d[k])
        s += ln
      return s
    print("--- ScriptObject.action(): globals:\n%s" % (report_dict(globals())))
    print("--- ScriptObject.action(): locals:\n%s" % (report_dict(locals())))
    gamelib.play_sound()

obj = ScriptObject()
script_mgr.add_script_object(obj)

Here is the debug output of the program:

--- game(): BEF exec: globals:
0000 __builtins__: 
0001 __doc__: None
0002 __file__: game.py
0003 __name__: __main__
0004 _game_global: BEF main()
0005 gamelib: 
0006 main: 
0007 report_dict: 
0008 script_mgr: 

--- game(): BEF exec: locals:

--- game(): AFT exec: globals:
0000 __builtins__: 
0001 __doc__: None
0002 __file__: game.py
0003 __name__: __main__
0004 _game_global: in main(), AFT execfile()
0005 gamelib: 
0006 main: 
0007 report_dict: 
0008 script_mgr: 

--- game(): AFT exec: locals:
0000 ScriptObject: __main__.ScriptObject
0001 gamelib: 
0002 obj: 
0003 pdb: 
0004 script_mgr: 

--- ScriptObject.action(): globals:
0000 __builtins__: 
0001 __doc__: None
0002 __file__: game.py
0003 __name__: __main__
0004 _game_global: in main(), AFT execfile()
0005 gamelib: 
0006 main: 
0007 report_dict: 
0008 script_mgr: 

--- ScriptObject.action(): locals:
0000 report_dict: 
0001 self: 


boom!

Instead of trying to put imports in game.py or the module level of script.py, I will follow Yukiko's suggestion to put import statements at the local scope of the script object member functions. This seems a little awkward to me, and there may be some better way to specify such imports for exec'd scripts, but at least I now understand what is happening.