Hot swapping python code (duck type functions?)

Question

I've been thinking about this far too long and haven't gotten any idea, maybe some of you can help.

I have a folder of python scripts, all of which have the same surrounding body (literally, I generated it from a shell script), but have one chunk that's different than all of them. In other words:

Top piece of code (always the same)
Middle piece of code (changes from file to file)
Bottom piece of code (always the same)

And I realized today that this is a bad idea, for example, if I want to change something from the top or bottom sections, I need to write a shell script to do it. (Not that that's hard, it just seems like it's very bad code wise).

So what I want to do, is have one outer python script that is like this:

Top piece of code
Dynamic function that calls the middle piece of code (based on a parameter)
Bottom piece of code

And then every other python file in the folder can simply be the middle piece of code. However, normal module wouldn't work here (unless I'm mistaken), because I would get the code I need to execute from the arguement, which would be a string, and thus I wouldn't know which function to run until runtime.

So I thought up two more solutions:

1. I could write up a bunch of if statements, one to run each script based on a certain parameter. I rejected this, as it's even worse than the previous design.

2. I could use:

   os.command(sys.argv[0] scriptName.py)

   which would run the script, but calling python to call python doesn't seem very elegant to me.

So does anyone have any other ideas? Thank you.

Answer 1

If you know the name of the function as a string and the name of module as a string, then you can do

mod = __import__(module_name)
fn = getattr(mod, fn_name)
fn()

Answer 2

However, normal module wouldn't work here (unless I'm mistaken), because I would get the code I need to execute from the arguement, which would be a string, and thus I wouldn't know which function to run until runtime.

It will work just fine - use __import__ builtin or, if you have very complex layout, imp module to import your script. And then you can get the function by module.__dict__[funcname] for example.

Answer 3

Importing a module (as explained in other answers) is definitely the cleaner way to do this, but if for some reason that doesn't work, as long as you're not doing anything too weird you can use exec. It basically runs the content of another file as if it were included in the current file at the point where exec is called. It's the closest thing Python has to a source statement of the kind included in many shells. As a bare minimum, something like this should work:

exec(open(filename).read(None))

Answer 4

Another possible solution is to have each of your repetitive files import the functionality from the main file

from topAndBottom import top, bottom
top()
# do middle stuff
bottom()

Answer 5

How about this?

function do_thing_one():
   pass

function do_thing_two():
   pass

dispatch = { "one" : do_thing_one,
             "two" : do_thing_two,
           }

# do something to get your string from the command line (optparse, argv, whatever)
# and put it in variable "mystring"

# do top thing
f = dispatch[mystring]
f()
# do bottom thing

Answer 6

In addition to the several answers already posted, consider the Template Method design pattern: make an abstract class such as

class Base(object):
    def top(self): ...
    def bottom(self): ...
    def middle(self): raise NotImplementedError
    def doit(self):
        self.top()
        self.middle()
        self.bottom()

Every pluggable module then makes a class which inherits from this Base and must override middle with the relevant code.

Perhaps not warranted for this simple case (you do still have to import the right module in order to instantiate its class and call doit on it), but still worth keeping in mind (together with its many Pythonic variations, which I have amply explained in many tech talks now available on youtube) for cases where the number or complexity of "pluggable pieces" keeps growing -- Template Method (despite its horrid name;-) is a solid, well-proven and highly scalable pattern [[sometimes a tad too rigid, but that's exactly what I address in those many tech talks -- and that problem doesn't apply to this specific use case]].