views:

32

answers:

1

Is there a way to import a Python module stored in a cStringIO data structure vs. physical disk file?

It looks like "imp.load_compiled(name, pathname[, file])" is what I need, but the description of this method (and similar methods) has the following disclaimer:

Quote: "The file argument is the byte-compiled code file, open for reading in binary mode, from the beginning. It must currently be a real file object, not a user-defined class emulating a file." [1]

I tried using a cStringIO object vs. a real file object, but the help documentation is correct - only a real file object can be used.

Any ideas on why these modules would impose such a restriction or is this just an historical artifact?

Are there any techniques I can use to avoid this physical file requirement?

Thanks, Malcolm

[1] http://docs.python.org/library/imp.html#imp.load_module

+2  A: 

Something like this perhaps?

import types
import sys

src = """
def hello(who):
    print 'hello', who
"""

def module_from_text(modulename, src):
    if modulename in sys.modules:
        module = sys.modules[modulename]
    else:
        module = sys.modules[modulename] = types.ModuleType(modulename)
    exec compile(src, '<no-file>', 'exec') in module.__dict__
    return module

module_from_text('flup', src)
import flup
flup.hello('world')

Which prints:

hello world

EDIT:

Evaluating code in this way treads nearish the realm of writing custom importers. It may be useful to look at PEP 302, and Doug Hellmann's PyMOTW: Modules and Imports.

Matt Anderson
Matt,Very nicely done!!! Thank you!Malcolm
Malcolm
Matt: I've been studying and experimenting with your code and it works great.Wonder if you can help me out with some of these questions:1. Why don't I need to do anything with the module value returned by module_from_text()?2. Why do I use module.__dict vs. globals(), locals() for the value of the exec statement's 'in' clause?3. Are there any downsides to this approach as long as my src value comes from a trusted source?Thank you,Malcolm
Malcolm
@Malcolm: 1. You can do what you'd like with the returned value, but you don't need it because the module is now in `sys.modules`. 2. Because you want the code to exec in the namespace of the module (and put definitions and assignments in there); if you `exec`'d in some other namespace, the assignments wouldn't end up in the module. 3. Not sure. This is similar, I believe, to how the import internals work, but I may have left something important out (like exception handling). See the post body for a couple references.
Matt Anderson
@Matt: Thank you so much for your help. Your explanation and associated references helped answer my questions. Regards, Malcolm
Malcolm