How to extract and then refer to variables defined in a python module?

Question

I'm trying to build a simple environment check script for my firm's test environment. My goal is to be able to ping each of the hosts defined for a given test environment instance. The hosts are defined in a file like this:

#!/usr/bin/env python
host_ip = '192.168.100.10'
router_ip = '192.168.100.254'
fs_ip = '192.168.200.10'

How can I obtain all of these values in a way that is iterable (i.e. I need to loop through and ping each ip address)?

I've looked at local() and vars(), but trying do something like this:

for key, value in vars():
    print key, value

generates this error:

ValueError: too many values to unpack

I have been able to extract the names of all variables by checking dir(local_variables) for values that don't contain a '__' string, but then I have a list of strings, and I can't figure out how to get from the string to the value of the same-named variable.

Answer 1

You need to do vars().iteritems(). (or .items())

Looping over a dictionary like vars() will only extract the keys in the dictionary.

Example.

>>> for key in vars(): print key
...
__builtins__
__name__
__doc__
key
__package__


>>> for key, value in vars().items(): print key, value
...
__builtins__ <module '__builtin__' (built-in)>
value None
__package__ None
key __doc__
__name__ __main__
__doc__ None

Answer 2

First off, I strongly recommend not doing it that way. Instead, do:

hosts = {
    "host_ip": '192.168.100.10',
    "router_ip": '192.168.100.254',
    "fs_ip": '192.168.200.10',
}

Then you can simply import the module and reference it normally--this gives an ordinary, standard way to access this data from any Python code:

import config
for host, ip in config.hosts.iteritems():
    ...

If you do access variables directly, you're going to get a bunch of stuff you don't want: the builtins (__builtins__, __package__, etc); anything that was imported while setting up the other variables, etc.

You'll also want to make sure that the context you're running in is different from the one whose variables you're iterating over, or you'll be creating new variables in locals() (or vars(), or globals()) while you're iterating over it, and you'll get "RuntimeError: dictionary changed size during iteration".

Answer 3

Here's a hack I use all the time. You got really close when you returned the list of string names, but you have to use the eval() function to return the actual object that bears the name represented by the string:

hosts = [eval('modulename.' + x) for x in dir(local_variables) if '_ip' in x]

If I'm not mistaken, this method also doesn't pose the same drawbacks as locals() and vars() explained by Glen Maynard.

Answer 4

If this really is all that your imported file contains, you could also read it as just a text file, and then parse the input lines using basic string methods.

iplistfile = open("iplist.py")

host_addr_map = {}
for line in iplistfile:
    if not line or line[0] == '#':
        continue

    host, ipaddr = map(str.strip, line.split('='))
    host_addr_map[host] = ipaddr

iplistfile.close()

Now you have a dict of your ip addresses, addressable by host name. To get them all, just use basic dict-style methods:

for hostname in host_addr_map:
    print hostname, host_addr_map[hostname]

print host_addr_map.keys()

This also has the advantage that it removes any temptation any misguided person might have to add more elaborate Python logic to what you thought was just a configuration file.

Answer 5

Glenn Maynard makes a good point, using a dictionary is simpler and more standard. That being said, here are a couple of tricks I've used sometimes:

In the file hosts.py:

#!/usr/bin/env python
host_ip = '192.168.100.10'
router_ip = '192.168.100.254'
fs_ip = '192.168.200.10'

and in another file:

hosts_dict = {}
execfile('hosts.py', hosts_dict)

or

import hosts
hosts_dict = hosts.__dict__

But again, those are both rather hackish.