emulating bash 'source' in python

Question

I have a script that looks something like this:

export foo=/tmp/foo                                          
export bar=/tmp/bar

Every time I build I run 'source init_env' (where init_env is the above script) to set up some variables

To accomplish the same in python I had this code running

reg = re.compile('export (?P<name>\w+)(\=(?P<value>.+))*')
for line in open(file):
    m = reg.match(line)
    if m:
        name = m.group('name')
        value = ''
        if m.group('value'):
            value = m.group('value')
        os.putenv(name, value)

But then someone decided it would be nice to add a line like:

export PATH="/foo/bar:/bar/foo:$PATH"     

to the init_env file. Obviously my python script fell apart. I could modify the python script to handle this line, but then it'll just break later on when someone comes up with a new feature to use in the init_env file

The question is if there is an easy way to run a bash command and let it modify my os.environ?

Answer 1

Rather than having your Python script source the bash script, it would be simpler and more elegant to have a wrapper script source init_env and then run your Python script with the modified environment.

#!/bin/bash
source init_env
/run/python/script.py

Answer 2

the python doc says if you want to manipulate the environment you should manipulate os.environ directly instead of using os.putenv(). i consider that a bug, but i digress.

the problem with your approach is that you are trying to interpret shell scripts. first you just interpreted the export statement, then you noticed people are using variable expansion, later people will put conditionals in their files, or process substitution. in the end you will have a full blown bash shell script language interpreter (with a gazillion bugs). don't do that.

let bash interpret the file for you and you just collect the result.

you can do it like this

import os
import pprint
import subprocess

# subprocess will run the command in sh
# in sh the command source is known as . (dot)
# the env at the end is to dump the environment to stdout
command = ". ./init_env && env"

proc = subprocess.Popen(command,
                        stdout = subprocess.PIPE,
                        shell = True,
                        cwd = os.getcwd())

for line in proc.stdout:
  (key, _, value) = line.partition("=")
  os.environ[key] = value

proc.communicate()

pprint.pprint(dict(os.environ))

note that since subprocess is running sh the statements allowed in init_env are limited to sh syntax. i am sure there is way to tell subprocess to run bash instead of sh.

make sure that you capture any errors in case sh fails to source init_env.

enjoy