Python - how to refer to relative paths of resources when working with code repository

Question

We are working with a code repository which is deployed both to windows and linux, sometimes on different directories. How should one of the modules inside the project refer to one of the non-python resources in the project (CSV file, etc.)? If we do something like

thefile=open('test.csv')

or

thefile=open('../somedirectory/test.csv')

It will work only when the script is run from one specific directory, or a subset of the directories. What I would like to do is something like:

path=getBasePathOfProject()+'/somedirectory/test.csv'
thefile=open(path)

Is this the right way? Is it possible? Thanks

Answer 1

Try to use a filename relative to the current files path. Example for './my_file':

fn = os.path.join(os.path.dirname(__file__), 'my_file')

Answer 2

You can use the build in file variable. It contains the path of the current file. I would implement getBaseOfProject in a module in the root of your project. There I would get the path part of file and would return that. This method can then be used everywhere in your project.

Answer 3

import os
cwd = os.getcwd()
path = os.path.join(cwd, "my_file")
f = open(path)

You also try to normalize your cwd using os.path.abspath(os.getcwd()). More info here.

Answer 4

I often use something similar to this:

import os
DATA_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), 'datadir'))

# if you have more paths to set, you might want to shorten this as
here = lambda x: os.path.abspath(os.path.join(os.path.dirname(__file__), x))
DATA_DIR = here('datadir') 

pathjoin = os.path.join
# ...
# later in script
for fn in os.listdir(DATA_DIR):
    f = open(pathjoin(DATA_DIR, fn))
    # ...

The variable

__file__

holds the file name of the script you write that code in, so you can make paths relative to script, but still written with absolute paths. It works quite well for several reasons:

• path is absolute, but still relative
• the project can still be deployed in a relative container

But you need to watch for platform compatibility - Windows' os.pathsep is different than UNIX.