[Python] 'import feedparser' works via SSH, but fails when in browser

Question

I installed feedparser via SSH, using

$ python setup.py install --home=~/httpdocs/python-libraries/feedparser-4.1/

I did that because I don't seem to have permission to properly run 'python setup.py install'

I am running the following python code in 'test.py'.

print "Content-type: text/html\n\n"
try:
    import feedparser
except:
    print "Cannot import feedparser.\n"

The code runs fine when I am logged in by SSH. But when I view it in a browser, it prints

Cannot import feedparser.

Any ideas?

Answer 1

Sounds like a PYTHONPATH problem. Try changing the code to this and see what happens:

import sys
print "Content-type: text/html\n\n"
try:
    print sys.path
    import feedparser
except:
    print "Cannot import feedparser.\n"
    print sys.path

This probably will show that your ~/httpdocs/python-libraries/ directory is not in the path and you will need to either change sys.path or arrage for a PYTHONPATH environment variable, to correct the problem.

When you change the path, make sure that you use the full directory name, not the ~ shortcut.

Answer 2

Maybe it's the problem with setting correct sys.path while running from shell vs from web server.

More about sys.path here: sys module.

I'd recomend to try adding ~/httpdocs/python-libraries/feedparser-4.1/ (best using full path, without ~/) to your sys.path before the import.

import sys
sys.path.append('/home/user/httpdocs/python-libraries/feedparser-4.1/')
print "Content-type: text/html\n\n"
try:
    import feedparser
except:
    print "Cannot import feedparser.\n"

Oh, and by the way, the httpdocs seems like a document root for your web server. Is it the best idea to put the library there? (well, unless there's the only place you can use...)

---

edit (as a general note)

It's best to avoid the syntax like:

try:
  something
except:
  print "error"

This gives you absolutly no information about the actual error you encounter. You can assume that if you try to import a module, you have ImportError there, but can't be sure.

This makes debugging a real hell. Been there, done that, have lost dozens of hours due to this :)

Whenever you can, try catching one exception type at a time. So:

try:
  import SomeModule
except ImportError:
  print "SomeModule can't be imported"

You can also get familiar with the traceback module. It's in the standard library and it's there so you can use it. So, your exception handling code could be something like this:

sys.path.append('/home/user/httpdocs/python-libraries/feedparser-4.1/')
try:
  import feedparser
except ImportError:
  print "Content-type: text/plain\n\n" # text/plain so we get the stacktrace printed well
  import traceback
  import sys
  traceback.print_exc(sys.stdout) # default is sys.stderr, which is error log in case of web server running your script, we want it on standart output
  sys.exit(1)
# here goes your code to execute when all is ok, including:
print "Content-type: text/html\n\n"

Answer 3

I tried both the sys.path and the $PYTHONPATH solutions, but they didn't seem to work. I didn't try adding PythonPath to httpd.conf or python.conf - that could have worked.

I ended up asking a root user to run # python setup.py install for me. That worked.

I was using Python 2.3.

Answer 4

Rather than catching the exception yourself, use the cgitb module just once at the top of your program, and you'll get better information about all errors.

import cgitb
cgitb.enable()