Using Twill from Python to open a link: " 'module' object has no attribute 'Popen' " What is it?

Question

Hello!!!

I have downloaded and installed Python 2.5.4 on my computer (my OS is Windows XP), downloaded “Goggle App Engine Software Development Kit” and created my first application in Python, which was a directory named helloworld that contained a small python file with the same name (helloworld.py). Here are the contents of that small file:

---

print 'Content-Type: text/plain'
print ''
print 'Hello, world!' 

---

Whenever I ran this application locally on my computer with “Goggle App Engine Software Development Kit”, my browser (FireFox) always showed me a white window with Hello, world! written in it.

Then I downloaded Twill and unpacked it into helloworld directory. Having installed Twill properly, I was able to execute some small commands from Twill shell. For example, I could turn to a web page by some link:

Then I wanted to perform the same operation directly from Python (i.e. by means of using Twill from Python.) Here is what the Twill documentation page says about it:

---

twill's Python API

Using TwillBrowser Making extensions

twill is essentially a thin shell around the mechanize package. All twill commands are implemented in the commands.py file, and pyparsing does the work of parsing the input and converting it into Python commands (see parse.py). Interactive shell work and readline support is implemented via the cmd module (from the standard Python library).

Using twill from Python

There are two fairly simple ways to use twill from Python. (They are compatible with each other, so you don't need to choose between them; just use whichever is appropriate.)

The first is to simply import all of the commands in commands.py and use them directly from Python. For example,

from twill.commands import *
go("http://www.python.org/")
showforms()

This has the advantage of being very simple, as well as being tied directly to the documented set of commands in the commands reference.

---

So I decided to use this way. I deleted the previous contents of helloworld.py and gave it the new contents:

---

from twill.commands import *
go("http://www.python.org/")
showforms()

---

But when I tried to run that file on my computer with “Goggle App Engine Software Development Kit”, my browser, instead of depicting the contents of www.python.org web site, gives me an error message: 'module' object has no attribute 'Popen' :

Please, take a look at the whole page here.

Here are the last three lines of that page:

---

: 'module' object has no attribute 'Popen'

  args = ("'module' object has no attribute 'Popen'",)

  message = "'module' object has no attribute 'Popen'" 

---

Can anybody, please, explain to me what this Popen attribute is all about and what I am doing wrong here?

Thank you all in advance.

---

Update 1

(this update is my response to the second answer provided below by leoluk)

Hello, leoluk!!!

I tried doing it this way:

config use_tidy 0
from twill.commands import *
go("http://www.python.org/")

but it didn't work. I received this error message:

<type 'exceptions.SyntaxError'>: invalid syntax (helloworld.py, line 1)
      args = ('invalid syntax', (r'E:\helloworld\helloworld.py', 1, 15, 'config use_tidy 0\n'))
      filename = r'E:\helloworld\helloworld.py'
      lineno = 1
      message = ''
      msg = 'invalid syntax'
      offset = 15
      print_file_and_line = None
      text = 'config use_tidy 0\n'

(You can see the whole page HERE)

Do You have any idea what it means and what went wrong?

Answer 1

I think you should use mechanize directly. Twill communicates with the system in a way that's not supported by Google App Engine.

import mechanize

browser = mechanize.Browser()

browser.open('http://www.python.org')

for f in browser.forms():
    print f # you'll have to extend it

Answer 2

you can't use anything in the Google App engine. Twill uses stuff not available on google app engine to work. So twill is not fully supported by app engine.

notably, the code is trying to call on an external command, tidy, and calling external commands in app engine doesn't work.

Answer 3

The tidy program does a nice job of producing correct HTML from mangled, broken, eeevil Web pages. By default, twill will run pages through tidy before processing them. This is on by default because the Python libraries that parse HTML are very bad at dealing with incorrect HTML, and will often return incorrect results on "real world" Web pages.

To disable this feature, set config do_run_tidy 0.