What is the best browser automation tool for Python?

Question

Hello, I want to write a following script: given a text file with the list of actions to be executed on a certain site it would use some browser's (IE probably, because I don't know anything about other drive-able ones) CSS rendering and JS executing capabilities to imitate a user doing those actions on a site.

So I've found this page and the "web testing" section of it and it lists:

• PAMIE
• PYXPCOM
• Windmill
• Selenium

The latter two run some sort of a server on the localhost (which doesn't seem to be well suitable decision).
Pamie has some sort of the worst documentation ever and lots of "Under Construction" pages on their site last updated in 2006.
And PyXPCOM seems to be created not specifically for FF. I may be a bad Googler but I still didn't find a decent example of using PyXPCOM for something like using FF.

Which way to guide the browser would you prefer for my purposes and why? TIA

Update:
I need to be able to render the JavaScript on the page. I guess using some server-based library is not a good solution since it is limited to one instance of browser at a time, so I won't be able to, for example, set 2 browser instances to 2 different proxies without having to make 2 servers (is this correct?).

Answer 1

You should try Win32Com as it gives you the possibility to drive programs using their COM server, and if they don't have one you can make it execute simple actions like emulate the pression of a key or move a window from foreground to background and back.
In the case of IE, since it has a COM server, you can use functions of the IE Python object to make it write things in forms, check boxes, download files and almost everything you could imagine.

Answer 2

Check out MozMill, it's for Firefox. Uses JavaScript, should be easier than COM.

Answer 3

I'm not sure if Qt's Webkit through PyQt can solve your issue but I can usually control a simple browser with Python in that way, for instance:

import sys
import time

from PyQt4.QtCore import *
from PyQt4.QtGui import *
from PyQt4.QtWebKit import *

app = QApplication(sys.argv)

web = QWebView()

settings = web.settings()
settings.setAttribute(QWebSettings.PluginsEnabled, True)
settings.setAttribute(QWebSettings.JavaEnabled, True)
settings.setAttribute(QWebSettings.JavascriptCanOpenWindows, True)
settings.setAttribute(QWebSettings.JavascriptCanAccessClipboard, True)
settings.setAttribute(QWebSettings.DeveloperExtrasEnabled, True)
settings.setAttribute(QWebSettings.ZoomTextOnly, True)


settings.setOfflineStoragePath('.')
settings.setIconDatabasePath (".")

url = 'http://stackoverflow.com'

web.load(QUrl(url))
web.show()

sys.exit(app.exec_())

Answer 4

You can use autoit COM object using pywin32.

Another option is to use selenium 2 (aka webdriver) python bindings.

Answer 5

I'd encourage you to look again at Selenium...that's really what you want to do. Do you need to actually render the page in a browser, or just emulate navigation and clicking?

Mechanize gives you stateful programmatic web browsing, and might be what you're looking for.