Getting Forms on Page in Python

Question

I'm working on a project (web vulnerability scanner).

I have compleate 30% of the program it can scan only GET Methods. but having a problem now. I have no idea how I shall make the program pentest the POST Method (the forms) .

I'm having a idea to make it Extract the form data/names from all the pages on the website. but having no idea how I shall do it. Any ideas? This is in Python.

Answer 1

Are you asking how to use urllib2 to execute a POST method?

You might want to look at the examples.

After trying some of that, you might want to post code with a more specific question.

Answer 2

Use BeautifulSoup for screen scraping.

For heavier scripting, use twill :

twill is a simple language that allows users to browse the Web from a command-line interface. With twill, you can navigate through Web sites that use forms, cookies, and most standard Web features.

With twill, you can easily fill forms and POST them back to a server. Twill has a Python API. A from-filling example:

from twill.commands import go, showforms, formclear, fv, submit

go('http://issola.caltech.edu/~t/qwsgi/qwsgi-demo.cgi/')
go('./widgets')
showforms()

formclear('1')
fv("1", "name", "test")
fv("1", "password", "testpass")
fv("1", "confirm", "yes")
showforms()

submit('0')

Answer 3

If you know how to collect the data/names from the form, you just need a way to deal with http POST method. I guess you will need a solution for sending multipart form-data.

You should look at the MultipartPostHandler:

http://odin.himinbi.org/MultipartPostHandler.py

And if you need to support unicode file names , see a fix at: http://peerit.blogspot.com/2007/07/multipartposthandler-doesnt-work-for.html