I'm trying to crawl websites using a crawler written in Python. I want to integrate Tor with Python meaning I want to crawl the site anonymously using Tor. I have no idea how to do that. Any help on how to do that will be very helpful. I tried doing this. It doesn't seem to work. I checked my ip it is still the same as the one before ...
My use case is that I'm just making a website that I want people all over the world to be able to use, and I want to be able to say things like "This happened at 5:33pm on October 5" and also "This happened 5 minutes ago," etc. Should I use the datetime module? Or just strftime? Or something fancier that isn't part of the std distro of...
Disclaimer: The context is a project I'm working on as part of my Master's degree. I guess it qualifies as homework. Introduction (feel free to skip to the bottom line) Curved 3D surfaces are commonly displayed as a large set of very small triangles. Each triangle has the following properties: 3 corners uniform color Such tha...
I have two JSON objects. One is python array which is converted using json,dumps() and other contains records from database and is serialized using json serializer. I want to combine them into a single JSON object. For eg: obj1 = ["a1", "a2", "a3"] obj2 = [ { "pk": "e1", "model": "AB.abc", "fields": { ...
Hi I have made a custom formwizard and incorporated it into my admin interface. Basically I have taken the change_form.html and left it under the admin interface url: (r'^admin/compilation/evaluation/add/$', EvaluationWizard([EvaluationForm1, EvaluationForm2])), It works, but the admin "session" is not kept. I can access the pag...
Is there any way to view a function's doc string when writing Python in VIM? For instance: def MyFunction(spam): """A function that foobars the spam returns eggs""" return foobar(spam).eggs() I'd like to be able to type MyFunction(spam0) and see the doc string, either as a tooltip or in the status bar or any other way th...
Mechanize (Python) is failing with 401 for me to open http digest URLs. I googled and tried debugging but no success. My code looks like this. import mechanize project = "test" baseurl = "http://trac.somewhere.net" loginurl = "%s/%s/login" % (baseurl, project) b = mechanize.Browser() b.add_password(baseurl, "user", "secret", "some Re...
I am not able to get the repository for core-plot. What I am doing is that I am typing this in the terminal: hg clone https://core-plot.googlecode.com/hg/ core-plot and this is what I get: Traceback (most recent call last): File "/usr/local/bin/hg", line 25, in mercurial.util.set_binary(fp) File "/Library/Python/2.5/site-packages...
Python sorts by byte value by default, which means é comes after z and other equally funny things. What is the best way to sort alphabetically in Python? Is there a library for this? I couldn't find anything. Preferrably sorting should have language support so it understands that åäö should be sorted after z in Swedish, but that ü shoul...
I need to empty the data on a socket (making sure that there is nothing to receive). Unfortunately, there is no function for this in the python socket module. I've implemented something this way: def empty_socket(sock): """remove the data present on the socket""" input = [sock] while 1: inputready, o, e = select.sel...
Is there a easy way to accomplish the same thing in Python as xsl accomplishes with: <xsl:strip-space elements="*"/> So for instance in the following for event, elem in ElementTree.iterparse("/tmp/example.xml"): if elem.tag == "example": print ElementTree.tostring(elem) when the example nodes are printed out all the sp...
Ok this is my problem. I am trying something like this: for i in big_list: del glist[:] for j in range(0:val) glist.append(blah[j]) Idea is to reset list and reuse it for next set of data points. The problem is for some reason if the first list had 3 points. Therefore it used glist[0] glist[1] glist[2] The next...
I'm using this script to connect to sample ftp server and list available directories: from ftplib import FTP ftp = FTP('ftp.cwi.nl') # connect to host, default port (some example server, i'll use other one) ftp.login() # user anonymous, passwd anonymous@ ftp.retrlines('LIST') # list directory contents ftp.quit() Ho...
The on-line documentation states that os.popen is now deprecated. All other deprecated functions duly raise a DeprecationWarning. For instance: >>> import os >>> [c.close() for c in os.popen2('ps h -eo pid:1,command')] __main__:1: DeprecationWarning: os.popen2 is deprecated. Use the subprocess module. [None, None] The function os.pop...
Hello, I'm trying to make a glob-like expansion of a set of DNA strings that have multiple possible bases. The base of my DNA strings contains the letters A, C, G, and T. However, I can have special characters like M which could be an A or a C. For example, say I have the string: ATMM I would like to take this string as input and o...
What is the proper way to use **kwargs in Python when it comes to default values? kwargs returns a dictionary, but what is the best way to set default values, or is there one? Should I just access it as a dictionary? Use get function? class ExampleClass: def __init__(self, **kwargs): self.val = kwargs['val'] sel...
Hello, Here is my problem. I'm working on a Jython program and I have to extract numbers from a PyJavaInstance: [{string1="foo", xxx1, xxx2, ..., xxxN, string2="bar"}] (where xxx are the floating point numbers). My question is how can I extract the numbers and put them in a more simple structure like a python list. Thank you in adv...
I'm telling my program to print out line 53 of an output. Is this error telling me that there aren't that many lines and therefore can not print it out? ...
To help me better understand lambda I wrote this short snippet that rotates and transforms a quad (I hope I got the math right). Now, I want to replace the three steps below with one liner lambdas, possibly in conjunction with map(). Im using a vector class but hopefully, the functions are clear as to what they do. self.orientation = v...
for a in ('90','52.6', '26.5'): if a == '90': z = (' 0',) elif a == '52.6': z = ('0', '5') else: z = ('25') for b in z: cmd = exepath + ' -a ' + str(a) + ' -b ' + str(b) process = Popen(cmd, shell=True, stderr=STDOUT, stdout=PIPE) outputstring = process.communicate()[0] ...