Module run via python myscript.py (not shell input) import uuid import time import multiprocessing def sleep_then_write(content): time.sleep(5) print(content) if __name__ == '__main__': for i in range(15): p = multiprocessing.Process(target=sleep_then_write, args=('Hello World',...
I'm processing a list of thousands of domain names from a DNSBL through dig, creating a CSV of URLs and IPs. This is a very time-consuming process that can take several hours. My server's DNSBL updates every fifteen minutes. Is there a way I can increase throughput in my Python script to keep pace with the server's updates? Edit: the sc...
I'm processing 100k domain names into a CSV based on results taken from Siteadvisor using urllib (not the best method, I know). However, my current script creates too many threads and Python runs into errors. Is there a way I can "chunk" this script to do X number of domains at a time (say, 10-20) to prevent these errors? Thanks in advan...
I'm trying to create a COM Object from a dll in a new thread in Python - so I can run a message pump in that thread: from comtypes.client import CreateObject import threading class MessageThread(threading.Thread): def __init__(self): threading.Thread.__init__(self) self.daemon = True def run(self): pri...
I would like to create a :memory: database in python and access it from different threads. Essentially something like: class T(threading.Thread): def run(self): self.conn = sqlite3.connect(':memory:') # do stuff with the database for i in xrange(N): T().start() and have all the connections referring to the sam...
I'm writing a threading.Thread subclass that implements 'callable' methods. Normally with an arbitrary method on a thread, the method runs in whatever thread calls it. These methods run in the thread that they are called on and either block further execution in the calling thread or return a means to get the result depending on which de...
I'm working on a subclass of threading.Thread which allows its methods to be called and run in the thread represented by the object that they are called on as opposed to the usual behavior. I do this by using decorators on the target method that place the call to the method in a collections.deque and using the run method to process the d...
I am considering utilizing Python to call various dlls that will perform things like accessing the LAN (on Windows) or making HTTP requests. These dlls might be poorly written and get stuck. My first question is, whether isolating these dll calls in Python threads will guarantee that the main Python thread will not get stuck? My second q...
I want to make a Google App Engine app that does the following: Client makes an asynchronous http request Server starts processing that request Client makes ajax http requests to get progress The problem is that the server processing (step #2) may take more than 30 seconds. I know that you can't have threads on Google Application E...