Bind different ip addresses to urllib2 object in seperate threads

Question

The following code binds specified ip address to socket in main program globally.

import socket
true_socket = socket.socket
def bound_socket(*a, **k):
    sock = true_socket(*a, **k)
    sock.bind((sourceIP, 0))
    return sock
socket.socket = bound_socket

Suppose main program has 10 threads, each with a urllib2 instance running inside the thread. How to bind 10 different ip addresses to each urllib2 object?

Answer 1

You can define a dictionary mapping thread identifier to IP address or use threading.local() global object to define it per thread:

socket_data = threading.local()
socket_data = bind_ip = None

true_socket = socket.socket

def bound_socket(*a, **k):
    sock = true_socket(*a, **k)
    if socket_data.bind_ip is not None:
        sock.bind((socket_data.bind_ip, 0))
    return sock

socket.socket = bound_socket

def thread_target(bind_ip):
    socket_data.bind_ip = bind_ip
    # the rest code

for bind_ip in [...]:
    thread = Thread(target=thread_target, args=(bind_ip,))
    # ...

But note, that is rather dirty hack. A better way is to extend connect() method in subclass of HTTPConnection and redefine http_open() method in subclass of HTTPHandler.