Python: better way to open lots of sockets

Question

I have the following program to open lot's of sockets, and hold them open to stress test one of our servers. There are several problem's with this. I think it could be a lot more efficient than a recursive call, and it's really still opening sockets in a serial fashion rather than parallel fashion. I realize there are tools like ab that could probably simulate what I'm trying to do, but I'm hoping to increase my python knowledge. Is this something I should be rewriting as either multi-threaded or multi-process?

> #!/usr/bin/env python
> 
> import socket, time, sys
> sys.setrecursionlimit(2000)
> 
> def open_socket(counter):   
>   sockname = "s" + str(counter)   
>   port = counter + 3000   
>   sockname = socket.socket()  
>   sockname.bind(('localhost', port))  
>   sockname.listen(1)   
>   if counter == 2:
>     time.sleep(20)   
>   elif counter > 2:
>     counter = counter -1
>     open_socket(counter)
> 
> open_socket(1500)

Answer 1

Well it's already multi process - put a sleep in before calling open_socket, and run , say , 500 of them from a shell:

for i in `seq 500`; do ./yourprogram & done

You're not a actually connecting to something though - seems you're setting up server sockets ? If you need to connect to something you surly should test it with parallelism(multiple threads, or run many processes like shown above or using asyncronous connect's). This should be of interest to read:

Answer 2

You can try using Twisted for this. It greatly simplifies networking on Python. Their site has some tutorials to get you started.

However, you could easily see using Python an overkill for this task. A faster option to hack up would be to just open 1500 instances of nc:

for i in {3000..4500};
do
    nc -l -p $i &
done

Answer 3

I was puzzled why you would use recursion instead of a simple loop. My guess is that with a simple loop, you would have overwritten the variable sockname again and again, so that Python's garbage collection would actually close the previous socket after you created the next one. The solution is to store them all in a list, to prevent Python from garbage-collecting them:

def open_socket(counter):
  sockets = []
  for i in range(counter):
     s = socket.socket()
     s.bind(('localhost', i+3000))
     s.listen(1)
     sockets.append(s)
  time.sleep(20)

Also notice that in your code, the first assignment to sockname is completely redundant, as it is overwritten by the second assignment.