Simple way to simulate a slow network in python.

Question

Scenario. I have a client with two network connections to a server. One connection uses a mobile phone, and the other is using a wlan connection.

The way that I have solved this is by having the server listen at two ports. But, the mobile connection should be slower than the wlan connection. The data that is sent, is usually just a line of text. I have solved the slower connection issue by allowing the mobile connection to send data in a five second interval, the wlan connection have an interval of one second.

But is there a better way of doing the slowdown? Perhaps by sending smaller packets, meaning I need to send more data?

Any ideas on a good way to slow down one of the connections?

Orjanp

A simple client example with only one connection.

def client():
 import sys, time, socket

 port = 11111
 host = '127.0.0.1'
 buf_size = 1024

 try:
  mySocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
  mySocket.connect((host, port))
 except socket.error, (value, message):
  if mySocket:
   mySocket.close()
  print 'Could not open socket: ' + message
  sys.exit(1)
 mySocket.send('Hello, server')
 data = mySocket.recv(buf_size)
 print data
 time.sleep(5)

 mySocket.close()

client()

A simple server listening to one port.

def server():
 import sys, os, socket

 port = 11111
 host = ''
 backlog = 5
 buf_size = 1024

 try:
  listening_socket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
  listening_socket.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1) 
  listening_socket.bind((host, port)) 
  listening_socket.listen(backlog)
 except socket.error, (value, message):
  if listening_socket:
   listening_socket.close()
  print 'Could not open socket: ' + message
  sys.exit(1)

 while True:
  accepted_socket, adress = listening_socket.accept()
  data = accepted_socket.recv(buf_size)
  if data:
   accepted_socket.send('Hello, and goodbye.')
  accepted_socket.close()

server()

Answer 1

Well, what makes a network connection "slower" than another, is due to latency and/or bandwidth. So if you want to have a realistic simulation, you need to find the bandwidth of your mobile phone connection, as well as its latency, and simulate that in your client program.

But you seem to imply that you send very little data, so bandwidth is probably not really going to affect your connection speed. So you can just simulate latency, and that's doing what you do: sleep(latency) between each packet sent. 5 second seems like a lot though.

But if you think bandwidth might be relevant, it's actually quite simple to simulate: bandwidth is the max number of bytes per second that you can send, and latency is the duration it's going to take to get to its destination.

How to do it: have a global timestamp "blockedUntil", representing the time until your connection becomes free again to send data. Initialise to 0 at the beginning of your program, as we assume it's not being used yet. Then, everytime you have a packet to send, If "_blockedUntil" is less than now(), set it to now(). Then calculate the time it would take to write to your fictitious "wire" by doing: packet.size() / bandwidth, that'll get you a time duration, add the latency, and add that to "blockedUntil".

Now compute dt = blockedUntil - now(), add the packet to the queue, and add a timer firing in "dt", which will pop the first packet in the queue, and send it.

There you go, you've simulated bandwidth and latency.

Edit: as someone mentioned there's the question of dropped packets too. You can simulate that by having a probability of dropping packets. Note: This is solely possible when one is manipulating packets from an unconnected protocol such as Ethernet or UDP. In the case of TCP for example, it won't work.

Answer 2

Aside from using an external tool to simulate the kind of network you're interested in, one good approach is to use a substitute implementation of socket.

This involves making the socket construction a parameter to your function, instead of importing the socket module and using it directly. For normal operation, you will pass in the real socket type, but when you want to test various adverse network conditions, you can pass in an implementation that simulates those conditions. For example, you might create a socket type which parameterizes latency and bandwidth (untested code, beware):

import time, socket

class ControllableSocket:
    def __init__(self, latency, bandwidth):
        self._latency = latency
        self._bandwidth = bandwidth
        self._bytesSent = 0
        self._timeCreated = time.time()
        self._socket = socket.socket()

    def send(self, bytes):
        now = time.time()
        connectionDuration = now - self._timeCreated
        self._bytesSent += len(bytes)
        # How long should it have taken to send how many bytes we've sent with our
        # given bandwidth limitation?
        requiredDuration = self._bytesSent / self._bandwidth
        time.sleep(max(requiredDuration - connectionDuration, self._latency))
        return self._socket.send(bytes)

If you implement the other socket methods, connect, recv, etc, you can substitute an instance of this class for an instance of the real socket type. This leaves all the rest of your program completely free of any knowledge of your simulation, simplifying it, while also letting you try out many different network configurations by just implementing a new socket type that simulates them.

This idea is one of the reasons Twisted explicitly separates the idea of "protocols" - objects which know how to interpret bytes from the network and generate new bytes to send to the network - from "transports" - objects which know how to get bytes off the network and put bytes onto it. The separation eases testing and allows novel configurations like this one, where a simulation of some other network conditions (which may be difficult to produce for real) is provided by the transport.

Answer 3

At the risk of not answering the question you asked, I would look for software that does this at a lower level.

Netlimiter does this for Windows. I think BWMeter and Bandwidth Controller can do it too.

pyshaper is a similar tool for Linux. Open source. You might just be able to import it into your Python program.

(Another thing to consider is that you might already have a router capable of shaping traffic the way you want. That's a pretty big dependency to add to your software, though, and it might be more work to configure.)