What have you used to test (functional/load/stress) your network service with its custom protocol?

Question

I recently created a turn-based game server that can accept 10s of thousands of simultaneous client connections (long story short - epoll on Linux). Communication is based on a simple, custom, line-based protocol. This server allows clients to connect, seek for other players in game matches, play said games (send moves, chat messages, etc.), and be notified when the game has ended.

What I'm looking to do now is test the server by simulating client connections. I'm hoping to support 10s of thousands of simultaneous connections, so this testing is very important to me. What do you guys use for your own testing?

Some things I'm researching now are: pexpect (python expect lib for the functional testing) and tsung for load testing.

I'd like to be able to just test from my laptop since I do not have a cluster of client machines to connect from. Perhaps I'd need to use ip aliasing or some-such in order to generate 100s of thousands of outbound sockets (limit is 65K per interface AFAIK).

Anyway, it seems to me like I need something fairly custom but I thought I'd ask before I went down that path.

Thanks!

Answer 1

I've used JMeter with custom sampler and assertion components before to do automated regression/load testing for a banking application with a custom protocol (Java RMI based API).

It's not exactly lightweight though, and you'll end up doing a lot of extra coding in the JMeter components to support your custom protocol. I'm guessing you'd have to code your own Java socket based client in this case.

But it gives you a lot of flexibility in defining the logic for testing the components, so you can do whatever you want inside there. It scales nicely as well, and allows you to throw a lot of concurrent connections at the system under test.

Answer 2

We are using HP LoadRunner it's the state of the art load testing product. (But also an expensive one). It can simulate thousands of requests to the server and provides metrics on response time etc..

Answer 3

I decided it was best to "roll my own" to start with.