How to program a connection pool?

Question

Is there a known algorithm for implementing a connection pool? If not what are the known algorithms and what are their trade-offs?
What design patterns are common when designing and programming a connection pool?
Are there any code examples implement a connection pool using boost.asio?
Is it a good idea to use a connection pool for persisting connections (not http)?
How is threading related to connection pooling? When do you need a new thread?

Answer 1

If you are looking for a pure thread-pooling policy (may be a connection or any resource) there are two simple approaches viz:-

1. Half Sync/Half Async Model (usually using using message queues to pass information).

2. Leaders/Followers Model (usually using request queues to pass information).

The first approach goes like this:-

• You create a pool of threads to handle a resource. Often this size (number of threads) needs to be configurable. Call these threads 'Workers'.
• You then create a master thread that will dispatch the work to the Worker threads. The application program dispatches the task as a message to the master thread.
• The master thread puts the same on the message Q of a chosen Worker thread and the Worker thread removes itself from the pool. Choosing and removing the Worker thread needs synchronization.
• After the Worker completes the task, it returns to the thread-pool.

The master thread itself can consume the tasks it gets in FCFS or a prioritized manner. This will depend on your implementation.

The second model (Leader/Followers) goes something like this:-

• Create a thread pool. Initially all are Workers. Then elect a Leader, automatically rest-all become followers. Note that electing a Leader has to be synchronized.
• Put all the data to be processed on a single request Q.
• The thread-pool Leader dequeues the task. It then immediately elects a new Leader and starts executing the task.
• The new Leader picks up the next task.

There may be other approaches as well, but the ones outlined above are simple that work with most use-cases.

Half Sync/Half Async Major Weakness:-

• Higher context switching, synchronization, and data copying overhead.

Leader/Follwers Major Weakness:-

• Implementation complexity of Leader election in thread pool.

Now you can decide for yourself the more correct approach. HTH,