Pre-generating GUIDs for use in python?

Question

I have a python program that needs to generate several guids and hand them back with some other data to a client over the network. It may be hit with a lot of requests in a short time period and I would like the latency to be as low as reasonably possible.

Ideally, rather than generating new guids on the fly as the client waits for a response, I would rather be bulk-generating a list of guids in the background that is continually replenished so that I always have pre-generated ones ready to hand out.

I am using the uuid module in python on linux. I understand that this is using the uuidd daemon to get uuids. Does uuidd already take care of pre-genreating uuids so that it always has some ready? From the documentation it appears that it does not.

Is there some setting in python or with uuidd to get it to do this automatically? Is there a more elegant approach then manually creating a background thread in my program that maintains a list of uuids?

Answer 1

Are you certain that the uuid module would in fact be too slow to handle the requests you expect in a timely manner? I would be very surprised if UUID generation accounted for a bottleneck in your application.

I would first build the application to simply use the uuid module and then if you find that this module is in fact slowing things down you should investigate a way to keep a pre-generated list of UUIDs around.

Answer 2

I have tested the performance of the uuid module for generating uuids:

>>> import timeit
>>> timer=timeit.Timer('uuid.uuid1()','import uuid')
>>> timer.repeat(3, 10000)
[0.84600019454956055, 0.8469998836517334, 0.84400010108947754]

How many do you need? Is 10000 per second not enough?

Answer 3

Suppose you have a thread to keep topping up a pool of uuid's.

Here is a very simple version

import uuid,threading,time

class UUID_Pool(threading.Thread):
    pool_size=10000
    def __init__(self):
        super(UUID_Pool,self).__init__()
        self.daemon=True
        self.uuid_pool=set(uuid.uuid1() for x in range(self.pool_size))

    def run(self):
        while True:
            while len(self.uuid_pool) < self.pool_size:
                self.uuid_pool.add(uuid.uuid1())
            time.sleep(0.01)              # top up the pool 100 times/sec

uuid_pool = UUID_Pool()
uuid_pool.start()
get_uuid = uuid_pool.uuid_pool.pop        # make a local binding
uuid=get_uuid()                           # ~60x faster than uuid.uuid1() on my computer

You'd also need to handle the case where your burst empties the pool by using uuid's faster than the thread can generate them.