I am designing a system to deal with an external web service. The service limits the number of requests that can be made over a certain period of time (T). The system allows for the batching of a certain number of requests (R). There are a certain number of operations that the service supports (O).
My code will process an unknown number of requests from users (I really have no idea at this point, could be one request a day, could be thousands a second, I need to build it with the assumption of thousands a second though). These results will be cached in a database for a period of time. When the database records are out of date the system will need to request the data from the web service again.
I can only access the web service via one IP address with one account (no cheating and getting an account per operation type, or one machine per operation type). The system will (hopefully) all run on a single server.
What I am trying to do (been thinking about it on and off for a couple of weeks without any results that I like) is come up with a system where:
- duplicate requests are merged (duplicate means they have the same request data)
- user requests have priority over system requests
- a system request can be changed to a user request (database update is in the queue and a user is requesting the same data)
- if there are not R user requests for the particular opereration then the remainder are taken from the system requests
- user requests are handled in the same order that they come in (except that once a user request is being handled R requests of the same type are handled).
So, for example, T is 1 second, R is 3, and O is 2. The following requests come into the system:
Request 1, user, operation A, data 1
Request 2, user, operation A, data 2
Request 3, user, operation A, data 1 <- duplicate of request 1
Request 4, system, operation B, data 3
Request 5, system, operation A, data 1 <- duplicate of request 3
Request 6, user, operation B, data 3 <- duplicate of Request 4
Request 7, system, operation A, data 4
Request 8, user, operation A, data 5
Request 9, user, operation A, data 6
Request 10, user, operation A, data 7
Request 11, user, operation B, data 8
Once you deal with the duplicates you would get this:
Request 1, user, operation A, data 1
Request 2, user, operation A, data 2
Request 4, user, operation B, data 3 <- promoted to user from system (msg 6)
Request 7, system, operation A, data 4
Request 8, user, operation A, data 5
Request 9, user, operation A, data 6
Request 10, user, operation A, data 7
Request 11, user, operation B, data 8
The requests should be handled in the following order:
T1 Request 1, Request 2, Request 8
T2 Request 4, Request 11
T3 Request 9, Request 10, Request 7
I think there will likely be 3-7 operation types. Some operation types will have more requests than others. System requests will likely be larger in number than user requests.
Is there a common way of dealing with this sort of problem? A pattern or technology? Am I overthinking it (unfortunatly I cannot get the usage statistics until after it is up and running, I cannot even reasonably guess at what they will be)?
The primary things I am trying to avoid are:
- having system requests handled over user requests ( a system request can wait for weeks, a user request must be processes as soon as it can be)
- not making the same request twice in the period that the data is cached in the database