views:

35

answers:

1

I'm in the process of converting a system from sql server to mongodb. I believe the project is a good candidate for mongodb, but that's not my question.

In the sql database, i have a stored procedure that I use to return a set of records that need processing. So, I have Processed BIT and LastProcessingRequestDate DATETIME fields in the sql database. Records take somewhere between 1 second and 5 minutes to process on the client side, before the client would Update the record to be Processed=1.

My stored procedure would return a TOP 100 records where Processed = 0 and LastProcessingRequestDate < DATEADD(minute, -10, NOW()), but before it returns each record, it would update LastProcessingRequestDate=NOW(). This would ensure that I don't get any records back that I'm already processing, before I've finished processing them (there is a single thread responsible for keeping a queue full of records to process, from which multiple threads dequeue from).

I'm wondering what the best practice would be to move this scenario over to MongoDB. One thought I had is to create a mongodb function which does something similar, but I've not seen any examples of this type of function. Another would be to simply use the same logic, but handle the updating of LastProcessingRequestDate on the client side. This seems less than ideal to me.

Does anyone have an example of how I could write a javascript function in mongodb to do this? Thanks!

A: 

I really think that some form of "client-side" or "business object / entity / data object" logic is in order here.

The other option is to basically build multiple collections and use those collection for story "to-process" items.

Gates VP
What I'm thinking is getting the document _id's and then doing something like: db.collection.update("{ _id : { $in : [1,2,3] } }", "{ $set : { lastProcessRequestDate : 12345678 } }");This would give the desired effect anyways, just not sure how 'efficient' it is.
Redth
Also, multiple collections wouldn't really solve my issue. I still need a way to flag documents as being "CurrentlyProcessing" so that I don't try and grab the same documents again for processing, before the document is finished processing.
Redth
I see where you are coming from. My only experience with the "$in" clause is that you don't want to make it too long. You're basically asking Mongo to do 100 index look-ups followed by 100 look-ups to set data. In fact, you'd be asking the same thing of MySQL, so I'm not sure that either way is really efficient :(
Gates VP