(Design Idea) Best Practise to wake up multiple process to do workflow

Question

Hi all

I 'm designing multiple process workflow engine. I deploy 4 services on Websphere and finding idea how to wake up them. Current solution is

1 Quartz will wake up process every minute , if last process already running, this process will not run duplicate ( use java's static variable to control) .

2 Process query maximum to 200 record.

2 Process will create concurrence thread maximum to 50 to process job.

3 Process sleep 5 second and check if there are free seat to process more.

4 If there are no record to process, process will fetch data next 200 record

5 if there are no data , process will end , wait quartz to wake up later

What you think about it ? Do you have other idea?

Answer 1

The phrase "best practice" is often used as an excuse for not thinking about things.

But I'd say that any solution involving sleeping for a number of seconds is not best practice.

Answer 2

I'm not sure how your system relates to workflow. The term 'Workflow' generally indicates that a system is managing tasks that require human input. You are describing a simple processing engine that takes its work from a database table. For that kind of work it will probably work OK, but isn't really 'best practice'.

I would suggest two changes.

1. Your application should have an internal thread pool to process jobs. Creating a new thread to process each job isn't necessary. Make the thread pool configurable so you can increase or decrease its size quickly and easily.

2. Your application should have a queue interface (eg JMS) to receive jobs rather than repeatedly reading a database table.