How to implement a job scheduler

Question

Hi,

i would like to implement a job scheduler(from scratch as it is for university purpose)

I want to run certain methods at intervals. the intervals may vary Do you have any suggestion? I really do not know where to start.

Thanks

Answer 1

I personally would start by reading all I can on schedulers, and then browsing the code of some of the open source Operating systems to get a feel for how they solve the problem.

Answer 2

I'd create a job object that had the number of times the job ran and a pointer to the function and then put them in a list and iterate over them like this:

while True
   for job in jobs
       if not job.finish()
          job.run()
          job.num_ran++

then depending on the length of jobs etc you could put in priorities and have two or three jobs list to give preferential treatment or optimal performance.

Answer 3

One suggestion that I have is to have your scheduler "wake up" at the minimum schedule interval, then check to see if any jobs are scheduled to run at that time. The alternatives to this method would be to calculate the next time a job is scheduled to run and wake the scheduler up at that time -- this would be much harder, I think -- or run continuously -- which would be a waste of resources. The idea would be that if jobs can be schedule to run every minute, then your scheduler would "wake up", start any jobs scheduled to run at that time, then go back to sleep until the next "wake up" time. You'll need a data structure for the job that keeps track of the the time interval between successive job runs and the time of the next scheduled run. When the scheduler runs simply run any job whose next time is now or in the past. When a job is run update the next run time based on the last time and the time interval. I'll leave it to you to handle all the corner cases.

Answer 4

Think of how you, as a person, would schedule small tasks that needed to be done at intervals. For example, push one button every minute, and another every 23 seconds. Starting at time 0, write out the time that something happens and what happens.

Don't laugh; MMORPG players do this all the time to figure out optimal action sequences given a set of powers and their recharge times.

Then rewrite it so that you don't have an absolute time axis, instead substituting "wait for x.x seconds" steps.

Now think of a way to figure out these steps directly, without using an absolute time axis. Basically, you have to keep track of the next execution time of each task.

Now write it as computer code. Bingo, a tickless scheduler with no need for a wall clock (i.e. system tick counter).

You can also do it with a tick counter; just figure out at what tick the next execution should happen, and if there's nothing to do right now, figure out how long to sleep.

I'm being vague and incomplete in places, but I hope this gets you started.

Answer 5

I would take a look at Quartz or Quartz.NET (depending on you language). They are both very good and provide source code for you to learn from.

Answer 6

@tvanfosson has a good suggestion.

What I've done in the past is to have a "two-layer" data structure. One is a list of points in time, the second is a list of jobs, one list per point in time (for some minimal granularity, in my case havinga per-second granularity is plenty).

The scheduler can then simply look at the first item in the top-level list and see when the next job(s) need to be started. If that is in the past or now, run each job. If it's in the future, it's trivial to sleep for then - now seconds and simply do the same thing over and over again.

Jobs that needed to be done on a regular basis (all of them, in my case) are responsible for re-scheduling themselves at a suitable interval.

One slight modification would be to have a data structure with pointers to the head and tail of the top-level queue and let that queue be implemented with a double-linked list, that way the typical case (job is at the end of the queue) be implemented efficiently and otherwise you can search from whatever end is closest to the time the job wants to be run.

Answer 7

Begin by listing as many things a scheduler needs to do as you can think of. Here is a "starter for ten":

• submit a job to be run once at a fixed time
• submit a job to be run repeatedly at fixed times
• submit a job to be run repeatedly at fixed intervals
• submit a job to be run at variable intervals
• remove a scheduled job
• change a job's run interval
• report on scheduled jobs
• get the current time
• check to see whether jobs need to be run
• run a scheduled job

Order and prioritze the requirements; note dependencies between features. You will see that in my list (and probably in yours too) the features fall into two broad categories:

1. storing and retrieving scheduler information
2. actually executing jobs

Pick a feature and start to design or coe it, using your favourite programming methodology. You will probably find it easier to start with the storage and retrival features. However, at some point soon you need to incorporate some execution features, to validate your design decisions regarding storage.

One of the fun aspects of testing things like this is the dependency on time. You don't what to have to test 'scheduled job runs on last ay of every month' in real time, but you also don't want to fiddle with the system clock.