views:

146

answers:

2

After some phase of testing of my task scheduler I encountered a deadlock pretty much random. I want to ask some help, especially I want to know if my approach can take to a deadlock or if the problem is elsewhere.
Before starting i'll say that the application is a fullscreen game. (in case it may influence anything)
I'll explain in words how the system is working.

The task Scheduler

1) Schedule 2 per CPU tasks at the start of each frame. (by scheduling i mean setting the private WaitHandle of the task to set allowing the task to do some work)

Here's a short code that summarize what the system is doing with each of the two task.

                Scheduled++;
                InternalLock.Reset();

After all the two tasks are scheduled they are made start by setting the private WaitHandle.

2) Wait for all the tasks to be complete. The wait is done by waiting for the internal WaitHandle that each task has to be signaled. (WaitOne() on each task)

Here's the code of the wait :

    if (Attese.Length > 0)
    {
        TmpIsSucceded = false;
        for (int i = 0; i < Attese.Length; i++)
        {
            WaitHandle actual = Attese[i].Wait;
            do
            {
                TmpIsSucceded = actual.WaitOne(150);
                if (!TmpIsSucceded)
                    EnginesManager.ProcessMessages();
            } while (!TmpIsSucceded);
        }
    }

The task

1) Is never closed until the end of the game.
2) Has 2 internal WaitHandle. One private that tells the task when there is work for him. One internal that is signaled when the task has ended its work. (the one that the task scheduler is waiting for)
3) When the task is complete by itself start another task available in a synchronized (by lock()) queue of the task scheduler (in the same way by setting the private waithandle of that task).

this is the main loop of the task :

       private void CoreThread()
       {
            while (_active)
            {
                PrivateLock.WaitOne(-1, false);
                while (Scheduled > 0)
                {
                    if (OnThreadExecute != null)
                        OnThreadExecute(this, null);
                    Scheduled--;
                    if (Scheduled == 0)
                    {
                        PrivateLock.Reset();
                        if (OnThreadEnd != null)
                            OnThreadEnd(this, null);
                        InternalLock.Set();
                    }
                }
            }
        }

InternalLock and PrivateLock are the two waitHandles. Please note that the InternalLock waithandle is only setted in this code. There is NO OTHER PLACE where InternalLock or PrivateLock are setted. (except for the code I posted)

When the deadlock happens the task scheduler is waiting for all the tasks to be completed, but one of the task never set the InternalLock waithandle. The "blocked" task is stopped at the "PrivateLock.WaitOne(-1, false);" line when the deadlock occur.

Anybody some clue about this deadlock?

Edit :

    internal void StartSchedule()
    {
        for (int i = 0; i < Tasks.Length; i++)
        {
            if (Tasks[i].Schedule())
                QTasks.Enqueue(Tasks[i]);
        }
        StartThreadAvailable();
    }

    private void StartThreadAvailable()
    {
        TempExecList.Clear();
        for (int i = 0; i < NThread; i++)
        {
            if (QTasks.Count > 0)
                TempExecList.Add(QTasks.Dequeue());
        }
        Int32 count = TempExecList.Count;
        for (int i = 0; i < count; i++)
            TempExecList[i].StartThread();
    }

    internal void StartThread()
    {
        PrivateLock.Set();
    }

here's the code where the Set() of the Private handle is called as asked.

Schedule() return always true in this case. (it only add 1 to the scheduled variable of the task and reset the InternalLock)

EDIT 2 :

Here's the code of the 2 classes as asked :

http://pastebin.com/m225f839e (GameTask)

http://pastebin.com/m389629cd (TaskScheduler)

+1  A: 

The implementation of Scheduled is not shown, but it looks to me like there could be a race between incrementers and decrementers of this variable. I think you may need to use InterlockedDecrement here, And I'd also be more comfortable if there was no room between the test for Scheduled > 0 and the test for Scheduled == 0.

more like this

PrivateLock.WaitOne(-1, false);
while (true)
{
    // fetch Sched, so the whole loop sees a single value
    int iSched = Scheduled--; // implementation should be Interlocked.Decrement()
    if (iSched <= 0)
    {
       if (iSched < 0)
       {
          // should never get here, throw exception?
       }
       PrivateLock.Reset();                
       if (OnThreadEnd != null)                
           OnThreadEnd(this, null);                
       InternalLock.Set();                
       break; // break out of while
    }

    if (OnThreadExecute != null)              
       OnThreadExecute(this, null);
}
John Knoeller
I added the whole code, and in the meantime i'll try your advice.
feal87
A: 

I found out the problem. It was a really stupid one...

I'll explain so anyone else having this problem can make use of the time i losed. XD Anyway very thanks to John Knoeller that helped me to narrow the problem with its insight. :D

Let's take a look at the Main Thread. The InternalLock.Set() set out the block on the task scheduler thread and says "go ahead". Let's say that the task in the task scheduler are only 1. And imagine this POSSIBLE situation

1) First step

TASK SCHEDULER - SCHEDULE
TASK 1 - WAIT

2) Second step

TASK SCHEDULER - WAITING
TASK 1 - WORK

3) Third step

TASK SCHEDULER - WAITING
TASK 1 - InternalLock.Set();

4) Fourth Step

TASK SCHEDULER - SCHEDULE
TASK 1 - |while (Scheduled > 0)|

At the fourth step cause the SCHEDULE in the main thread incremented the scheduled variable. This way the while did not end causing all the disturbances in the code (and the deadlock). I fixed by simply adding a break after the InternalLock.Set(); Now there are no problem.

(to the people who said that i was accessing the state without synchronizing. Please note that the Schedule function is called only once before even ANY of the thread are given work, so it does not matter if it is synchronized or not. The problem was a really stupid one. My bad. :D)

Thanks!!!

feal87