DataTable internal index is corrupted

Question

I am working with a .NET WinForms app in C#, running against the 3.5 .NET framework. In this app, I am setting the .Expression member of a DataColumn in a DataTable, like so:

DataColumn column = dtData.Columns["TestColumn"];
column.Expression = ["some expression"];

The 2nd line, where I actually set Expression, will sometimes result in the following exception:

FileName=
LineNumber=0
Source=System.Data
TargetSite=Int32 RBInsert(Int32, Int32, Int32, Int32, Boolean)
System.InvalidOperationException: DataTable internal index is corrupted: '5'.
   at System.Data.RBTree`1.RBInsert(Int32 root_id, Int32 x_id, Int32 mainTreeNodeID, Int32 position, Boolean append)
   at System.Data.RBTree`1.RBInsert(Int32 root_id, Int32 x_id, Int32 mainTreeNodeID, Int32 position, Boolean append)
   at System.Data.Index.InitRecords(IFilter filter)
   at System.Data.Index.Reset()
   at System.Data.DataTable.ResetInternalIndexes(DataColumn column)
   at System.Data.DataTable.EvaluateExpressions(DataColumn column)
   at System.Data.DataColumn.set_Expression(String value)

There is no perceptible rhyme or reason as to when the error will occur; in loading the same data set, it may work fine but then reloading it will fail, and vice versa. This leads me to think it is related to a race condition, where another write operation is occurring on the DataTable as I'm trying to modify one of its columns. However, the code relating to DataTables is not multi-threaded and runs only on the UI thread.

I have searched the web and Microsoft forums, and there is much discussion and confusion over this issue. Back when the issue was first reported in 2006, the thought was that it was an flaw in the .NET framework, and there were some supposed hotfixes released that were presumably rolled into later versions of the .NET framework. However, people have reported mixed results in applying those hotfixes, which are no longer applicable to the current framework.

Another prevailing theory is that there are operations on the DataTable which, though seemingly innocuous, are actually write operations. For example, creating a new DataView based on a DataTable is actually a write operation on the table itself, because it creates an internal index in the DataTable for later reference. These write operations are not thread-safe, so it sometimes happens that a race condition leads to an unthread-safe write coinciding with our access of the DataTable. This, in turn, causes the internal index of the DataTable to become corrupted, leading to the exception.

I have tried putting lock blocks around each DataView creation in the code, but, as I mentioned before, code utilizing the DataTable is not threaded, and the lock's had no effect, in any case.

Has anyone seen this and successfully solved / worked around it?

Answer 1

can't you just use:

dtData.Columns.Add("TestColumn", typeof(Decimal), "Price * Quantity");

Answer 2

No, unfortunately I can not. Loading the DataTable has already occurred by the time I get a hold of it to apply an Expression to one of its DataColumn's. I could remove the column and then re-add it using the code you suggested, but is there a particular reason why that would solve the internal index is corrupted problem?

Answer 3

You mention "not threadsafe". Are you manipulating the object from different threads? If so then that might very well be the reason for the corruption.

Answer 4

Personally, this particular bug has been my nemesis for 3 weeks in various fashions. I have solved it in one part of my code base and it shows up elsewhere (I believe I finally squashed it tonight). The exception info is rather unhelpful, and a way to force a reindex would have been a nice feature given the lack of MS to solve the problem.

I wouldn't look for MS's hotfix -- they have a KB article on it, then redirect you to an ASP.Net fix that is completely unrelated.

Ok - enough complaining. Let's see what has actually helped me in resolving this particular issue in the various places I've encountered it:

• Avoid using Default Views, and modifying the Default View if possible. Btw, .Net 2.0 has a number of reader/writer locks on creating views, so they are not the issue they were pre 2.0.
• Call AcceptChanges() where possible.
• Be careful about .Select(expression), since there is no reader/writer lock in this code -- and it is the only place (at least according to a person on the usenet so take it w/ a grain of salt -- however, this is very similar to your issue -- so using Mutexes may help)
• Set AllowDBNull to the column in question (questionable value, but reported on the usenet -- I've used it only in places where it makes sense)
• Make sure that you are not setting null (C#)/Nothing (VB) to a DataRow field. Use DBNull.Value instead of null. In your case you may wish to check that the field is not null, the expression syntax does supports the IsNull(val, alt_val) operator.

• This has probably helped me the most (absurd as it sounds): If a value is not changing, don't assign it. So in your case use this instead of your outright assignment:

  if (column.Expression != "some expression") column.Expression = "some expression";

(I removed the square brackets, not sure why they were there).

Answer 5

My understanding, from long and painful haggling over this problem, is that it's an artifact of non-thread-safe write operations, which usually you didn't know you were making.

In my case, the culprit seemed to be the BindingSource. I found that I needed to suspend binding, perform whatever operation I was trying, and then resume binding when I was done, and the problem went away. This was 18 months ago, so I'm not clear on the details anymore, but I remember getting the impression that the BindingSource is doing some kind of operation on its own thread. (This makes less sense to me now than it did at the time.)

Another potential source of trouble is the DataTable's RowChanging event. If you do something that modifies the table in that event handler, expect bad things.

Answer 6

We are seeing the same error. The error happens often in the same place in our code. It appears to be related to setting row values when we should not. Our code is not running multi-threaded; everything happens on the UI thread.

I don't have an answer yet, but I wanted to add the RBTree source code, in case someone can make sense out of it:

   private int RBInsert(int root_id, int x_id, int mainTreeNodeID, int position, bool append)
    {
        this._version++;
        int nodeId = 0;
        int num = (root_id == 0) ? this.root : root_id;
        if ((this._accessMethod != TreeAccessMethod.KEY_SEARCH_AND_INDEX) || append)
        { /* SNIP */ }
        else
        {
            while (num != 0)
            {
                this.IncreaseSize(num);
                nodeId = num;
                int num6 = (root_id == 0) ? this.CompareNode(this.Key(x_id), this.Key(num)) : this.CompareSateliteTreeNode(this.Key(x_id), this.Key(num));
                if (num6 < 0)
                {
                    num = this.Left(num);
                }
                else
                {
                    if (num6 > 0)
                    {
                        num = this.Right(num);
                        continue;
                    }
                    if (root_id != 0)
                    {
                        // The exception occurs here:
                        throw ExceptionBuilder.InternalRBTreeError(RBTreeError.InvalidStateinInsert);
                    }
                    /* SNIP the rest of a long function */

It so happens that RBTreeError is an enumeration, and RBTreeError.InvalidStateinInsert = 5.