tags:

views:

61

answers:

2

I'm trying to walk through a bunch of items, each item has an array of List<> objects that I want to convert to an array of arrays.. Here's the code to do that:

foreach (IngredientNode i in _snapshot._ingredientMap.Values)
{
   for (int c = 0; c < NUM_TAGS; c++)
   {
      if (i.RecipesByTag[c] == null) continue;
      i.RecipesByTag[c] = i.RecipesByTag[c].ToArray<RecipeNode>();
   } <--- EXCEPTION
}

RecipesByTag has a static type of IEnumerable<RecipeNode>[]. However, its dynamic type is List<RecipeNode>[]. I want to go through each one of those and convert the dynamic type of RecopeNode[]. Under the debugger, this works and i.RecipesByTag gets converted. However, the last curly brace then throws the exception:

Attempted to access an element as a type incompatible with the array.

I have a feeling there's some sort of stack corruption going on. Can someone explain what's happening at a technical level here? Thanks!

Mike

+3  A: 

You shouldn't have to specify the type argument for the ToArray method, it should be inferred from it's usage, if you are using strongly typed collections. This is a generic type casting problem. You are trying to put elements in an array of some incompatible type.

Your problem should boil to this (these arrays are covariant):

object[] obj = new string[1];
obj[0] = 5; // compiles fine, yields runtime error

Now, the same thing, with different types (these arrays are also covariant):

IEnumerable<int>[] x = new List<int>[1];
x[0] = new int[1]; // compiles fine, yields runtime error

It should be obvious why the type system doesn't like this. Basically, you look at it as if it was an array of IEnumerable<int> but it's actually an array of List<int>. You can not put an unrelated type array of int, into that array.

I believe Eric Lippert explains this very well on his blog.

John Leidegren
.NET Runtime 2.0, C# 3.5 compiler.. BTW, the above will result in the same error.
Mike
As far as I can tell, it's a runtime bug, err limitation. If you have an array of enumerables, you can't set one of the elements in that array to another array. The compiler allows it because an array is an enumerable type, so the static types match up. However, when an array spontaneously becomes multidimentional, the universe implodes and the whole memory structure is corrupted. The runtime isn't smart enough to move everything over in memory.. I figured out a work-around, I'll post the answer.
Mike
It's not a runtime bug. It's the covariant type rules that apply to arrays. You can actually use arrays such that you don't get a compiler error but a runtime error.
John Leidegren
I never said it was a bug, I said limitation :) It makes sense once I think about how arrays are actually laid out in memory. However, it does show a bit about how the compiler works. When I say foo[5] = someArray, then it expects foo to be a multidimentional array. It's not smart enough to just make foo[5] a pointer to another array somewhere on the heap.
Mike
Fine, "As far as I can tell, it's a runtime bug, err limitation." I read that as, it's a bug, but did not even notice the err limitation part. The problem is not in memory allocation but has to do how the runtime reasons about the type information. I've updated my answer to reflect this. You should be able to create a isolated test case, if you so wish.
John Leidegren
The object/string example above isn't quite valid as a comparison. Since obj[0] has a dynamic type of string, and you're assigning a value with a static type of int, you'll get a runtime error (a cast exception?).In my case, the static type was Enumerable<>. Both the existing value, which had a dynamic type of List<> and the value I was assigning, which had a dynamic type of Array, can both be implicitly upcast to Enumerable<>. It wasn't a casting issue, it's a limitation with the runtime where-as the runtime cannot create an array of pointers to other arrays. So I think anyway..
Mike
I'll show you, it's gonna take me some time though to work this out.
John Leidegren
Thanks for your example, I think I have a pretty clear idea of what's going on.. Anyway, this is purely for my intellectual curiosity, I think I'm gonna redesign this data structure to eliminate the need to convert lists to arrays.Basically what I need is an way to dynamically add a bunch of vertices to nodes in a graph, but when I'm done, convert them to arrays. The reason I want to convert them to arrays is there might seriously be hundreds of thousands of vertices and I have a feeling that using arrays will take up wayyyyy less memory; and creating the graph doesn't have to be fast.
Mike
@Mike - I think that's a good idea. But memory wise a `List<T>` is just 12 (16 on 64-bit) bytes bigger than an array. Unless you got trillions of `List<T>`s you don't need to work with arrays directly. I find them somewhat akward to work with in a managed environment (like generics more). If you need to iterate through big arrays you can always used unsafe/fixed to minimize the index checking overhead, or you can roll your own hybrid list which does both. I'd hold off with that though until you've actually hit a performance issue which is raw CPU speed related.
John Leidegren
Yeah, I didn't know List<> was that compact.. I figured it had various instance methods in memory and stuff.. You're right about optimizing prematurely, it might be best to simplify the code and then test with some super-large datasets to see where the real problems are.
Mike
@Mike - I've cleaned up the answer a bit, removed the bad, in with the good. I'm not an expert on this co-/contra-/invariance thing but it's these rules that govern why this isn't allowed.
John Leidegren
I'm thinking an array actually might be the best choice here because reading by index needs to be very fast. I'm accessing this data through a simulated annealing algorithm, in other words, I'll be accessing these arrays several hundred thousand times per second.. However, once it's coded I'll comment out my List->Array conversion and see what the actual speed difference is.. As you mentioned, I could use some unsafe code as well..
Mike
A: 

Okay I think I figured out what's going on.. I have an array of Enumerables. At first, this was an array of pointers to list objects. I instead made this an array of other arrays, in other words I converted my array into a multidimentional array. Since a multidimentional array is consecutive in memory (as opposed to an array of pointers to other arrays), doing this completely corrupted the array in memory. At least that's my theory.

What I did is completely recreate i.RecipesByTag from scratch. Something like this:

List<RecipeNode[]> temp = new List<RecipeNode[]>();

for (int c = 0; c < NUM_TAGS; c++)
{
   RecipeNode[] nodes = (i.RecipesByTag[c] == null) ? null : i.RecipesByTag[c].ToArray<RecipeNode>();
   temp.Add(nodes);
}

i.RecipesByTag = temp.ToArray();

This works perfectly.

Mike