views:

195

answers:

3

I'm working on a problem where i need to process multi dimensional data in memory using C#. My requirement resemble OLAP cubes but are not as complex. for example i don't need calculations or aggregation or stuff like that. I basically would like to reference data using multidimensional keys. for example:

var key = new Key();
key["Dim1"] = "DimValue1";
key["Dim2"] = "DimValue2";
key["Time"] = 1999;
DataSet[key] = 4.43434m;

And it would allow me to iterate on the values or slices of the dataset. Have you come across such a library in C#?

A: 

You could create a dictionary where the key type is a struct that you declare. That doesn't give you automatic iteration of slices, although you could achieve it by filtering.

recursive
+1  A: 

This may not meet your need, but I've found a simple way to deal with multi-key datasets is to create an object which contains all your 'key' fields and 'value' keys (as many of each as you need) and then create Lookup expressions for each of your keys.

For example:

class MyData
{
    // Your keys
    public string Dim1;
    public string Dim2;
    public string Time;

    // Your values
    public string Value;
}

would be 'indexed' and retrieved like this:

// add all your data to a list or collection
var data = new List<MyData>();

// this provides the entry point to our dataset
var lookupDim1 = data.ToLookup(d => d.Dim1);
var lookupDim2 = data.ToLookup(d => d.Dim2);
var lookupTime = data.ToLookup(d => d.Time);

// sample retrievals
IEnumerable<MyData> sampleData1 = lookupDim1["DimValue1"];
var sampleData2 = lookupDim2["DimValue2"].Intersect( lookupTime["1999"] );
Kirk Broadhurst
This will come in handy, thanks for the idea.
Richard Hein
Very nice idea. I didn't know about lookups. But i think this would perform very slow. I think at best O(n) for retrieval if not O(n^m) where m is the number of keys.
Delucia
How big is your data set? It ran very quickly for me when using over 5 million records, where each was up to around 4KB. I don't profess to know how the Lookup LINQ works but it's very quick.
Kirk Broadhurst
A: 

I think a key/value store like MongoDB and Redis is close to what I need. However I'm not 100% sure. Since I don't care about persistence, in memory story like Redis is more suitable.

Delucia