ActionScript: Array Limitations/ How to handle lots of Data?

Question

I am running into trouble with a multidimensional array.

This works well with up to about 2000 "sets"

var datasets:Array = new Array(
    // one "set"
    new Array( 
        new Array( time:int, x:int, y:int, type:int ), // one "point"
                /* ... */
        /* ~70 more points */
    ),

    /* ... */
    /* ~3500 more sets */
)

Is there a better way to handle that many Items?

Maybe going for XML datafiles is a more convenient approach?

Answer 1

First of all, a stylistic suggestion. The following:

new Array(x, y, z)

can be expressed by the shortcut syntax:

[x, y, z]

This syntax makes it easier to express nested arrays, eg:

[x, y, [z1, z2, z3]]

Now that we've got that out of the way... there really a "right" answer for your question but one thing that occurs to me is that you will have an awful lot of nested arrays in play this way. If each "set" has a fixed number of items, what about putting them in one long array and then just doing the math to determine how to retrieve the value for a given x,y position within the array. Here's what I mean, take this example of three "sets" of three values:

[[a1, a2, a3], [b1, b2, b3], [c1, c2, c3]]

Instead of nesting them you could simply store them in a flat manner like this:

[a1, a2, a3, b1, b2, b3, c1, c2, c3]

Then when you want to retrieve a value for set 2 (zero-based it would be set "1") and value 3 (zero-based it would be value "2", you would simply access array[setIndex * 3 + valueIndex]

The other other thing I will say is that the structure should really be driven by the means with which you need to parse and scan through it. If you provide more information on those requirements, we could provide a better answer for you.

Answer 2

You could use Objects instead.

Multidimensional Arrays don't really exists in AS3 ( they do but you don't have access to most of the Array methods ).

You can't really compare having your data in an Array and having it on a data file though. It really depends on how often you need to access it, how you process it , compute it etc...

In any case you could always save your data as a JSON string, rather than XML.

var datasets:Array = [
    // one "set"
    // i leave the types as in your example but you 
    // would , of course , put your values instead.
    { 
        { time:int, x:int, y:int, type:int }, // one "point"
                /* ... */
        /* ~70 more points */
    },

    /* ... */
    /* ~3500 more sets */
]

Answer 3

Creating Arrays is surprisingly performance-intensive in Flex 3. Especially if you're going to nest a huge bunch of them like that, you're going to be way better off with a different solution. A couple of suggestions depending on what you're doing with the data:

1. If you can get by storing some of it in a file or database, do that. That's what files and databases are for, after all: storing lots of data.
2. If you really need to keep everything in memory, try some more advanced collections. See if you can find a more efficient way to arrange your data using ArrayCollection or Dictionary.

Either way, you'll probably want to make a custom collectionish object for interacting with your file-db or more-advanced-collections solution. I know it's lame to do that extra work yourself, but with that much data you're really pushing past what Arrays are meant for and a custom solution is going to be your best bet.

Answer 4

If you are running against Flash Player 10+ you can use Vector.<T> instead, is significantly faster than Array for reads.

On top of that (though unrelated to FP10), I'd move your time/x/y/type to a CustomPoint class:

var dataSets : Vector.<Vector.<CustomPoint>> = Vector.<Vector.<CustomPoint>>([
    Vector.<CustomPoint>([
        new CustomPoint(time, x, y, type),
        new CustomPoint(time, x, y, type)
    ]),

    // ...
]);