Adobe Air/Flex filtering large datasets

Question

Hi I have an Air app that downloads a data set of around 100`000 objects and puts the objects in an ArrayCollection.

I would like to apply various filters to the data set in numerous screens throughout the app. I am a little worried about possible performance issues if I make multiple copies of the collection. However, if I don't copy the collection any filters applied will be reflected in all the screens and this is not the behavior I need.

What would be the best way to give multiple views of this large data collection? Has anyone had any experience with this kind of scenario. Any help much appreciated.

Answer 1

You could use a class that acts as a proxy for the original ArrayCollection and the data stored in it. You can set the filter function on the proxy and have the proxy delegate most of its other functionality to the referenced ArrayCollection. You can probably start by subclassing ListCollectionView to do this.

Answer 2

You can have single array with original data but also have different ArrayCollections underlaing on same array. Thus, you can apply any filter to instances of ArrayCollections.

As example:

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml"
    layout="vertical"
    creationComplete="init();">
 <mx:Script>
  <![CDATA[
   import mx.collections.ArrayCollection;
   private var buffer:Array;

   [Bindable]
   private var listData1:ArrayCollection;

   [Bindable]
   private var listData2:ArrayCollection;

   private function init():void
   {
    buffer = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];

    listData1 = new ArrayCollection();
    listData1.source = buffer;

    listData2 = new ArrayCollection();
    listData2.source = buffer;

    listData1.filterFunction = filter1
    listData2.filterFunction = filter2;

    listData1.refresh();
    listData2.refresh();
   }

   private function filter1(item:Object):Boolean
   {
    var i:Number = Number(item);

    if (i >= 5)
    {
     return true;
    }

    return false;
   }

   private function filter2(item:Object):Boolean
   {
    var i:Number = Number(item);

    if (i <= 5)
    {
     return true;
    }

    return false;
   }
  ]]>
 </mx:Script>
 <mx:HBox>
  <mx:List id="list1"
     dataProvider="{listData1}"/>
  <mx:List id="list2"
     dataProvider="{listData2}"/>
 </mx:HBox>
</mx:Application>

Answer 3

I guess there are 2 options:

1. don't worry about performance: even if you have multiple collections, they will all point to the same data objects via references. Just create new ArrayCollections and pass in the objects as an array. You can then apply filters to the individual collections.

2. process the filters eagerly by applying a filter for each view and then copy the result into a new ArrayCollection. Once you filtered a collection, create a new one with as the source filteredCollection.toArray(). The same applies as the above: the collections will contain references to objects, not value copies.