views:

377

answers:

13

I have a JavaScript array of houses like so,

{
'homes' : 
[{
"home_id"         : "1",
"address"         : "321 Main St",
"city"            : "Dallas",
"state"           : "TX",
"zip"             : "75201",
"price"           : "925",
"sqft"            : "1100",
"year_built"      : "2008",
"account_type_id" : "2",
"num_of_beds"     : "2",
"num_of_baths"    : "2.0",
"geolat"          : "32.779625",
"geolng"          : "-96.786064",
"photo_id"        : "14",
"photo_url_dir"   : "\/home_photos\/thumbnail\/2009\/06\/10\/"
},
..........
]}

I want to provide 3 different search methods.

How can I return a subset of this homes area array that has:

  • The price between X and Y
  • The bathrooms >= Z
  • The # of bedrooms == A or == B or == C

For example, how would I create the psuedo code like:

homes.filter {price >= 150000, price <= 400000, bathrooms >= 2.5, bedrooms == 1 | bedrooms == 3}
+2  A: 

Take a look at the $.grep function in the jQuery library. Even if you don't use the library, you can look at the code and see how they accomplished the task you are trying to do.

EDIT: Here's some code:

 function filter(houses, filters) // pass it filters object like the one in @J-P's answer
 {
      retArr = $.grep(houses, 
           function(house)
           {
                for(var filter in filters)
                {
                     function test = filters[filter];
                     if(!test(house)) return false;
                }
                return true;
           });

      return retArr;
 }
Ryan Lynch
+1 jQuery is designed for this - this the word "query" in the title.
David Lively
The $.grep looks to work on simply array fine but how would I use this for my example array type above?
GeorgeG
I'm reminded here of the quote "If you only have a hammer, you tend to see every problem as a nail." - jQuery and $.grep don't seem relevant to this question. If I'm wrong feel free to correct me. This question seems to be more about iterating over a data structure.
artlung
I mentioned $.grep because the questioner wanted to be able to filter an array based on some conditional statements and get subset of the original array, and using jQuery makes the amount of code you need to write a bit smaller. Using $.grep you just have to write the conditional statements into a callback function and have it return their combined result. I'll modify my answer to include some code, maybe that will make my reasoning more clear.
Ryan Lynch
I'd be curious to see this working, the examples I'm seeing using grep this way are typically on strings or integers, not objects with multiple fields. Interesting approach.
artlung
That's what i like about a well written library like jQuery, it provides you with a great extensible toolkit. You hammer and nail analogy isn't too far off unfortunately, but I find that for every problem like this there are several more in your code where using jQuery or a similar library will simplify the code you have to write to solve it. This would be even better if you combined it with the use of eval in your solution.
Ryan Lynch
How would I call this. Would I do something like ... -> "homes.filter(price >= 150000 });" ???
GeorgeG
I decided instead of editing my current answer with a better solution I would just post another answer. Check that one out instead of this one.
Ryan Lynch
A: 

I am assuming you want the syntax sugar of being able to call filter direction on the homes array to generate a new array of only the filter accepted results.

Use a object prototype to define a filter function on the array object that accepts a hash of parameter filters. Then you can easily use that hash to generate and return a new array of homes that match the desired property.

Martin Dale Lyness
I really don't care what the syntax sugar looks like, as long as it generates the subset array.
GeorgeG
+5  A: 

Take a look at JSONPath http://code.google.com/p/jsonpath/

Something like jsonPath(json, "$..homes[?(@.price<400000)]").toJSONString() should work.

JAQL also looks interesting for filtering JSON. http://www.jaql.org/

Kenan
JSONPath looks really interesting. Thanks for mentioning it!
artlung
+5  A: 

Here you go:

var filteredArray = filter(myBigObject.homes, {

    price: function(value) {
        value = parseFloat(value);
        return value >= 150000 && value <= 400000;
    },

    num_of_baths: function(value) {
        value = parseFloat(value);
        return value >= 2.5;
    },

    num_of_beds: function(value) {
        value = parseFloat(value);
        return value === 1 || value === 3;
    }

});

And the filter function:

function filter( array, filters ) {

    var ret = [],
        i = 0, l = array.length,
        filter;

    all: for ( ; i < l; ++i ) {

        for ( filter in filters ) {
            if ( !filters[filter](array[i][filter]) ) {
                continue all;
            }
        }

        ret[ret.length] = array[i];

    }

    return ret;

}
J-P
good solution, though i think you forgot to get rid of `matches`
meder
@meder, yup, you're right. Edited :)
J-P
I don't understand how this would work since you have hardcoded in the values I want to filter on. I want to be able to do something similar to --> "homes.filter {price >= A, price <= B, bathrooms >= C, bedrooms == D | bedrooms == E}"
GeorgeG
Do you mind showing me how I would call this filter function. That might clear up some of my confusion
GeorgeG
@GeorgeG, use it exactly as I have. The second parameter to `filter` is an object, with property-names corresponding to those you want to test, and functions that do the testing. The functions return true when the test is passed, and false when it doesn't.
J-P
+2  A: 
Array.prototype.select = function(filter) {

    if (!filter) return this;

    var result = [], item = null;
    for (var i = 0; i < this.length; i++) {
     item = this[i];
     if (filter(item)) {
      result.push(item);
     }
    }

    return result;
}

function filterHomes(homes) {
    var a = 1, b = 2, c = 3, x = 4, y = 5, z = 6;
    return homes.select(function(item) {
     return between(item.price, x, y) && item.num_of_baths >= z && inArray(item.num_of_beds, [a, b, c]);
    });
}

function between(value, min, max) {
    return value >= min && value <= max;
}

function inArray(value, values) {
    for (var i = 0; i < values.length; i++) {
     if (value === values[i]) return true;
    }
    return false;
}
ChaosPandion
That's an interesting solution, extending the Array.prototype to include a filter function. Nice.
Ryan Lynch
Yeah this is exactly what would suggest.
Martin Dale Lyness
How would this be of benefit since you are hardcoding in my filter to use a global JS variables
GeorgeG
I am not really sure what you mean. The variables become part of a closure in this context. The point of this is to extend the Array object to allow you to filter its contents however you wish.
ChaosPandion
@ChaoesPandion, Would you mind showing me how to call your code. That might clear up my confusion
GeorgeG
Just look at the filterHomes function to see how to use it.
ChaosPandion
+2  A: 

I like this question so I'm taking a shot at it. How about a "chained" style of code, where the object you have returns itself, a la some of the JavaScript DOM frameworks.

I'm calling your Object MyObj:

MyObj.filter('home.price >= 150000')
     .filter('home.price <= 400000')
     .filter('home.num_of_baths >= 2.5')
     .filter('home.num_of_beds == 1 || home.bedrooms == 3');

And here's the source code, this example works.

var MyObj = {
    filter : function(rule_expression) {
     var tmpHomes = [];
     var home = {};
     for(var i=0;i<this.homes.length;i++) {
      home = this.homes[i];
      if (eval(rule_expression)) {
       tmpHomes.push(home);
      }
     }
     this.homes = tmpHomes;
     return this;
    },
    homes: [
    {
     "home_id"         : 1,
     "address"         : "321 Main St",
     "city"            : "Dallas",
     "state"           : "TX",
     "zip"             : "75201",
     "price"           : 300000,
     "sqft"            : 1100,
     "year_built"      : 2008,
     "account_type_id" : 2,
     "num_of_beds"     : 1,
     "num_of_baths"    : 2.5,
     "geolat"          : 32.779625,
     "geolng"          : -96.786064,
     "photo_id"        : "14",
     "photo_url_dir"   : "\/home_photos\/thumbnail\/2009\/06\/10\/foo.jpg"
    },
    {
     "home_id"         : 2,
     "address"         : "555 Hello World Way",
     "city"            : "Dallas",
     "state"           : "TX",
     "zip"             : "75201",
     "price"           : 200000,
     "sqft"            : 900,
     "year_built"      : 1999,
     "account_type_id" : 2,
     "num_of_beds"     : 1,
     "num_of_baths"    : 1.0,
     "geolat"          : 32.779625,
     "geolng"          : -96.786064,
     "photo_id"        : "14",
     "photo_url_dir"   : "\/home_photos\/thumbnail\/2009\/06\/10\/foo.jpg"
    },
    {
     "home_id"         : 3,
     "address"         : "989 Foo St",
     "city"            : "Dallas",
     "state"           : "TX",
     "zip"             : "75201",
     "price"           : 80000,
     "sqft"            : 1100,
     "year_built"      : 2003,
     "account_type_id" : 2,
     "num_of_beds"     : 3,
     "num_of_baths"    : 3,
     "geolat"          : 32.779625,
     "geolng"          : -96.786064,
     "photo_id"        : "14",
     "photo_url_dir"   : "\/home_photos\/thumbnail\/2009\/06\/10\/foo.jpg"
    },
    {
     "home_id"         : 4,
     "address"         : "1560 Baz Rd",
     "city"            : "Dallas",
     "state"           : "TX",
     "zip"             : "75201",
     "price"           : 100000,
     "sqft"            : 1100,
     "year_built"      : 2008,
     "account_type_id" : 2,
     "num_of_beds"     : 3,
     "num_of_baths"    : 1.5,
     "geolat"          : 32.779625,
     "geolng"          : -96.786064,
     "photo_id"        : "14",
     "photo_url_dir"   : "\/home_photos\/thumbnail\/2009\/06\/10\/foo.jpg"
    }
    ]
};
artlung
Downvoted for working code!
artlung
Another good solution, it certainly looks pretty close to the psuedo code in the question.
Ryan Lynch
@Ryan Lynch, yeah, I was pleased that my off-the-cuff code almost worked. Then I did testing and refined it. The JSONPath though, looks really useful for this too. Best of luck in your project!
artlung
I like the answer although I am a little leery of eval.
ChaosPandion
I agree with the being leery about eval, but I think in this case since you are trying to do SQL like queries on some JSON it creates a fairly elegant solution. I combined this response with extending the Array Prototype in another answer, check it out.
Ryan Lynch
+8  A: 

Javascript 1.6 (FF, Webkit-based) has built-in Array.filter function, so there's no need to reinvent the wheel.

result = homes.
  filter(function(p) { return p.price >= 150000 }).
  filter(function(p) { return p.price <= 400000 }).
  filter(function(p) { return p.bathrooms >= 2.5 }) etc

for a msie fallback see the page linked above.

stereofrog
What version of Firefox and Website (Chrome/Safari). I read the llink above but there was no mention of IE that I saw in the overview
GeorgeG
http://en.wikipedia.org/wiki/Javascript#Versions
stereofrog
I think that the more powerful way of doing this is to wrap the Array.prototype.filter function in a separate function so that you can evaluate the test statement in the scope of the element being tested using either apply or eval within a with block. That way you can write the test functions in a more generic way so that they could be used on any object or any key contained in the object. Check out my second solution to see what I mean.
Ryan Lynch
@Ryan: of course, there are more effective ways, my code is just an example of how to use Array.filter
stereofrog
Appreciated, @stereofrog. Actually I wish I had seen your post earlier. I "discovered" the Array.prototype.filter method by accident when I tried to extend the Array.prototype by writing a function with the same name. You could have saved me 5 minutes of debugging and a facepalm.
Ryan Lynch
+1  A: 

The Underscore.js library works well for these tasks. It uses native JavaScript commands when they are present. I've managed to get of many iterative loops using this library.

Bonus, the latest version is chainable, like jQuery.

Nosredna
A: 
o = ({
'homes' : 
[{
    "home_id"         : "1",
    "address"         : "321 Main St",
    "city"            : "Dallas",
    "state"           : "TX",
    "zip"             : "75201",
    "price"           : "20000",
    "sqft"            : "1100",
    "year_built"      : "2008",
    "account_type_id" : "2",
    "num_of_beds"     : "3",
    "num_of_baths"    : "3.0",
    "geolat"          : "32.779625",
    "geolng"          : "-96.786064",
    "photo_id"        : "14",
    "photo_url_dir"   : "\/home_photos\/thumbnail\/2009\/06\/10\/"
}
]})

Array.prototype.filterBy = function( by ) {
    outer: for ( 
    var i = this.length,
        ret = {},
        obj; 
        i--; 
    ) 
    {
    obj = this[i];
    for ( var prop in obj ) {

        if ( !(prop in by) ) continue

        if ( by[prop](obj[prop]) ) {
     ret[prop] = obj[prop]
        }

    }
    }

    return ret;
}

var result = o.homes.filterBy({
    price:function(price) {
        price = parseFloat(price)
        return price >= 15000 && price <=40000
    },
    num_of_baths:function(bathroom){
        bathroom = parseFloat(bathroom)
        return bathroom > 2.5
    },
    num_of_beds:function(bedroom){
        bedroom = parseFloat(bedroom)
        return bedroom === 1 || bedroom === 3
    }
});

for ( var p in result ) alert(p + '=' + result[p])
meder
+1  A: 

I've revised this answer several times, and people can look at the wiki if there interested in the revisions, but this is the final solution I've come up with. It's very similar to many of the others posted here, the key difference being that I've extended the Array.prototype wrapped the Array.prototype.filter function such that the test statements are evaluated in the scope of the array element instead of the array.

The main advantage I see with this solution rather than simply using the filter method directly is that it allows you to write generic tests that test the value of this[key]. Then you can create a generic form elements associated with a specific tests, and use metadata from the objects to be filtered (houses in this case) to associate specific instances of the form elements with keys. This not only makes the code more reusable, but I think it makes actually constructing the queries programmatically a bit more straightfoward. Also, since you can define generic form elements (say a multi-select for distinct values, or a set of dropdowns for a range), you could also create a more powerful query interface where additional form elements could be injected dynamically allowing users to create complex and customized queries. Here is what the code breaks down into:

Before doing anything you should first check to see if the filter function exists and extend the Array.prototype if it doesn't as mentioned in other solutions. Then extend the Array.prototype to wrap the filter function. I've made it so that the arguments are optional in case someone wants to employ a test function that doesn't take any for some reason, I've also tried to include errors to help you implement the code:

Array.prototype.filterBy = function(testFunc, args)
{
 if(args == null || typeof args != 'object') args = [args];
 if(!testFunc || typeof testFunc != 'function') throw new TypeError('argument 0 must be defined and a function');
 return this.filter(function(elem)
    {
     return testFunc.apply(elem, args);
 });
};

This takes a test function, and an argument array, and defines an inline function as the callback for the Array.prototype.filter function which calls function.prototype.apply to execute the test function in the scope of the array element being tested with the specified arguments. You can then write a suite of generic testing functions like these:

testSuite = {

  range : function(key, min, max)
  {
       var min = parseFloat(min);
       var max = parseFloat(max);
       var keyVal = parseFloat(this[key]);
       if(!min || !max|| !keyVal) return false;
       else return keyVal >= min && keyVal <= max;
  },

  distinct : function(key, values)
  {
       if(typeof key != 'string') throw new TypeError('key must be a string');
       if(typeof values != 'object') values = [values];
       var keyVal = this[key];
       if(keyVal == undefined) return false;


       for(var i in values)
       {
          var value = values[i];
          if(typeof value == 'function') continue;
          if(typeof value == 'string')
          {
               if(keyVal.toString().toLowerCase() == value.toLowerCase()) return true;
               else continue; 
          }
          else
          {
              keyVal = parseFloat(keyVal);
              value = parseFloat(value);
              if(keyVal&&value&&(keyVal==value)) return true;
              else continue;

         }
       }
       return false;
  }
};

These are just the test functions associated with the requirements in your question, however you could create additional ones that do more complex things like test string values against a regular expression, or test key values that aren't simple datatypes.

Then you extend the object containing the houses array like so (I've called it housesData, call it whatever it is in your code):

 housesData.filterBy = function(tests)
 {
      ret = this.homes.slice(0);
      if(tests)
      {
           for(var i in tests)
           {
               var test = tests[i];
               if(typeof test != 'object') continue;
               if(!test.func || typeof test.func != 'function') throw new TypeError('argument 0 must be an array or object containing test objects, each with a key "func" of type function');
               else ret = ret.filterBy(test.func, test.args ? test.args : []);
           }
      }
      return ret;
 }

You could then call this function like so to get a result set using the generic functions defined above:

 result = housesData.filterBy([{func:range,args:['price','150000','400000'],
                      {func:distinct,args:['num_of_bedsf',[1, 2, 3]]}]);

The way I would actually forsee using this however is to serialize the generic form elements I mentioned earlier into an array or hashmap of the test objects. Here's a simple example I used to test this code (I used jQuery, because it's easier, all the previous code is in example.js, along with @artlung's dummy homes array):

 <?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE html 
      PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
     "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"&gt;
 <html xmlns="http://www.w3.org/1999/xhtml" xmlns:myNS="http://uri.for.your.schema" xml:lang="en" lang="en">
<head>
    <script src="jquery-1.3.2.js" type="text/javascript" language="javascript"></script>
    <script src="example.js" type="text/javascript" language="javascript"></script>
    <script type="text/javascript" language="javascript">
  runQuery = function(event)
  {
    var tests = [];
    $('#results').attr('value', '');;
    controls = $('#theForm > fieldset');
    for(var i = 0; i < controls.length ; i++)
    {
      var func;
      var args = [];
      control = controls.eq(i);
      func = testSuite[control.attr('myNS:type')];
      args.push(control.attr('myNS:key'));
      var inputs = $('input', control);
      for(var j=0; j< inputs.length; j++)
      {
        args.push(inputs[j].value);
      }
      tests.push({func:func,args:args});
    }
    result = housesData.filterBy(tests);
    resultStr = '';
    for(var i = 0; i < result.length; i++)
    {
      resultStr += result[i]['home_id'];
      if(i < (result.length -1)) resultStr += ', ';
    }
    $('#results').attr('value', resultStr);
  }
 </script>
</head>
<body>
  <form id="theForm" action="javascript:null(0)">
    <fieldset myNS:type="range" myNS:key="price">
     <legend>Price:</legend>
     min: <input type="text" myNS:type="min"></input>
     max: <input type="text" myNS:type="max"></input>
    </fieldset>
    <fieldset myNS:type="distinct" myNS:key="num_of_beds">
     <legend>Bedrooms</legend>
     bedrooms: <input type="text" myNS:type="value"></input>
    </fieldset>
    <button onclick="runQuery(event);">Submit</button>
  </form>
  <textarea id="results"></textarea>
    </body>
 </html>
Ryan Lynch
The really cool part about handling the query this way is that you could read the keys from the JSON and dynamically inject form elements for them on demand, allowing people to create complex queries. A user could for example, select a range of zip codes along with distinct zip code values outside of that range. You could also add additional keys to the house objects and not have to update code for the site.
Ryan Lynch
Ryan, I really appreciate your detailed answer. Quick question, so the only code I need is the following? Houses.filterBy = function(tests) { try { var tmpHouses = this.houses.slice(0); for(test in tests) { tmpHouses = tmpHouses.filterBy(test.func, test.args); { } catch(e) { // error handling return []; } return tmpHouses; }
GeorgeG
Ha, I wouldn't call it so much detailed as thoroughly revised. ;-) If you are going to go the function.prototype.apply route (which as people have pointed out is probably better than using eval) you'll need to extend the Array prototype with the filterBy function from the code block above the one you quoted, as well as write the test functions to be passed to it. You could use the distinct and range functions I wrote earlier, just have them evaluate the statemet using this[key] and return the result instead of the statement as a string.
Ryan Lynch
Here George, this answer is kind of long and rambling so I'm just gonna revise it to make it more focused on the solution you are looking for. Hopefully that will be more helpful.
Ryan Lynch
A: 

I agree with stereofrog about not reinventing the wheel, and just using the native Array filter method.

The function definition shown on the Mozilla page will add filter if it's missing, regardless of the browser, by using object detection (if (!Array.prototype.filter))

Any of the methods suggesting eval and/or multiple for loops are both slow and potentially unsafe.

You didn't say so, but I'm assuming the values will be coming from user input (a form) so I'd do something like:

var filtered_results = obj.homes
    .filter(function (home)
    {
        return parseInt(home.price) >= document.getElementsByName('price_gt')[0].value;
    })
    .filter(function (home)
    {
        return parseInt(home.price) <= document.getElementsByName('price_lt')[0].value;
    })
    .filter(function (home)
    {
        return parseFloat(home.num_of_baths) >= document.getElementsByName('baths_gt')[0].value;
    });

better yet, don't iterate through the list N times and just do:

var filtered_results = obj.homes.filter(function (home)
{
    return (
        parseInt(home.price) >= document.getElementsByName('price_gt')[0].value &&
        parseInt(home.price) <= document.getElementsByName('price_lt')[0].value &&
        parseFloat(home.num_of_baths) >= document.getElementsByName('baths_gt')[0].value
    );
});

I know this doesn't match the requested syntax but this approach is faster and more flexible (by not using eval or hardcoded values).

Ryan Lynch
I added the generic functions to my (second) answer to explain what I mean about flexibility.
Ryan Lynch
A: 

Ryan Lynch - If you're using strings for the comparisons ('value operator comparison') you can use plain JavaScript (value operator comparison). The logical inflexibility is the same, only with less speed and security (via eval()).

The lack of flexibility in my example comes from knowing which fields (price, baths, etc) and which operators we're interested in, but the original poster's request listed specific fields and comparisons (price < X, baths > Y, etc).

GeorgeG - if you're interested, and my assumptions about the filter values coming from a form are correct, I can write up a more generic "for each of the user-specified values, filter the results as requested" approach but I'll wait for confirmation and, if possible, a code sample.

When I said your solution was less flexible, I said so because you hardcoded in the object and its properties into your test function (i.e. home.price). I added a similar solution that uses apply instead of eval to execute statements in the element context. My goal here is to make the code depend as little as possible on specific properties of the objects being tested, and on where the values are coming from (i.e. form elements). This makes the code reusable beyond just the questioner.
Ryan Lynch
I should also point out that I'm not trying to insult your solution, because in truth my solution is similar to yours and several others here. I just think that the most reusable way to run the tests is to execute them in the context of the element being tested, by wrapping the filter function in such a way that you can use eval or apply to run the test in the scope of the array element.
Ryan Lynch
+1  A: 

Funny how most of these answers do nothing but try to put the matter of iterating over an array more elegantly, when that's not even what GeorgeG needs.

Assuming you have a huge JSON, you want something fast. Iterating is not that. You need jOrder (http://github.com/danstocker/jorder), that does search 100 times faster on a 1000 row table than iterative filtering. The bigger the table the higher the ratio.

Here's what you do.

Since jOrder can't handle more than one inequality filter at a time you'll have to do it in three steps but it will still be magnitudes faster.

First, create a jOrder table based on your raw data, and add a price index on it:

var table = jOrder(data.homes)
    .index('price', ['price'], { ordered: true, grouped: true, type: jOrder.number });

You can re-use this table in future searches. Now you start by getting the price filtered results, and wrapping it in another jOrder table, and finally adding a bathroom index:

var price_filtered = jOrder(table.where([{ price: { lower: 150000, upper: 400000 } }]))
    .index('bathrooms', ['num_of_baths'], { ordered: true, grouped: true, type: jOrder.number });

Then we go on by doing the same for bathrooms, adding an index on bedrooms, which doesn't have to be ordered since we won't use inequality filters here:

var bath_filtered = jOrder(price_filtered.where([{ num_of_baths: { lower: 2.5 } }]))
    .index('bedrooms', ['num_of_beds'], { grouped: true });

Finally, you get the fully filtered set by:

var filtered = jOrder(bath_filtered.where([{ num_of_beds: 1 }, { num_of_beds: 3 }]));

Of course you can wrap all this into a function that takes the three arguments:

  • { price: { lower: 150000, upper: 400000 } }
  • { num_of_baths: { lower: 2.5 } }
  • [{ num_of_beds: 1 }, { num_of_beds: 3 }]

and returns filtered.

Dan Stocker