How do you use a custom comparator with SingleColumnValueFilter on HBase?

Question

I am trying to filter rows from a HBase table using two SingleColumnValueFilter objects to bring back all records that fall within a range of long values for the column. According to the documentation for the SingleColumnValueFilter, it does a lexicographic compare of the column value unless you pass it your own comparator. The api shows the SingleColumnValueFilter taking a WritableByteArrayComparable as a way to accomplish this.

I wrote a class extending the WritableByteArrayComparable and overwriting the compare method.

    public class LongWritableComparable extends WritableByteArrayComparable {

    public LongWritableComparable() {
        super();
    }

    public LongWritableComparable(byte[] value) {
        super(value);
    }

    public LongWritableComparable(Long value) {
        super(Bytes.toBytes(value));
    }

    @Override
    public int compareTo(byte[] otherValue) {
        byte[] thisValue = this.getValue();
        long thisLong = Bytes.toLong(thisValue);
        long otherLong = Bytes.toLong(otherValue);

        if (thisLong == otherLong) {
            return 0;
        }
        if (thisLong < otherLong) {
            return -1;
        }
        return 1;
    }
}

I am settting up the filter using:

SingleColumnValueFilter lowerBoundAddressResourceFilter = new SingleColumnValueFilter(
                    graphTable.getResource().getName(),
                    Bytes.toBytes(HBaseProperties.getInstance().getQualifierIpAdressLongRepresentation()),
                    CompareFilter.CompareOp.GREATER_OR_EQUAL,
                    new LongWritableComparable(lowerRangeValue));

but am seeing the following exception:

2010-08-25 14:24:10,034 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer:
org.apache.hadoop.hbase.UnknownScannerException: Name: -1
        at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1880)
        at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
2010-08-25 14:24:10,052 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 0 on 60020, call next(-1, 1000) from 172.16.32.215:53206: error: org.apache.hadoop.hbase.UnknownScannerException: Name: -1
org.apache.hadoop.hbase.UnknownScannerException: Name: -1
        at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1880)
        at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)

Looking on the web the consensus for the cause of this execption is that the code accessing the resultScanner.next() is taking too long causing the lease to the regionServer to timeout, but this exception is getting thrown before

ResultScanner resultScanner = table.getScanner(scan);

returns with a ResultScanner that I can iterate through.

Does anyone have or know of an example of using a custom comparator with SingleColumnValueFilter that works or any insight into what I am doing wrong?

Answer 1

It appears that the jar with your custom comparator needs to be placed on the region server(s) where the table you are querying reside in the $HBASE_HOME/lib folder. After placing the jar restart your cluster and the query should work.