Hbase 過濾器API

原創 阿豪963 2020-07-01 13:17

filter ==> SQL 中的Where

filter的執行流程：

過濾器在客戶端創建,然後通過RPC發送到服務器上,由服務器執行

 
基礎過濾器：

 
比較器：

Comparator 

Description 

LongComparator

Assumes the given value array is a Java Long number and uses Bytes.toLong() to convert it. 

BinaryComparator

Uses Bytes.compareTo() to compare 當前值與閥值

BinaryPrefixComparator

Bytes.compareTo() 進行匹配，但是從左端開始前綴匹配

NullComparator

判斷當前值是否爲null

BitComparator

Performs a bitwise comparison, providing a BitwiseOp enumeration with AND, OR, and XOR operators. 

RegexStringComparator

正則表達式匹配

SubstringComparator

子字符串比對

  •  RowFilter 行鍵過濾器：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.*;
import org.apache.hadoop.hbase.util.Bytes;
 
import java.io.IOException;
 
/**
 * 基於行鍵上的過濾器
 */
public class FilterInHbase {
    public static void main(String[] args) throws IOException{
        Configuration configuration = HBaseConfiguration.create();
        Connection connection = ConnectionFactory.createConnection(configuration);
        //建立user表的連接
        Table table =connection.getTable(TableName.valueOf("user"));
        Scan scan=new Scan();
        //掃描列族info 列age
        scan.addColumn(Bytes.toBytes("info"),Bytes.toBytes("age"));
 
        System.out.println("行過濾器");
        //比較過濾器
        //這兒是指找出行小於或者等於"510824118261011172"的所有行
        Filter filter1 = new RowFilter(CompareFilter.CompareOp.LESS_OR_EQUAL, new BinaryComparator(Bytes.toBytes("813782218261011172")));
        //添加過濾器到掃描器中
        scan.setFilter(filter1);
        ResultScanner scanner1 = table.getScanner(scan);
        for(Result res:scanner1){
            System.out.println(res);
        }
        scanner1.close();
 
        System.out.println("正則過濾器");
        //正則過濾器
        //過濾行鍵以2結束的
        Filter filter2 = new RowFilter(CompareFilter.CompareOp.EQUAL,
                new RegexStringComparator(".*2$")
                );
        scan.setFilter(filter2);
        ResultScanner scanner2 = table.getScanner(scan);
        for (Result res:scanner2){
            System.out.println(res);
        }
        scanner2.close();
 
        //子串過濾器
        //過濾行鍵中包含了"61826"這個字符串
        System.out.println("子串過濾器");
        Scan scan3=new Scan();
        //掃描列族info 列age
        scan3.addColumn(Bytes.toBytes("info"),Bytes.toBytes("age"));
        Filter filter3=new RowFilter(CompareFilter.CompareOp.EQUAL,
                new SubstringComparator("61826")
                );
        scan3.setFilter(filter3);
        ResultScanner scanner3=table.getScanner(scan3);
        for(Result res:scanner3){
            System.out.println(res);
        }
        scanner3.close();
 
        table.close();
        connection.close();
    }
}
 
/**
Result:
 
 行過濾器 < 813782218261011172
 keyvalues={224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0}
 keyvalues={510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0}
 keyvalues={524382618264914241/info:age/1472196193913/Put/vlen=2/seqid=0}
 keyvalues={673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0}
 keyvalues={813782218261011172/info:age/1472196212550/Put/vlen=2/seqid=0}
 正則過濾器 已2結尾的行鍵
 keyvalues={510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0}
 keyvalues={673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0}
 keyvalues={813782218261011172/info:age/1472196212550/Put/vlen=2/seqid=0}
 子串過濾器 包含了"61826"的行鍵
 keyvalues={224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0}
 keyvalues={524382618264914241/info:age/1472196193913/Put/vlen=2/seqid=0}
 keyvalues={673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0}
 **/

 

  • FamilyFilter列族過濾器：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.BinaryComparator;
import org.apache.hadoop.hbase.filter.CompareFilter;
import org.apache.hadoop.hbase.filter.FamilyFilter;
import org.apache.hadoop.hbase.filter.Filter;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
/**
 * 列族過濾器
 * 比較列族來返回結果
 * 用戶可以在列族一級篩選所需數據
 */
public class FamilyFilterInHbase {
    public static void main(String[] args) throws IOException {
        Configuration configuration = HBaseConfiguration.create();
        Connection connection = ConnectionFactory.createConnection(configuration);
        //建立表的連接
        Table table = connection.getTable(TableName.valueOf("user"));
 
        //比較過濾器 現在表有2個列族 info  ship 當然實際有可能很多哦
        //取info < "kiss" <ship < "wings"
        Filter filter1 = new FamilyFilter(CompareFilter.CompareOp.GREATER, new BinaryComparator(Bytes.toBytes("kiss")));
        Scan scan = new Scan();
        scan.setFilter(filter1);
        ResultScanner scanner = table.getScanner(scan);
        for (Result result : scanner){
           System.out.println(result);
        }
        scanner.close();
 
        Get get1 = new Get(Bytes.toBytes("673782618261019142"));
        get1.setFilter(filter1);
        Result result1=table.get(get1);
        System.out.println("Result of get1(): " + result1);
 
        //添加列族過濾器 info
        Filter filter2= new FamilyFilter(CompareFilter.CompareOp.EQUAL,new BinaryComparator(Bytes.toBytes("info")));
        //獲取一行數據
        Get get2 = new Get(Bytes.toBytes("673782618261019142"));
        //但是get列族ship 那麼==>> Result of get():keyvalues=NONE  [本身衝突 所以無數據]
        //如果get列族info 那麼==>> Result of get2():keyvalues={673782618261019142/...
        get2.addFamily(Bytes.toBytes("ship"));
        get2.setFilter(filter2);
        Result result2 =table.get(get2);
        System.out.println("Result of get2():"+result2);
 
        scanner.close();
        table.close();
        connection.close();
 
    }
}
/**
 LESS "kiss"
 keyvalues={224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0}
 keyvalues={510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0, 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0, 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0, 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0, 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0}
 keyvalues={524382618264914241/info:age/1472196193913/Put/vlen=2/seqid=0, 524382618264914241/info:height/1472196194783/Put/vlen=3/seqid=0, 524382618264914241/info:name/1472196193255/Put/vlen=8/seqid=0, 524382618264914241/info:phone/1472196195125/Put/vlen=11/seqid=0, 524382618264914241/info:weight/1472196194970/Put/vlen=3/seqid=0}
 keyvalues={673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0, 673782618261019142/info:height/1472196211761/Put/vlen=3/seqid=0, 673782618261019142/info:name/1472196211678/Put/vlen=7/seqid=0, 673782618261019142/info:phone/1472196211956/Put/vlen=11/seqid=0, 673782618261019142/info:weight/1472196211841/Put/vlen=3/seqid=0}
 keyvalues={813782218261011172/info:age/1472196212550/Put/vlen=2/seqid=0, 813782218261011172/info:height/1472196212605/Put/vlen=3/seqid=0, 813782218261011172/info:name/1472196212480/Put/vlen=8/seqid=0, 813782218261011172/info:phone/1472196212713/Put/vlen=11/seqid=0, 813782218261011172/info:weight/1472196212651/Put/vlen=3/seqid=0}
 
 GREATER "kiss"
 keyvalues={224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0}
 keyvalues={510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0, 510824118261011172/ship:email/1472196213422/Put/vlen=12/seqid=0, 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0}
 keyvalues={524382618264914241/ship:addr/1472196195270/Put/vlen=7/seqid=0, 524382618264914241/ship:email/1472196195371/Put/vlen=13/seqid=0, 524382618264914241/ship:salary/1472196195485/Put/vlen=4/seqid=0}
 keyvalues={673782618261019142/ship:addr/1472196212059/Put/vlen=8/seqid=0, 673782618261019142/ship:email/1472196212176/Put/vlen=12/seqid=0, 673782618261019142/ship:salary/1472196212284/Put/vlen=4/seqid=0}
 keyvalues={813782218261011172/ship:addr/1472196212762/Put/vlen=4/seqid=0, 813782218261011172/ship:email/1472196212802/Put/vlen=12/seqid=0, 813782218261011172/ship:salary/1472196212840/Put/vlen=5/seqid=0}
 
 APPEND(LESS "kiss")
 Result of get(): keyvalues={673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0, 673782618261019142/info:height/1472196211761/Put/vlen=3/seqid=0, 673782618261019142/info:name/1472196211678/Put/vlen=7/seqid=0, 673782618261019142/info:phone/1472196211956/Put/vlen=11/seqid=0, 673782618261019142/info:weight/1472196211841/Put/vlen=3/seqid=0}
 APPEND(GREATER "kiss")
 Result of get1(): keyvalues={673782618261019142/ship:addr/1472196212059/Put/vlen=8/seqid=0, 673782618261019142/ship:email/1472196212176/Put/vlen=12/seqid=0, 673782618261019142/ship:salary/1472196212284/Put/vlen=4/seqid=0}
 
 //filter "info" get "ship"  
 Result of get():keyvalues=NONE
 //filter "info" get "info"
 Result of get2():keyvalues={673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0, 673782618261019142/info:height/1472196211761/Put/vlen=3/seqid=0, 673782618261019142/info:name/1472196211678/Put/vlen=7/seqid=0, 673782618261019142/info:phone/1472196211956/Put/vlen=11/seqid=0, 673782618261019142/info:weight/1472196211841/Put/vlen=3/seqid=0}
 
 **/

 

  •  ValueFilter值過濾器：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.*;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
 
/**
 * 值過濾器
 * 根據值進行篩選 可以聯合RegexStringComparator 進行設計
 */
public class FilterOfValue {
    public static void main(String[] args) throws IOException {
        Configuration configuration = HBaseConfiguration.create();
        Connection connection = ConnectionFactory.createConnection(configuration);
        //建立表的連接
        Table table = connection.getTable(TableName.valueOf("user"));
 
        //值中包含了177的過濾器
        Filter filter = new ValueFilter(CompareFilter.CompareOp.EQUAL,
                new SubstringComparator("1771392142")
                );
 
        Scan scan = new Scan();
        scan.setFilter(filter);
        ResultScanner scanner = table.getScanner(scan);
        for (Result result : scanner){
            for (Cell cell:result.rawCells()){
                System.out.println("Cell: "+cell+", Value: "+Bytes.toString(cell.getValueArray(),cell.getValueLength()));
            }
        }
        scanner.close();
 
        Get get1 = new Get(Bytes.toBytes("673782618261019142"));
        get1.setFilter(filter);
        Result result1=table.get(get1);
        for (Cell cell : result1.rawCells()) {
            System.out.println("Get1 Cell: " + cell + ", Value: " +
                    Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                            cell.getValueLength()));
        }
 
        Get get2 = new Get(Bytes.toBytes("813782218261011172"));
        get2.setFilter(filter);
        Result result2=table.get(get2);
        for (Cell cell : result2.rawCells()) {
            System.out.println("Get2 Cell: " + cell + ", Value: " +
                    Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                            cell.getValueLength()));
        }
 
        table.close();
        connection.close();
 
    }
}
 
 
/**
 原數據:                                                                               
 673782618261019142                              column=info:phone, timestamp=1472196211956, value=17713921424                                                                               
 813782218261011172                              column=info:phone, timestamp=1472196212713, value=12713921424
 *輸出結果:
 Cell: 673782618261019142/info:phone/1472196211956/Put/vlen=11/seqid=0, Value: 73782618261019142infophoneVŻt�17713921424
 Get1 Cell: 673782618261019142/info:phone/1472196211956/Put/vlen=11/seqid=0, Value: 17713921424
**/

 

  • DependentColumnFilter 參考列過濾器
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.*;
import org.apache.hadoop.hbase.util.Bytes;
 
import java.io.IOException;
 
 
/**
 * 參考列過濾器
 * 根據列名進行篩選
 */
public class FilterOfDependentColumnFilter {
    private static Table table=null;
    public static Table getTable() {
        if(table==null){
            try {
                Configuration configuration = HBaseConfiguration.create();
                Connection connection = ConnectionFactory.createConnection(configuration);
                //建立表的連接
                return connection.getTable(TableName.valueOf("user"));
            }catch (IOException e){
                return table;
            }
        }
        return table;
    }
 
    public static void filter(boolean drop,CompareFilter.CompareOp oper,ByteArrayComparable comparable) throws IOException {
        Filter filter;
        if (comparable != null) {
            filter = new DependentColumnFilter(Bytes.toBytes("info"), Bytes.toBytes("phone"), drop, oper, comparable);
        } else {
            filter = new DependentColumnFilter(Bytes.toBytes("info"), Bytes.toBytes("phone"), drop);
        }
        Scan scan = new Scan();
        scan.setFilter(filter);
        ResultScanner scanner = getTable().getScanner(scan);
        for (Result result : scanner) {
            for (Cell cell : result.rawCells()) {
                System.out.println("Cell: " + cell + ", Value: " +
                        Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                cell.getValueLength()));
            }
        }
        scanner.close();
        Get get = new Get(Bytes.toBytes("673782618261019142"));
        get.setFilter(filter);
        Result result = getTable().get(get);
        for (Cell cell : result.rawCells()) {
            System.out.println("Cell: " + cell + ", Value: " +
                    Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                            cell.getValueLength()));
        }
    }
 
 
 
 
    public static void main(String[] args) throws IOException {
        filter(true, CompareFilter.CompareOp.NO_OP, null);
        filter(false, CompareFilter.CompareOp.NO_OP, null);
        filter(true, CompareFilter.CompareOp.EQUAL,
                new BinaryPrefixComparator(Bytes.toBytes("17713921424")));
        filter(false, CompareFilter.CompareOp.EQUAL,
                new BinaryPrefixComparator(Bytes.toBytes("17713921424")));
        filter(true, CompareFilter.CompareOp.EQUAL,
                new RegexStringComparator(".*\\.5"));
        filter(false, CompareFilter.CompareOp.EQUAL,
                new RegexStringComparator(".*\\.5"));
    }
}
/**
 
 **/

 

  • PrefixFilter 前綴過濾器：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.*;
import org.apache.hadoop.hbase.util.Bytes;
 
import java.io.IOException;
 
 
/**
 * 前綴過濾器
 * 根據行鍵的前綴進行過濾
 */
public class FilterOfPrefixFilter {
    public static void main(String[] args) throws IOException {
        Configuration configuration = HBaseConfiguration.create();
        Connection connection = ConnectionFactory.createConnection(configuration);
        //建立表的連接
        Table table = connection.getTable(TableName.valueOf("user"));
        Filter filter = new PrefixFilter(Bytes.toBytes("510824"));
 
        Scan scan = new Scan();
        scan.setFilter(filter);
        ResultScanner scanner = table.getScanner(scan);
        for (Result result : scanner) {
            for (Cell cell : result.rawCells()) {
                System.out.println("Cell: " + cell + ", Value: " +
                        Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                cell.getValueLength()));
            } }
        scanner.close();
        Get get = new Get(Bytes.toBytes("row-5"));
        get.setFilter(filter);
        Result result = table.get(get);
        for (Cell cell : result.rawCells()) {
            System.out.println("Cell: " + cell + ", Value: " +
                    Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                            cell.getValueLength()));
        }
 
        scanner.close();
        table.close();
        connection.close();
 
    }
}
/**
 Cell: 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0, Value: 188
 Cell: 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0, Value: yangyang
 Cell: 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0, Value: 18013921626
 Cell: 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0, Value: 138
 Cell: 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0, Value: shanghai
 Cell: 510824118261011172/ship:email/1472196213422/Put/vlen=12/seqid=0, Value: [email protected]
 Cell: 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0, Value: 50000
**/

 

  •  PageFilter 分頁過濾器
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.Filter;
import org.apache.hadoop.hbase.filter.PageFilter;
import org.apache.hadoop.hbase.filter.PrefixFilter;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
/**
 * 分頁過濾器
 */
public class FilterOfPage {
    private static final byte[] POSTFIX = new byte[] { 0x00 };
 
 
    public static void main(String[] args) throws IOException {
        Configuration configuration = HBaseConfiguration.create();
        Connection connection = ConnectionFactory.createConnection(configuration);
        //建立表的連接
        Table table = connection.getTable(TableName.valueOf("user"));
        //分頁大小
        Filter filter = new PageFilter(3);
 
        int totalRows = 0;
        byte[] lastRow = null;
 
        while (true) {
            Scan scan = new Scan();
            scan.setFilter(filter);
            if (lastRow != null) {
                byte[] startRow = Bytes.add(lastRow, POSTFIX);
                System.out.println("start row: " +
                        Bytes.toStringBinary(startRow));
                scan.setStartRow(startRow);
            }
            ResultScanner scanner = table.getScanner(scan);
            int localRows = 0;
            Result result;
            while ((result = scanner.next()) != null) {
                System.out.println(localRows++ + ": " + result);
                totalRows++;
                lastRow = result.getRow();
            }
            scanner.close();
            if (localRows == 0) break;
        }
        System.out.println("total rows: " + totalRows);
 
        table.close();
        connection.close();
 
    }
}
/**
 0: keyvalues={224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0}
 1: keyvalues={510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0, 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0, 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0, 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0, 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0, 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0, 510824118261011172/ship:email/1472196213422/Put/vlen=12/seqid=0, 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0}
 2: keyvalues={524382618264914241/info:age/1472196193913/Put/vlen=2/seqid=0, 524382618264914241/info:height/1472196194783/Put/vlen=3/seqid=0, 524382618264914241/info:name/1472196193255/Put/vlen=8/seqid=0, 524382618264914241/info:phone/1472196195125/Put/vlen=11/seqid=0, 524382618264914241/info:weight/1472196194970/Put/vlen=3/seqid=0, 524382618264914241/ship:addr/1472196195270/Put/vlen=7/seqid=0, 524382618264914241/ship:email/1472196195371/Put/vlen=13/seqid=0, 524382618264914241/ship:salary/1472196195485/Put/vlen=4/seqid=0}
 start row: 524382618264914241\x00
 0: keyvalues={673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0, 673782618261019142/info:height/1472196211761/Put/vlen=3/seqid=0, 673782618261019142/info:name/1472196211678/Put/vlen=7/seqid=0, 673782618261019142/info:phone/1472196211956/Put/vlen=11/seqid=0, 673782618261019142/info:weight/1472196211841/Put/vlen=3/seqid=0, 673782618261019142/ship:addr/1472196212059/Put/vlen=8/seqid=0, 673782618261019142/ship:email/1472196212176/Put/vlen=12/seqid=0, 673782618261019142/ship:salary/1472196212284/Put/vlen=4/seqid=0}
 1: keyvalues={813782218261011172/info:age/1472196212550/Put/vlen=2/seqid=0, 813782218261011172/info:height/1472196212605/Put/vlen=3/seqid=0, 813782218261011172/info:name/1472196212480/Put/vlen=8/seqid=0, 813782218261011172/info:phone/1472196212713/Put/vlen=11/seqid=0, 813782218261011172/info:weight/1472196212651/Put/vlen=3/seqid=0, 813782218261011172/ship:addr/1472196212762/Put/vlen=4/seqid=0, 813782218261011172/ship:email/1472196212802/Put/vlen=12/seqid=0, 813782218261011172/ship:salary/1472196212840/Put/vlen=5/seqid=0}
 start row: 813782218261011172\x00
 2016-08-29 17:17:57,197 INFO  [main] client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x56c13890bf003a
 total rows: 5
**/
  • KeyOnlyFilter 行鍵過濾器

  • FirstKeyOnlyFilter 首次行鍵過濾器

  • FirstKeyValueMatchingQualifiersFilter 

  • InclusiveStopFilter 包含結束的過濾器

  • FuzzyRowFilter 模糊行匹配過濾器

  • ColumnCountGetFilter 列計數過濾器
    可以使用這個過濾器來限制每行最多取回多少列，

    當一行的列數到設定的最大值，這個過濾器會停止整個掃描操作。

    適合在get方法中使用

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.hbase.Cell;
    import org.apache.hadoop.hbase.HBaseConfiguration;
    import org.apache.hadoop.hbase.TableName;
    import org.apache.hadoop.hbase.client.*;
    import org.apache.hadoop.hbase.filter.ColumnCountGetFilter;
    import org.apache.hadoop.hbase.util.Bytes;
    import java.io.IOException;
     
    /**
     * ColumnCountGetFilter 列數過濾器
     */
    public class FilterOfColumnCountGetFilter {
        public static void main(String args[]) throws IOException{
            Configuration configuration = HBaseConfiguration.create();
            Connection connection = ConnectionFactory.createConnection(configuration);
            Table table = connection.getTable(TableName.valueOf("user"));
     
            //限制返回的列數
            ColumnCountGetFilter columnCountGetFilter = new ColumnCountGetFilter(3);
            // column=info:age,column=info:height,column=info:name,column=info:phone,column=info:weight,
            Get get = new Get(Bytes.toBytes("224382618261914241"));
            get.setFilter(columnCountGetFilter);
            Result result = table.get(get);
            System.out.println("Result of columnCountGetFilter get: ");
            for (Cell cell : result.rawCells()) {
                System.out.println("Cell: " + cell + ", Value: " +
                        Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                cell.getValueLength()));
            }
            Get get1 = new Get(Bytes.toBytes("224382618261914241"));
            Result result1 = table.get(get1);
            System.out.println("Result of get: ");
            for (Cell cell : result1.rawCells()) {
                System.out.println("Cell: " + cell + ", Value: " +
                        Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                cell.getValueLength()));
            }
            table.close();
            connection.close();
        }
    }
     
    /**
     Result of columnCountGetFilter get:
     Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
     Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
     Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
     Result of get:
     Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
     Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
     Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
     Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
     Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
     Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
     Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: [email protected]
     Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
    **/

     

  • ColumnPaginationFilter 列分頁過濾
    HBase 的列可以很多，所以出現了列分頁
    該過濾器可以對一行的所有列進行分頁
    構造函數：
    ColumnPaginationFilter(int limit, int offset)
    ColumnPaginationFilter(int limit, byte[] columnOffset)
    limit 限制取回來列數  offset 偏移位就是開始位置 byte[] 字符串/書籤偏移從哪裏開始分頁。
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.hbase.Cell;
    import org.apache.hadoop.hbase.HBaseConfiguration;
    import org.apache.hadoop.hbase.TableName;
    import org.apache.hadoop.hbase.client.*;
    import org.apache.hadoop.hbase.filter.ColumnPaginationFilter;
    import org.apache.hadoop.hbase.util.Bytes;
     
    import java.io.IOException;
     
    /**
     * ColumnPageFilter 列分頁過濾器
     */
    public class FilterOColumnPageFilter {
        public static void main(String args[]) throws IOException{
            Configuration configuration = HBaseConfiguration.create();
            Connection connection = ConnectionFactory.createConnection(configuration);
            Table table = connection.getTable(TableName.valueOf("user"));
            //限制返回的列數 從第4列開始取 取3列數據
            ColumnPaginationFilter columnPaginationFilter = new ColumnPaginationFilter(3,4);
            Get get = new Get(Bytes.toBytes("224382618261914241"));
            get.setFilter(columnPaginationFilter);
            Result result = table.get(get);
            System.out.println("Result of ColumnPageFilter get: ");
            for (Cell cell : result.rawCells()) {
                System.out.println("Cell: " + cell + ", Value: " +
                        Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                cell.getValueLength()));
            }
     
            //限制返回的列數 從第name列開始取 取3列數據
            ColumnPaginationFilter columnPaginationFilter2 = new ColumnPaginationFilter(3,Bytes.toBytes("name"));
            Get get2 = new Get(Bytes.toBytes("224382618261914241"));
            get2.setFilter(columnPaginationFilter2);
            Result result2 = table.get(get2);
            System.out.println("Result of ColumnPageFilter get: ");
            for (Cell cell : result2.rawCells()) {
                System.out.println("Cell: " + cell + ", Value: " +
                        Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                cell.getValueLength()));
            }
     
            Get get1 = new Get(Bytes.toBytes("224382618261914241"));
            Result result1 = table.get(get1);
            System.out.println("Result of get: ");
            for (Cell cell : result1.rawCells()) {
                System.out.println("Cell: " + cell + ", Value: " +
                        Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                cell.getValueLength()));
            }
            table.close();
            connection.close();
        }
    }
     
    /**
     Result of ColumnPageFilter get:
     Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
     Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
     Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: [email protected]
     Result of ColumnPageFilter get:
     Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
     Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
     Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
     Result of get:
     Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
     Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
     Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
     Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
     Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
     Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
     Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: [email protected]
     Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
    **/

 

  • ColumnPrefixFilter 列前綴過慮器 
    該過濾器通過對列名稱進行前綴匹配過濾
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.hbase.Cell;
    import org.apache.hadoop.hbase.HBaseConfiguration;
    import org.apache.hadoop.hbase.TableName;
    import org.apache.hadoop.hbase.client.*;
    import org.apache.hadoop.hbase.filter.ColumnPrefixFilter;
    import org.apache.hadoop.hbase.filter.Filter;
    import org.apache.hadoop.hbase.util.Bytes;
    import java.io.IOException;
     
    /**
     * ColumnPrefixFilter 列前綴過濾器
     */
    public class FilterOfColumnPrefixFilter {
        public static void main(String args[]) throws IOException{
            Configuration configuration = HBaseConfiguration.create();
            Connection connection = ConnectionFactory.createConnection(configuration);
            Table table = connection.getTable(TableName.valueOf("user"));
            //取列名已ag開頭的
            Filter columnPrefixFilter = new ColumnPrefixFilter(Bytes.toBytes("a"));
            Get get = new Get(Bytes.toBytes("224382618261914241"));
            get.setFilter(columnPrefixFilter);
            Result result = table.get(get);
            System.out.println("Result of columnPrefixFilter get: ");
            for (Cell cell : result.rawCells()) {
                System.out.println("Cell: " + cell + ", Value: " +
                        Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                cell.getValueLength()));
            }
     
            Get get1 = new Get(Bytes.toBytes("224382618261914241"));
            Result result1 = table.get(get1);
            System.out.println("Result of get: ");
            for (Cell cell : result1.rawCells()) {
                System.out.println("Cell: " + cell + ", Value: " +
                        Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                cell.getValueLength()));
            }
            table.close();
            connection.close();
        }
    }
     
    /**
     Result of columnPrefixFilter get:
     Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
     Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
     Result of get:
     Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
     Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
     Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
     Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
     Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
     Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
     Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: [email protected]
     Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
    **/

  • MultipleColumnPrefixFilter 多個列前綴過濾器
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.hbase.Cell;
    import org.apache.hadoop.hbase.HBaseConfiguration;
    import org.apache.hadoop.hbase.TableName;
    import org.apache.hadoop.hbase.client.*;
    import org.apache.hadoop.hbase.filter.Filter;
    import org.apache.hadoop.hbase.filter.MultipleColumnPrefixFilter;
    import org.apache.hadoop.hbase.util.Bytes;
     
    import java.io.IOException;
     
    /**
     * MultipleColumnPrefixFilter 多個列前綴過濾器
     */
    public class FilterOfMultipleColumnPrefixFilter {
        public static void main(String args[]) throws IOException{
            Configuration configuration = HBaseConfiguration.create();
            Connection connection = ConnectionFactory.createConnection(configuration);
            Table table = connection.getTable(TableName.valueOf("user"));
            //取列名已a開頭的 還有已h 開頭的
            Filter filter = new MultipleColumnPrefixFilter(new byte[][] {Bytes.toBytes("a"),Bytes.toBytes("h")});
            Get get = new Get(Bytes.toBytes("224382618261914241"));
            get.setFilter(filter);
            Result result = table.get(get);
            System.out.println("Result of columnPrefixFilter get: ");
            for (Cell cell : result.rawCells()) {
                System.out.println("Cell: " + cell + ", Value: " +
                        Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                cell.getValueLength()));
            }
     
            Get get1 = new Get(Bytes.toBytes("224382618261914241"));
            Result result1 = table.get(get1);
            System.out.println("Result of get: ");
            for (Cell cell : result1.rawCells()) {
                System.out.println("Cell: " + cell + ", Value: " +
                        Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                cell.getValueLength()));
            }
            table.close();
            connection.close();
        }
    }
     
    /**
     Result of columnPrefixFilter get:
     Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
     Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
     Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
     Result of get:
     Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
     Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
     Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
     Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
     Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
     Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
     Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: [email protected]
     Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
    **/

     

  • ColumnRangeFilter 列範圍過濾器
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.hbase.Cell;
    import org.apache.hadoop.hbase.HBaseConfiguration;
    import org.apache.hadoop.hbase.TableName;
    import org.apache.hadoop.hbase.client.*;
    import org.apache.hadoop.hbase.filter.ColumnRangeFilter;
    import org.apache.hadoop.hbase.filter.Filter;
    import org.apache.hadoop.hbase.util.Bytes;
     
    import java.io.IOException;
     
    /**
     * ColumnRangeFilter 列範圍過濾器
     */
    public class FilterOfColumnRangeFilter {
        public static void main(String args[]) throws IOException{
            Configuration configuration = HBaseConfiguration.create();
            Connection connection = ConnectionFactory.createConnection(configuration);
            Table table = connection.getTable(TableName.valueOf("user"));
            //minColumn - minimum value for the column range. If if it's null, there is no lower bound.
            //minColumnInclusive - if true, include minColumn in the range. 如果是true 就要包含minColumn
            //maxColumn - maximum value for the column range. If it's null,
            //maxColumnInclusive - if true, include maxColumn in the range. there is no upper bound.
           //從email到phone範圍內的所有列 第二個參數爲true所以包含了email <br>　　    Filter filter = new ColumnRangeFilter(Bytes.toBytes("email"), true, Bytes.toBytes("phone"), false);
            Get get = new Get(Bytes.toBytes("224382618261914241"));
            get.setFilter(filter);
            Result result = table.get(get);
            System.out.println("Result of ColumnRangeFilter get: ");
            for (Cell cell : result.rawCells()) {
                System.out.println("Cell: " + cell + ", Value: " +
                        Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                cell.getValueLength()));
            }
     
            Get get1 = new Get(Bytes.toBytes("224382618261914241"));
            Result result1 = table.get(get1);
            System.out.println("Result of get: ");
            for (Cell cell : result1.rawCells()) {
                System.out.println("Cell: " + cell + ", Value: " +
                        Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                cell.getValueLength()));
            }
            table.close();
            connection.close();
        }
    }
     
    /**
     Result of ColumnRangeFilter get:
     Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
     Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
     Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: [email protected]
     Result of get:
     Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
     Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
     Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
     Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
     Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
     Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
     Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: [email protected]
     Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
     2016-08-31 17:53:28,394 INFO  [main] client.ConnectionManager$HConnectionImplementati
    **/

     


  • SingleColumnValueFilter 單列值過濾器
    用一列的值決定是否一行數據被過濾
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.hbase.Cell;
    import org.apache.hadoop.hbase.HBaseConfiguration;
    import org.apache.hadoop.hbase.TableName;
    import org.apache.hadoop.hbase.client.*;
    import org.apache.hadoop.hbase.filter.CompareFilter;
    import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
    import org.apache.hadoop.hbase.filter.SubstringComparator;
    import org.apache.hadoop.hbase.util.Bytes;
    import java.io.IOException;
     
    /**
     * SingleColumnValueFilter 單列過濾器
     */
    public class FilterOfSingleColumnValueFilter {
        public static void main(String args[]) throws IOException{
            Configuration configuration = HBaseConfiguration.create();
            Connection connection = ConnectionFactory.createConnection(configuration);
            Table table = connection.getTable(TableName.valueOf("user"));
     
            //  510824118261011172 column=ship:email, timestamp=1472196213422, [email protected]
            SingleColumnValueFilter singleColumnValueFilter = new SingleColumnValueFilter(Bytes.toBytes("ship"),Bytes.toBytes("email"), CompareFilter.CompareOp.EQUAL,new SubstringComparator("[email protected]"));
            singleColumnValueFilter.setFilterIfMissing(true);
     
            Scan scan = new Scan();
            scan.setFilter(singleColumnValueFilter);
            ResultScanner results = table.getScanner(scan);
            for (Result result:results){
                for (Cell cell :result.rawCells()){
                    System.out.println("Cell: "+cell+",Value:" + Bytes.toString(cell.getValueArray(),cell.getValueOffset(), cell.getValueLength()));
                }
            }
            results.close();
            // 224382618261914241 column=ship:email, timestamp=1472196211530, [email protected]
            Get get = new Get(Bytes.toBytes("224382618261914241"));
            get.setFilter(singleColumnValueFilter);
            Result result = table.get(get);
            System.out.println("Result of get: ");
            for (Cell cell : result.rawCells()) {
                System.out.println("Cell: " + cell + ", Value: " +
                        Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                cell.getValueLength()));
            }
     
            Get get1 = new Get(Bytes.toBytes("510824118261011172"));
            get1.setFilter(singleColumnValueFilter);
            Result result1 = table.get(get1);
            System.out.println("Result of get1: ");
            for (Cell cell : result1.rawCells()) {
                System.out.println("Cell: " + cell + ", Value: " +
                        Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                cell.getValueLength()));
            }
     
            table.close();
            connection.close();
        }
    }
     
    /**
     Cell: 510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0,Value:18
     Cell: 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0,Value:188
     Cell: 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0,Value:yangyang
     Cell: 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0,Value:18013921626
     Cell: 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0,Value:138
     Cell: 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0,Value:shanghai
     Cell: 510824118261011172/ship:email/1472196213422/Put/vlen=12/seqid=0,Value:[email protected]
     Cell: 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0,Value:50000
     Result of get:
     Result of get1:
     Cell: 510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0, Value: 18
     Cell: 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0, Value: 188
     Cell: 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0, Value: yangyang
     Cell: 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0, Value: 18013921626
     Cell: 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0, Value: 138
     Cell: 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0, Value: shanghai
     Cell: 510824118261011172/ship:email/1472196213422/Put/vlen=12/seqid=0, Value: [email protected]
     Cell: 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0, Value: 50000
    **/

     

  • SingleColumnValueExcludeFilter 單列排除過濾器
    單列排除過濾器繼承自SingleColumnValueFilter，過濾的方式還是按照SingleColumnValueFilter去過濾，
    但是最後的結果集去除了作爲過濾條件的列
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
            //  510824118261011172 column=ship:email, timestamp=1472196213422, [email protected]
            SingleColumnValueExcludeFilter singleColumnValueExcludeFilter = new SingleColumnValueExcludeFilter(Bytes.toBytes("ship"),Bytes.toBytes("email"), CompareFilter.CompareOp.EQUAL,new SubstringComparator("[email protected]"));
            singleColumnValueExcludeFilter.setFilterIfMissing(true);
     
            Scan scan = new Scan();
            scan.setFilter(singleColumnValueExcludeFilter);
            ResultScanner results = table.getScanner(scan);
            for (Result result:results){
                for (Cell cell :result.rawCells()){
                    System.out.println("Cell: "+cell+",Value:" + Bytes.toString(cell.getValueArray(),cell.getValueOffset(), cell.getValueLength()));
                }
            }
            results.close();
            // 224382618261914241 column=ship:email, timestamp=1472196211530, [email protected]
            Get get = new Get(Bytes.toBytes("224382618261914241"));
            get.setFilter(singleColumnValueExcludeFilter);
            Result result = table.get(get);
            System.out.println("Result of get: ");
            for (Cell cell : result.rawCells()) {
                System.out.println("Cell: " + cell + ", Value: " +
                        Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                cell.getValueLength()));
            }
     
            Get get1 = new Get(Bytes.toBytes("510824118261011172"));
            get1.setFilter(singleColumnValueExcludeFilter);
            Result result1 = table.get(get1);
            System.out.println("Result of get1: ");
            for (Cell cell : result1.rawCells()) {
                System.out.println("Cell: " + cell + ", Value: " +
                        Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                cell.getValueLength()));
            }
    /**
     * Cell: 510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0,Value:18
     Cell: 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0,Value:188
     Cell: 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0,Value:yangyang
     Cell: 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0,Value:18013921626
     Cell: 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0,Value:138
     Cell: 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0,Value:shanghai
     Cell: 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0,Value:50000
     Result of get:
     Result of get1:
     Cell: 510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0, Value: 18
     Cell: 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0, Value: 188
     Cell: 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0, Value: yangyang
     Cell: 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0, Value: 18013921626
     Cell: 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0, Value: 138
     Cell: 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0, Value: shanghai
     Cell: 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0, Value: 50000
     */

     

  • TimestampsFilter 時間過濾器
    使用時間戳的值來過濾值
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.hbase.HBaseConfiguration;
    import org.apache.hadoop.hbase.TableName;
    import org.apache.hadoop.hbase.client.*;
    import org.apache.hadoop.hbase.filter.Filter;
    import org.apache.hadoop.hbase.filter.TimestampsFilter;
    import java.io.IOException;
    import java.util.ArrayList;
    import java.util.List;
     
    /**
     * TimestampsFilter 時間過濾器
     */
    public class FilterOfTimestampsFilter {
     
        public static void main(String args[]) throws IOException{
            Configuration configuration = HBaseConfiguration.create();
            Connection connection = ConnectionFactory.createConnection(configuration);
            Table table = connection.getTable(TableName.valueOf("user"));
            List<Long> ts = new ArrayList<Long>();
            ts.add(new Long("1472196195270"));
            ts.add(new Long("1472196212480"));
            ts.add(new Long(15));
            Filter filter = new TimestampsFilter(ts);
            Scan scan1 = new Scan();
            scan1.setFilter(filter);
            ResultScanner scanner1 = table.getScanner(scan1);
            for(Result result:scanner1){
                System.out.println(result);
            }
            scanner1.close();
            Scan scan2 = new Scan();
            scan2.setFilter(filter);
            //加了時間範圍 故意多加了1s 1472196212480+1=1472196212481
            scan2.setTimeRange(1472196195271L, 1472196212481L);
            ResultScanner scanner2 = table.getScanner(scan2);
            System.out.println("Add time range:");
            for (Result result : scanner2) {
                System.out.println(result);
            }
            scanner2.close();
        }
    }
    /**
     keyvalues={524382618264914241/ship:addr/1472196195270/Put/vlen=7/seqid=0}
     keyvalues={813782218261011172/info:name/1472196212480/Put/vlen=8/seqid=0}
     Add time range:
     keyvalues={813782218261011172/info:name/1472196212480/Put/vlen=8/seqid=0}
    **/
  • RandomRowFilter  隨機行過濾器
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.hbase.Cell;
    import org.apache.hadoop.hbase.HBaseConfiguration;
    import org.apache.hadoop.hbase.TableName;
    import org.apache.hadoop.hbase.client.*;
    import org.apache.hadoop.hbase.filter.Filter;
    import org.apache.hadoop.hbase.filter.RandomRowFilter;
    import org.apache.hadoop.hbase.util.Bytes;
     
    import java.io.IOException;
     
    /**
     * RandomRowFilter 隨機行過濾器
     */
    public class FilterOfRandomRowFilter {
        public static void main(String args[]) throws IOException{
            Configuration configuration = HBaseConfiguration.create();
            Connection connection = ConnectionFactory.createConnection(configuration);
            Table table = connection.getTable(TableName.valueOf("user"));
            //該構造參數0-1之間,如果爲負數全部過濾,大於1全部通過 0.2f表的該行數據20%的概率通過
            Filter filter = new RandomRowFilter(0.2f);
            Scan scan = new Scan();
            scan.setFilter(filter);
            ResultScanner results = table.getScanner(scan);
            for (Result result:results) {
                for (Cell cell : result.rawCells()) {
                    System.out.println("Cell: " + cell + ", Value: " +
                            Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                    cell.getValueLength()));
                }
            }
            table.close();
            connection.close();
        }
    }
     
    /**
     Cell: 510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0, Value: 18
     Cell: 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0, Value: 188
     Cell: 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0, Value: yangyang
     Cell: 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0, Value: 18013921626
     Cell: 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0, Value: 138
     Cell: 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0, Value: shanghai
     Cell: 510824118261011172/ship:email/1472196213422/Put/vlen=12/seqid=0, Value: [email protected]
     Cell: 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0, Value: 50000
    **/

    　　

  • Decorating Filters  裝飾過濾器或附加過濾器
  • SkipFilter 跳過過濾器 根據構造器中的過濾器爲基準跳過行
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    94
    95
    96
    97
    98
    99
    100
    101
    102
    103
    104
    105
    106
    107
    108
    109
    110
    111
    112
    113
    114
    115
    116
    117
    118
    119
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.hbase.Cell;
    import org.apache.hadoop.hbase.HBaseConfiguration;
    import org.apache.hadoop.hbase.TableName;
    import org.apache.hadoop.hbase.client.*;
    import org.apache.hadoop.hbase.filter.*;
    import org.apache.hadoop.hbase.util.Bytes;
    import java.io.IOException;
     
    /**
     * SkipFilter 跳過(忽略)過濾器
     * similarface
     * [email protected]
     */
    public class FilterOfSkipFilter {
        public static void main(String args[]) throws IOException{
            Configuration configuration = HBaseConfiguration.create();
            Connection connection = ConnectionFactory.createConnection(configuration);
            Table table = connection.getTable(TableName.valueOf("user"));
            //510824118261011172                                     column=info:height, timestamp=1472196213056, value=188
            //673782618261019142                                     column=info:weight, timestamp=1472196211841, value=188
            //如果列值中含有188這個數值,那麼這列將會跳過
            Filter filter = new ValueFilter(CompareFilter.CompareOp.NOT_EQUAL,new BinaryComparator(Bytes.toBytes("188")));
     
            Scan scan = new Scan();
            scan.setFilter(filter);
            ResultScanner results = table.getScanner(scan);
            for (Result result:results) {
                for (Cell cell : result.rawCells()) {
                    System.out.println("Cell: " + cell + ", Value: " +
                            Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                    cell.getValueLength()));
                }
            }
            Scan scan1 = new Scan();
            //skipfilter的構造參數時filter
            //  filter==>如果列值中含有188這個數值,那麼這列將會跳過 filter2==>就是如果列值中含有188這個數值那麼整個行都會被跳過 表示不會出現[510824118261011172,673782618261019142]
            Filter filter2 = new SkipFilter(filter);
            scan1.setFilter(filter2);
            ResultScanner scanner2= table.getScanner(scan1);
            for(Result result:scanner2){
                for (Cell cell : result.rawCells()) {
                    System.out.println("SKIP Cell: " + cell + ", Value: " +
                            Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                    cell.getValueLength()));
                }
            }
            results.close();
            scanner2.close();
            table.close();
            connection.close();
        }
    }
     
    /**
     Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
     Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
     Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
     Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
     Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
     Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
     Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: [email protected]
     Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
     Cell: 510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0, Value: 18
     Cell: 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0, Value: yangyang
     Cell: 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0, Value: 18013921626
     Cell: 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0, Value: 138
     Cell: 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0, Value: shanghai
     Cell: 510824118261011172/ship:email/1472196213422/Put/vlen=12/seqid=0, Value: [email protected]
     Cell: 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0, Value: 50000
     Cell: 524382618264914241/info:age/1472196193913/Put/vlen=2/seqid=0, Value: 30
     Cell: 524382618264914241/info:height/1472196194783/Put/vlen=3/seqid=0, Value: 168
     Cell: 524382618264914241/info:name/1472196193255/Put/vlen=8/seqid=0, Value: zhangsan
     Cell: 524382618264914241/info:phone/1472196195125/Put/vlen=11/seqid=0, Value: 13212321424
     Cell: 524382618264914241/info:weight/1472196194970/Put/vlen=3/seqid=0, Value: 168
     Cell: 524382618264914241/ship:addr/1472196195270/Put/vlen=7/seqid=0, Value: beijing
     Cell: 524382618264914241/ship:email/1472196195371/Put/vlen=13/seqid=0, Value: [email protected]
     Cell: 524382618264914241/ship:salary/1472196195485/Put/vlen=4/seqid=0, Value: 3000
     Cell: 673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0, Value: 19
     Cell: 673782618261019142/info:height/1472196211761/Put/vlen=3/seqid=0, Value: 178
     Cell: 673782618261019142/info:name/1472196211678/Put/vlen=7/seqid=0, Value: zhaoliu
     Cell: 673782618261019142/info:phone/1472196211956/Put/vlen=11/seqid=0, Value: 17713921424
     Cell: 673782618261019142/ship:addr/1472196212059/Put/vlen=8/seqid=0, Value: shenzhen
     Cell: 673782618261019142/ship:email/1472196212176/Put/vlen=12/seqid=0, Value: [email protected]
     Cell: 673782618261019142/ship:salary/1472196212284/Put/vlen=4/seqid=0, Value: 8000
     Cell: 813782218261011172/info:age/1472196212550/Put/vlen=2/seqid=0, Value: 19
     Cell: 813782218261011172/info:height/1472196212605/Put/vlen=3/seqid=0, Value: 158
     Cell: 813782218261011172/info:name/1472196212480/Put/vlen=8/seqid=0, Value: wangmazi
     Cell: 813782218261011172/info:phone/1472196212713/Put/vlen=11/seqid=0, Value: 12713921424
     Cell: 813782218261011172/info:weight/1472196212651/Put/vlen=3/seqid=0, Value: 118
     Cell: 813782218261011172/ship:addr/1472196212762/Put/vlen=4/seqid=0, Value: xian
     Cell: 813782218261011172/ship:email/1472196212802/Put/vlen=12/seqid=0, Value: [email protected]
     Cell: 813782218261011172/ship:salary/1472196212840/Put/vlen=5/seqid=0, Value: 10000
      
     SKIP Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
     SKIP Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
     SKIP Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
     SKIP Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
     SKIP Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
     SKIP Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
     SKIP Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: [email protected]
     SKIP Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
     SKIP Cell: 524382618264914241/info:age/1472196193913/Put/vlen=2/seqid=0, Value: 30
     SKIP Cell: 524382618264914241/info:height/1472196194783/Put/vlen=3/seqid=0, Value: 168
     SKIP Cell: 524382618264914241/info:name/1472196193255/Put/vlen=8/seqid=0, Value: zhangsan
     SKIP Cell: 524382618264914241/info:phone/1472196195125/Put/vlen=11/seqid=0, Value: 13212321424
     SKIP Cell: 524382618264914241/info:weight/1472196194970/Put/vlen=3/seqid=0, Value: 168
     SKIP Cell: 524382618264914241/ship:addr/1472196195270/Put/vlen=7/seqid=0, Value: beijing
     SKIP Cell: 524382618264914241/ship:email/1472196195371/Put/vlen=13/seqid=0, Value: [email protected]
     SKIP Cell: 524382618264914241/ship:salary/1472196195485/Put/vlen=4/seqid=0, Value: 3000
     SKIP Cell: 813782218261011172/info:age/1472196212550/Put/vlen=2/seqid=0, Value: 19
     SKIP Cell: 813782218261011172/info:height/1472196212605/Put/vlen=3/seqid=0, Value: 158
     SKIP Cell: 813782218261011172/info:name/1472196212480/Put/vlen=8/seqid=0, Value: wangmazi
     SKIP Cell: 813782218261011172/info:phone/1472196212713/Put/vlen=11/seqid=0, Value: 12713921424
     SKIP Cell: 813782218261011172/info:weight/1472196212651/Put/vlen=3/seqid=0, Value: 118
     SKIP Cell: 813782218261011172/ship:addr/1472196212762/Put/vlen=4/seqid=0, Value: xian
     SKIP Cell: 813782218261011172/ship:email/1472196212802/Put/vlen=12/seqid=0, Value: [email protected]
     SKIP Cell: 813782218261011172/ship:salary/1472196212840/Put/vlen=5/seqid=0, Value: 10000
    **/

    　　

  • WhileMatchFilter 當匹配到就中斷掃描
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.hbase.Cell;
    import org.apache.hadoop.hbase.HBaseConfiguration;
    import org.apache.hadoop.hbase.TableName;
    import org.apache.hadoop.hbase.client.*;
    import org.apache.hadoop.hbase.filter.*;
    import org.apache.hadoop.hbase.util.Bytes;
     
    import java.io.IOException;
     
    /**
     * WhileMatchFilter 全匹配過濾器 [當匹配到就中斷掃描]
     * similarface
     * [email protected]
     */
    public class FilterOfWhileMatchFilter {
        public static void main(String args[]) throws IOException{
            Configuration configuration = HBaseConfiguration.create();
            Connection connection = ConnectionFactory.createConnection(configuration);
            Table table = connection.getTable(TableName.valueOf("user"));
            //匹配 510824118261011172 的行
            Filter filter1=new RowFilter(CompareFilter.CompareOp.NOT_EQUAL,new BinaryComparator(Bytes.toBytes("510824118261011172")));
            Scan scan = new Scan();
            scan.setFilter(filter1);
            ResultScanner results = table.getScanner(scan);
            for (Result result:results) {
                for (Cell cell : result.rawCells()) {
                    System.out.println("Cell: " + cell + ", Value: " +
                            Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                    cell.getValueLength()));
                }
            }
            //  匹配 510824118261011172 的行馬上中斷本次掃描操作
            Filter filter2 = new WhileMatchFilter(filter1);
            Scan scan1 = new Scan();
            scan1.setFilter(filter2);
            ResultScanner scanner2= table.getScanner(scan1);
            for(Result result:scanner2){
                for (Cell cell : result.rawCells()) {
                    System.out.println("WhileMatchFilter Cell: " + cell + ", Value: " +
                            Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                    cell.getValueLength()));
                }
            }
            results.close();
            scanner2.close();
            table.close();
            connection.close();
        }
    }
     
    /**
     Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
     Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
     ...
     Cell: 524382618264914241/ship:addr/1472196195270/Put/vlen=7/seqid=0, Value: beijing
     Cell: 524382618264914241/ship:email/1472196195371/Put/vlen=13/seqid=0, Value: [email protected]
     ...
     Cell: 813782218261011172/ship:salary/1472196212840/Put/vlen=5/seqid=0, Value: 10000
     WhileMatchFilter Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
     ...
     WhileMatchFilter Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
    **/

     

  • FilterList 過濾列表［多個過濾器一起起作用］
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    94
    95
    96
    97
    98
    99
    100
    101
    102
    103
    104
    105
    106
    107
    108
    109
    110
    111
    112
    113
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.hbase.Cell;
    import org.apache.hadoop.hbase.HBaseConfiguration;
    import org.apache.hadoop.hbase.TableName;
    import org.apache.hadoop.hbase.client.*;
    import org.apache.hadoop.hbase.filter.*;
    import org.apache.hadoop.hbase.util.Bytes;
    import java.io.IOException;
    import java.util.ArrayList;
    import java.util.List;
     
    /**
     * FilterList 過濾器列表[多個過濾器組合在一起]
     * similarface
     * [email protected]
     */
    public class FilterOfFilterList {
        public static void main(String args[]) throws IOException{
            Configuration configuration = HBaseConfiguration.create();
            Connection connection = ConnectionFactory.createConnection(configuration);
            Table table = connection.getTable(TableName.valueOf("user"));
            List<Filter> filters = new ArrayList<Filter>();
            Filter filter1=new RowFilter(CompareFilter.CompareOp.GREATER_OR_EQUAL,
                    new BinaryComparator(Bytes.toBytes("524382618264914241")));
            filters.add(filter1);
     
            Filter filter2=new RowFilter(CompareFilter.CompareOp.LESS_OR_EQUAL,
                    new BinaryComparator(Bytes.toBytes("813782218261011172")));
     
            filters.add(filter2);
            //列名過濾器
            Filter filter3=new QualifierFilter(CompareFilter.CompareOp.EQUAL,
                    new RegexStringComparator("age"));
     
            filters.add(filter3);
            //行鍵 Between "524382618264914241" and "813782218261011172"  and column="age"
            FilterList filterList1 = new FilterList(filters);
     
            Scan scan = new Scan();
            scan.setFilter(filterList1);
     
            ResultScanner results = table.getScanner(scan);
            for (Result result:results) {
                for (Cell cell : result.rawCells()) {
                    System.out.println("Cell: " + cell + ", Value: " +
                            Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                    cell.getValueLength()));
                }
            }
            results.close();
            //行鍵 Between "524382618264914241" or "813782218261011172"  or column="age"
            FilterList filterList2 = new FilterList(FilterList.Operator.MUST_PASS_ONE,filters);
            scan.setFilter(filterList2);
            ResultScanner scanner2= table.getScanner(scan);
            for(Result result:scanner2){
                for (Cell cell : result.rawCells()) {
                    System.out.println("MUST_PASS_ONE Cell: " + cell + ", Value: " +
                            Bytes.toString(cell.getValueArray(), cell.getValueOffset(),
                                    cell.getValueLength()));
                }
            }
            scanner2.close();
            table.close();
            connection.close();
        }
    }
     
    /**
     
     Cell: 524382618264914241/info:age/1472196193913/Put/vlen=2/seqid=0, Value: 30
     Cell: 673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0, Value: 19
     Cell: 813782218261011172/info:age/1472196212550/Put/vlen=2/seqid=0, Value: 19
     MUST_PASS_ONE Cell: 224382618261914241/info:age/1472196211169/Put/vlen=2/seqid=0, Value: 24
     MUST_PASS_ONE Cell: 224382618261914241/info:height/1472196211234/Put/vlen=3/seqid=0, Value: 158
     MUST_PASS_ONE Cell: 224382618261914241/info:name/1472196211088/Put/vlen=4/seqid=0, Value: lisi
     MUST_PASS_ONE Cell: 224382618261914241/info:phone/1472196211427/Put/vlen=11/seqid=0, Value: 13213921424
     MUST_PASS_ONE Cell: 224382618261914241/info:weight/1472196211386/Put/vlen=3/seqid=0, Value: 128
     MUST_PASS_ONE Cell: 224382618261914241/ship:addr/1472196211487/Put/vlen=7/seqid=0, Value: chengdu
     MUST_PASS_ONE Cell: 224382618261914241/ship:email/1472196211530/Put/vlen=11/seqid=0, Value: [email protected]
     MUST_PASS_ONE Cell: 224382618261914241/ship:salary/1472196211594/Put/vlen=4/seqid=0, Value: 5000
     MUST_PASS_ONE Cell: 510824118261011172/info:age/1472196213020/Put/vlen=2/seqid=0, Value: 18
     MUST_PASS_ONE Cell: 510824118261011172/info:height/1472196213056/Put/vlen=3/seqid=0, Value: 188
     MUST_PASS_ONE Cell: 510824118261011172/info:name/1472196212942/Put/vlen=8/seqid=0, Value: yangyang
     MUST_PASS_ONE Cell: 510824118261011172/info:phone/1472196213237/Put/vlen=11/seqid=0, Value: 18013921626
     MUST_PASS_ONE Cell: 510824118261011172/info:weight/1472196213169/Put/vlen=3/seqid=0, Value: 138
     MUST_PASS_ONE Cell: 510824118261011172/ship:addr/1472196213328/Put/vlen=8/seqid=0, Value: shanghai
     MUST_PASS_ONE Cell: 510824118261011172/ship:email/1472196213422/Put/vlen=12/seqid=0, Value: [email protected]
     MUST_PASS_ONE Cell: 510824118261011172/ship:salary/1472196214963/Put/vlen=5/seqid=0, Value: 50000
     MUST_PASS_ONE Cell: 524382618264914241/info:age/1472196193913/Put/vlen=2/seqid=0, Value: 30
     MUST_PASS_ONE Cell: 524382618264914241/info:height/1472196194783/Put/vlen=3/seqid=0, Value: 168
     MUST_PASS_ONE Cell: 524382618264914241/info:name/1472196193255/Put/vlen=8/seqid=0, Value: zhangsan
     MUST_PASS_ONE Cell: 524382618264914241/info:phone/1472196195125/Put/vlen=11/seqid=0, Value: 13212321424
     MUST_PASS_ONE Cell: 524382618264914241/info:weight/1472196194970/Put/vlen=3/seqid=0, Value: 168
     MUST_PASS_ONE Cell: 524382618264914241/ship:addr/1472196195270/Put/vlen=7/seqid=0, Value: beijing
     MUST_PASS_ONE Cell: 524382618264914241/ship:email/1472196195371/Put/vlen=13/seqid=0, Value: [email protected]
     MUST_PASS_ONE Cell: 524382618264914241/ship:salary/1472196195485/Put/vlen=4/seqid=0, Value: 3000
     MUST_PASS_ONE Cell: 673782618261019142/info:age/1472196211733/Put/vlen=2/seqid=0, Value: 19
     MUST_PASS_ONE Cell: 673782618261019142/info:height/1472196211761/Put/vlen=3/seqid=0, Value: 178
     MUST_PASS_ONE Cell: 673782618261019142/info:name/1472196211678/Put/vlen=7/seqid=0, Value: zhaoliu
     MUST_PASS_ONE Cell: 673782618261019142/info:phone/1472196211956/Put/vlen=11/seqid=0, Value: 17713921424
     MUST_PASS_ONE Cell: 673782618261019142/info:weight/1472196211841/Put/vlen=3/seqid=0, Value: 188
     MUST_PASS_ONE Cell: 673782618261019142/ship:addr/1472196212059/Put/vlen=8/seqid=0, Value: shenzhen
     MUST_PASS_ONE Cell: 673782618261019142/ship:email/1472196212176/Put/vlen=12/seqid=0, Value: [email protected]
     MUST_PASS_ONE Cell: 673782618261019142/ship:salary/1472196212284/Put/vlen=4/seqid=0, Value: 8000
     MUST_PASS_ONE Cell: 813782218261011172/info:age/1472196212550/Put/vlen=2/seqid=0, Value: 19
     MUST_PASS_ONE Cell: 813782218261011172/info:height/1472196212605/Put/vlen=3/seqid=0, Value: 158
     MUST_PASS_ONE Cell: 813782218261011172/info:name/1472196212480/Put/vlen=8/seqid=0, Value: wangmazi
     MUST_PASS_ONE Cell: 813782218261011172/info:phone/1472196212713/Put/vlen=11/seqid=0, Value: 12713921424
     MUST_PASS_ONE Cell: 813782218261011172/info:weight/1472196212651/Put/vlen=3/seqid=0, Value: 118
     MUST_PASS_ONE Cell: 813782218261011172/ship:addr/1472196212762/Put/vlen=4/seqid=0, Value: xian
     MUST_PASS_ONE Cell: 813782218261011172/ship:email/1472196212802/Put/vlen=12/seqid=0, Value: [email protected]
     MUST_PASS_ONE Cell: 813782218261011172/ship:salary/1472196212840/Put/vlen=5/seqid=0, Value: 10000
    **/

發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章

CDH配置Kerberos和Sentry詳解

1.安全之Kerberos安全認證 1 Kerberos概述 1.1 什麼是Kerberos Kerberos是一種計算機網絡授權協議，用來在非安全網絡中，對個人通信以安全的手段進行身份認證。這個詞又指麻省理工學院爲這個協議開發的一套計算

原創 2024-05-20 21:36:31

高效調度新篇章：詳解DolphinScheduler 3.2.0生產級集羣搭建

轉載自tuoluzhe8521 導讀：通過簡化複雜的任務依賴關係， DolphinScheduler爲數據工程師提供了強大的工作流程管理和調度能力。在3.2.0版本中，DolphinScheduler帶來了一系列新功能和改進，使其在生產環

原創 2024-05-15 21:22:54

14 hive安裝

1.hive 1.1  解壓        tar -zxf apache-hive-1.2.2-bin.tar.gz -C ..        mv apache-hive-1.2.2-bin/ hive-1.2.2  1.2 修改配置

osc_pjx77r92 2024-05-14 00:41:14

Impala數據文件的碎碎念

Impala目前支持Hadoop中幾種常見的文件格式 Parquet 、 ORC 、 Text 、 Avro 、 RCFile 和 SequenceFile 。下面簡要說明各種格式的使用、限制和一些注意事項。 不同的文件格式有着不同的適用場

原創 2024-05-12 21:38:18

O2OA(翱途)開發平臺爲待辦創建配置郵件通知

O2OA(翱途)開發平臺[下稱O2OA開發平臺或者O2OA]的消息機制允許開發者通過配置實現郵件通知。本篇主要闡述如何實現待辦消息的郵件通知。   一、先決條件： 1、O2Server服務器正常運行，系統安裝部署請參考

原創 2024-05-09 23:10:17

從零開始學架構V2-初識架構設計-1

一、架構設計的主要目的 爲了解決軟件系統複雜度帶來的問題 二、複雜性來源 軟件的架構設計是一個非常複雜的過程；基於業務&技術現狀、公司成本、團隊規模、團隊技術能力、近三年業務發展規模預測、技術發展趨勢等條件篩選出合適的技術、編寫多種架構設計

原創 2024-04-25 23:56:25

利用 Amazon EMR Serverless、Amazon Athena、Apache Dolphinscheduler 以及本地 TiDB 和 HDFS 在混合部署環境中構建無服務器數據倉庫

引言 在數據驅動的世界中，企業正在尋求可靠且高性能的解決方案來管理其不斷增長的數據需求。本系列博客從一個重視數據安全和合規性的 B2C 金融科技客戶的角度來討論雲上雲下混合部署的情況下如何利用亞馬遜雲科技雲原生服務、開源社區產品以及第三方

原創 2024-04-25 21:18:23

2024 開源數據工程生態系統全景圖

點擊藍字 關注我們 作者 | ALIREZA SADEGHI翻譯 | Debra Chen 01 簡介

微衆開源 2024-04-23 21:30:36

告別手動調度，海豚調度器 3.1.x 集羣部署讓你輕鬆管理多機！

轉載自第一片心意 1 前言 由於海豚調度器官網的集羣部署文檔寫的較亂，安裝過程中需要跳轉到很多地方進行操作，所以自己總結了一篇可以直接跟着從頭到尾進行操作的文檔，以方便後續的部署、升級、新增節點、減少節點的相關操作。 2. 提前準備 2.

原創 2024-04-23 21:18:20

入職3年-我如何做一名AI產品經理

引言 從2021年校招加入京東開始，我一直從事AI產品經理的工作，有幸見證了AI行業的熱情從一臺臺服務器燒到了全世界各個角落，也見證了京東AI中臺團隊的影響力如何一步步的擴大。從21年的迷茫到24年的堅定，很慶幸我正走在適合自己的道路上，

原創 2024-04-22 11:16:31

Hive引擎底層初探

1、什麼是Hive Hive是一個基於Hadoop的數據倉庫工具,用於處理和分析大規模結構化數據。Hive提供了類似SQL的查詢語言(HiveQL)，使得熟悉SQL的用戶能夠查詢數據。Hive將SQL查詢轉換爲MapReduce任務，以在

原創 2024-04-17 11:18:21

用海豚調度器定時調度從Kafka到HDFS的kettle任務腳本

在實際項目中，從Kafka到HDFS的數據是每天自動生成一個文件，按日期區分。而且Kafka在不斷生產數據，因此看看kettle是不是需要時刻運行？能不能按照每日自動生成數據文件？ 爲了測試實際項目中的海豚定時調度從Kafka到HDFS的K

原創 2024-04-15 21:18:44

解密數倉的SQL ON ANYWHERE技術

本文分享自華爲雲社區《GaussDB DWS的SQL ON ANYWHERE技術解密》，作者：tooooooooooomy。 1. 前言 適用版本：【8.1.1（及以上）】 查詢分析是大數據要解決的核心問題之一，雖然大數據相關的處理引

原創 2024-04-03 10:32:41

Apache DolphinScheduler-3.2.0集羣部署教程

集羣部署方案(2 Master + 3 Worker) Apache DolphinScheduler官網：https://dolphinscheduler.apache.org/zh-cn Apache DolphinScheduler

原創 2024-03-11 21:25:56

Apache Linkis 1.3.0 適配 華爲MRS+Scriptis 實戰分享

一、概述 團隊有需求要在頁面上同時使用sql和python語法對數據進行分析，在調研過程中發現linkis可以滿足需要，遂將其引入內網，由於使用的是華爲MRS，與開源的軟件有所不同， 又進行了二次開發適配，本文將分享使用經驗，希望對有需

微衆開源 2024-02-23 21:45:28