hive 去掉重複數據，顯示第一條
例如：
name adx tran_id cost ts
ck 5 125.168.10.0 33.00 1407234660
ck 5 187.18.99.00 33.32 1407234661
ck 5 125.168.10.0 33.24 1407234661
第三行的tran_id和第一行的重複需要刪除
1.同一張表使用兩次，用第一個表的distinct id join第二個就可以得到了
select t1.tran_id ,t2.*
from
(select distinct tran_id from table) t1
join
table t2
on t1.tran_id=t2.tran_id
2.使用row_number
select * from (select *,row_number() over (partition by tran_idorder by timestamp asc) num from table) t where t.num=1;
row_number() over (partition by tran_idorder by timestamp desc) num
取num=1 就是取 group by 按timestamp 排序的第一條數據
按每個 guid group 然後 按timestamp 排序 然後 加行標
ROW_NUMBER() OVER函數的基本用法
語法：ROW_NUMBER() OVER(PARTITION BY COLUMN ORDER BY COLUMN)
簡單的說row_number()從1開始，爲每一條分組記錄返回一個數字，這裏的ROW_NUMBER() OVER (ORDER BY xlh DESC) 是先把xlh列降序，再爲降序以後的沒條xlh記錄返回一個序號。
示例：
xlh row_num
1700 1
1500 2
1085 3
710 4
row_number() OVER (PARTITION BY COL1 ORDER BY COL2) 表示根據COL1分組，在分組內部根據 COL2排序，而此函數計算的值就表示每組內部排序後的順序編號（組內連續的唯一的)
實例：
初始化數據
create table employee (empid int ,deptid int ,salary decimal(10,2))
insert into employee values(1,10,5500.00)
insert into employee values(2,10,4500.00)
insert into employee values(3,20,1900.00)
insert into employee values(4,20,4800.00)
insert into employee values(5,40,6500.00)
insert into employee values(6,40,14500.00)
insert into employee values(7,40,44500.00)
insert into employee values(8,50,6500.00)
insert into employee values(9,50,7500.00)
數據顯示爲
empid deptid salary
1 10 5500.00
2 10 4500.00
3 20 1900.00
4 20 4800.00
5 40 6500.00
6 40 14500.00
7 40 44500.00
8 50 6500.00
9 50 7500.00
需求：根據部門分組，顯示每個部門的工資等級
預期結果：
empid deptid salary rank
1 10 5500.00 1
2 10 4500.00 2
4 20 4800.00 1
3 20 1900.00 2
7 40 44500.00 1
6 40 14500.00 2
5 40 6500.00 3
9 50 7500.00 1
8 50 6500.00 2
SQL腳本：
SELECT *, Row_Number() OVER (partition by deptid ORDER BY salary desc) rank FROM employee