hive - 解析 json

內置函數：get_json_object(json串,解析路徑)

解析路徑說明：

$ :跟對象

. :子對象

[] :數組下標

* :所有

舉例：

數據樣例：

{"movie":"1190","rate":"4.8","timestamp":"978300760","uid":"145325"}

{"movie":"1191","rate":"3.7","timestamp":"978200613","uid":"234566"}

{"movie":"1190","rate":"4.5","timestamp":"977500902","uid":"474532"}

...

將數據導入hive：

create table rating(line string);#建表，只有一列數據 load data local inpath '/user/rating.json' into table rating;#從本地導數據

 

1)求評分次數排名前3的電影

SELECT get_json_object(t.line, '$.movie') as movieid, count(*) as tatol FROM rating t GROUP BY 1 ORDER BY 2 LIMIT 3;

• t.line是要解析的json串，
• $定位到第一級目錄{
• .定位到第二級目錄"movie":"1190","rate":"4.8","timestamp":"978300760","uid":"145325"
• movie：通過key定位到value值1190

 

2)將整個json文件解析成一張表

1)建一個用於存放數據的表

create table rate(movie int,rate float,time bigint,uid bigint);

 

2)解析並加載數據

insert into table rate select get_json_object(t.line, '$.movie') as movie, get_json_object(t.line, '$.rate') as rate, get_json_object(t.line, '$.timestamp') as time, get_json_object(t.line, '$.uid') as uid FROM rating t;

 

3)解析較複雜json

{

"status": 0,

"data": {"search_data":[{

"name": "奈良市",

"location": {

"lat": 34.685087

}]

}

1-解析status

get_json_object(t.line, '$.status');

 

2-解析name

get_json_object(t.line, '$.data.search_data.[0].name');

 

3-解析lat

get_json_object(t.line, '$.data.search_data.[0].location.lat');