背景:在讀取hive表中某一些字段的時候,有的json字符串其中會包括數組,那麼想要讀取這個數組並且轉換爲多行該怎麼操作那?
操作:
1、數據舉例:
["[{\"pet_skill_avg_level\":0,\"pet_guard_star\":0,\"pet_type\":0,\"pet_step\":0,\"pet_skill_num\":0,\"pet_adv_score\":0,\"pet_level\":0,\"pet_fight_star\":0,\"pet_id\":\"0\"},{\"pet_skill_avg_level\":0,\"pet_guard_star\":0,\"pet_type\":0,\"pet_step\":0,\"pet_skill_num\":0,\"pet_adv_score\":0,\"pet_level\":0,\"pet_fight_star\":0,\"pet_id\":\"0\"},{\"pet_skill_avg_level\":0,\"pet_guard_star\":0,\"pet_type\":0,\"pet_step\":0,\"pet_skill_num\":0,\"pet_adv_score\":0,\"pet_level\":0,\"pet_fight_star\":0,\"pet_id\":\"0\"},{\"pet_skill_avg_level\":0,\"pet_guard_star\":0,\"pet_type\":0,\"pet_step\":0,\"pet_skill_num\":0,\"pet_adv_score\":0,\"pet_level\":0,\"pet_fight_star\":0,\"pet_id\":\"0\"},{\"pet_skill_avg_level\":0,\"pet_guard_star\":0,\"pet_type\":0,\"pet_step\":0,\"pet_skill_num\":0,\"pet_adv_score\":0,\"pet_level\":0,\"pet_fight_star\":0,\"pet_id\":\"0\"}]"]
這是其中一個字段的json串,格式是一個數組
最終想要的是數組中的五條數據分成五行。
實現方法:
SELECT
col
FROM
(
select
split(
regexp_replace(regexp_extract(params ['pet_info'],'^\\[(.+)\\]$',1 ),
'\\}\\,\\{', '\\}\\|\\|\\{'),
'\\|\\|'
) AS pet_info
from
db_a.dwd_event_log
where
p_date = '${DATE}'
and app_id = 165018
and event = 'pet_flow'
-- 只取一條進行查詢
LIMIT 1
) info lateral view explode(info.pet_info) ss as col
步驟解釋:
1、regexp_extract(params [‘pet_info’],’^\[(.+)\]$’,1
將該字段的數據 清洗成這個格式
{"pet_skill_avg_level":0,"pet_guard_star":0,"pet_type":0,"pet_step":0,"pet_skill_num":0,"pet_adv_score":0,"pet_level":0,"pet_fight_star":0,"pet_id":"0"},{"pet_skill_avg_level":0,"pet_guard_star":0,"pet_type":0,"pet_step":0,"pet_skill_num":0,"pet_adv_score":0,"pet_level":0,"pet_fight_star":0,"pet_id":"0"},{"pet_skill_avg_level":0,"pet_guard_star":0,"pet_type":0,"pet_step":0,"pet_skill_num":0,"pet_adv_score":0,"pet_level":0,"pet_fight_star":0,"pet_id":"0"},{"pet_skill_avg_level":0,"pet_guard_star":0,"pet_type":0,"pet_step":0,"pet_skill_num":0,"pet_adv_score":0,"pet_level":0,"pet_fight_star":0,"pet_id":"0"},{"pet_skill_avg_level":0,"pet_guard_star":0,"pet_type":0,"pet_step":0,"pet_skill_num":0,"pet_adv_score":0,"pet_level":0,"pet_fight_star":0,"pet_id":"0"}
2、將上面的格式每個之間的間隔符換成 ‘||’ (注意轉義)
3、用 split 將字符串拆分成數組
4、用 lateral view explode 行轉列,轉換成多行
效果: