hive讀取json數組並轉換成多行(列轉行)

背景:在讀取hive表中某一些字段的時候,有的json字符串其中會包括數組,那麼想要讀取這個數組並且轉換爲多行該怎麼操作那?
操作:
1、數據舉例:
["[{\"pet_skill_avg_level\":0,\"pet_guard_star\":0,\"pet_type\":0,\"pet_step\":0,\"pet_skill_num\":0,\"pet_adv_score\":0,\"pet_level\":0,\"pet_fight_star\":0,\"pet_id\":\"0\"},{\"pet_skill_avg_level\":0,\"pet_guard_star\":0,\"pet_type\":0,\"pet_step\":0,\"pet_skill_num\":0,\"pet_adv_score\":0,\"pet_level\":0,\"pet_fight_star\":0,\"pet_id\":\"0\"},{\"pet_skill_avg_level\":0,\"pet_guard_star\":0,\"pet_type\":0,\"pet_step\":0,\"pet_skill_num\":0,\"pet_adv_score\":0,\"pet_level\":0,\"pet_fight_star\":0,\"pet_id\":\"0\"},{\"pet_skill_avg_level\":0,\"pet_guard_star\":0,\"pet_type\":0,\"pet_step\":0,\"pet_skill_num\":0,\"pet_adv_score\":0,\"pet_level\":0,\"pet_fight_star\":0,\"pet_id\":\"0\"},{\"pet_skill_avg_level\":0,\"pet_guard_star\":0,\"pet_type\":0,\"pet_step\":0,\"pet_skill_num\":0,\"pet_adv_score\":0,\"pet_level\":0,\"pet_fight_star\":0,\"pet_id\":\"0\"}]"]

這是其中一個字段的json串,格式是一個數組
在這裏插入圖片描述
最終想要的是數組中的五條數據分成五行。
實現方法:

SELECT
    col
FROM
    (
        select
            split(
                regexp_replace(regexp_extract(params ['pet_info'],'^\\[(.+)\\]$',1 ), 
                '\\}\\,\\{', '\\}\\|\\|\\{'),
                '\\|\\|'
            ) AS pet_info
        from
            db_a.dwd_event_log
        where
            p_date = '${DATE}'
            and app_id = 165018
            and event = 'pet_flow'
            -- 只取一條進行查詢
        LIMIT 1
    ) info lateral view explode(info.pet_info) ss as col

步驟解釋:
1、regexp_extract(params [‘pet_info’],’^\[(.+)\]$’,1
將該字段的數據 清洗成這個格式

{"pet_skill_avg_level":0,"pet_guard_star":0,"pet_type":0,"pet_step":0,"pet_skill_num":0,"pet_adv_score":0,"pet_level":0,"pet_fight_star":0,"pet_id":"0"},{"pet_skill_avg_level":0,"pet_guard_star":0,"pet_type":0,"pet_step":0,"pet_skill_num":0,"pet_adv_score":0,"pet_level":0,"pet_fight_star":0,"pet_id":"0"},{"pet_skill_avg_level":0,"pet_guard_star":0,"pet_type":0,"pet_step":0,"pet_skill_num":0,"pet_adv_score":0,"pet_level":0,"pet_fight_star":0,"pet_id":"0"},{"pet_skill_avg_level":0,"pet_guard_star":0,"pet_type":0,"pet_step":0,"pet_skill_num":0,"pet_adv_score":0,"pet_level":0,"pet_fight_star":0,"pet_id":"0"},{"pet_skill_avg_level":0,"pet_guard_star":0,"pet_type":0,"pet_step":0,"pet_skill_num":0,"pet_adv_score":0,"pet_level":0,"pet_fight_star":0,"pet_id":"0"}

2、將上面的格式每個之間的間隔符換成 ‘||’ (注意轉義)
3、用 split 將字符串拆分成數組
4、用 lateral view explode 行轉列,轉換成多行
效果:
在這裏插入圖片描述

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章