9（20）本周回流用户数20

第11章 需求五：本周回流用户数
本周回流=本周活跃-本周新增-上周活跃
11.1 DWS层
使用日活明细表dws_uv_detail_day作为DWS层数据
11.2 ADS层
1）建表语句
hive (gmall)>
drop table if exists ads_back_count;
create external table ads_back_count(
dt string COMMENT ‘统计日期’,
wk_dt string COMMENT ‘统计日期所在周’,
wastage_count bigint COMMENT ‘回流设备数’
)
row format delimited fields terminated by ‘\t’
location ‘/warehouse/gmall/ads/ads_back_count’;
2）导入数据：
hive (gmall)>
insert into table ads_back_count
select
‘2019-02-20’ dt,
concat(date_add(next_day(‘2019-02-20’,‘MO’),-7),’’,date_add(next_day(‘2019-02-20’,‘MO’),-1)) wk_dt,
count(*)
from
(
select t1.mid_id
from
(
select mid_id
from dws_uv_detail_wk
where wk_dt=concat(date_add(next_day(‘2019-02-20’,‘MO’),-7),’’,date_add(next_day(‘2019-02-20’,‘MO’),-1))
)t1
left join
(
select mid_id
from dws_new_mid_day
where create_date<=date_add(next_day(‘2019-02-20’,‘MO’),-1) and create_date>=date_add(next_day(‘2019-02-20’,‘MO’),-7)
)t2
on t1.mid_id=t2.mid_id
left join
(
select mid_id
from dws_uv_detail_wk
where wk_dt=concat(date_add(next_day(‘2019-02-20’,‘MO’),-7*2),’_’,date_add(next_day(‘2019-02-20’,‘MO’),-7-1))
)t3
on t1.mid_id=t3.mid_id
where t2.mid_id is null and t3.mid_id is null
)t4;
3）查询结果
hive (gmall)> select * from ads_back_count;
11.3 编写脚本
1）在hadoop102的/home/atguigu/bin目录下创建脚本
[atguigu@hadoop102 bin]$ vim ads_back_log.sh
在脚本中编写如下内容
#!/bin/bash

if [ -n “$1” ];then
do_date=$1
else
do_date=date -d "-1 day" +%F
fi

hive=/opt/module/hive/bin/hive
APP=gmall

echo “-----------导入日期$do_date-----------”

sql="
insert into table "$APP".ads_back_count select '$do_date’ dt,
concat(date_add(next_day(‘KaTeX parse error: Expected group after '_' at position 21: …te','MO'),-7),'_̲',date_add(next…do_date’,‘MO’),-1)) wk_dt,
count()
from
(
select t1.mid_id
from
(
select mid_id
from “$APP".dws_uv_detail_wk where wk_dt=concat(date_add(next_day('$do_date’,‘MO’),-7),’_’,date_add(next_day('$do_date','MO'),-1)) )t1 left join ( select mid_id from "$APP”.dws_new_mid_day
where create_date<=date_add(next_day(‘$do_date','MO'),-1) and create_date>=date_add(next_day('$do_date’,‘MO’),-7)
)t2
on t1.mid_id=t2.mid_id
left join
(
select mid_id
from "$APP".dws_uv_detail_wk where wk_dt=concat(date_add(next_day('$do_date’,‘MO’),-72),’_’,date_add(next_day(’$do_date’,‘MO’),-7-1))
)t3
on t1.mid_id=t3.mid_id
where t2.mid_id is null and t3.mid_id is null
)t4;
"

$hive -e "$sql"
2）增加脚本执行权限
[atguigu@hadoop102 bin]$ chmod 777 ads_back_log.sh
3）脚本使用
[atguigu@hadoop102 module]$ ads_back_log.sh 2019-02-20
4）查询结果
hive (gmall)> select * from ads_back_count;
5）脚本执行时间
企业开发中一般在每周一凌晨30分~1点