（1）數倉搭建1

第3章數倉搭建
3.0配置Hadoop支持Snappy壓縮
1）將編譯後支持Snappy壓縮的Hadoop jar包解壓縮，並將lib/native目錄中所有文件上傳到hadoop102的/opt/module/hadoop-2.7.2/lib/native目錄，並分發到hadoop103 hadoop104。
2）重新啓動Hadoop。
3）檢查支持的壓縮方式
[atguigu@hadoop102 native]$ hadoop checknative
hadoop: true /opt/module/hadoop-2.7.2/lib/native/libhadoop.so
zlib: true /lib64/libz.so.1
snappy: true /opt/module/hadoop-2.7.2/lib/native/libsnappy.so.1
lz4: true revision:99
bzip2: false

hadoop@hadoop102 ~]$cd /opt/module/hadoop-2.7.2/lib/native/ [hadoop@hadoop102 /opt/module/hadoop-2.7.2/lib/native]$ll
total 4372
-rw-r–r--. 1 hadoop hadoop 1122774 May 22 2017 libhadoop.a
-rw-r–r--. 1 hadoop hadoop 1487268 May 22 2017 libhadooppipes.a
lrwxrwxrwx. 1 hadoop hadoop 18 Nov 9 15:33 libhadoop.so -> libhadoop.so.1.0.0
-rwxr-xr-x. 1 hadoop hadoop 673164 May 22 2017 libhadoop.so.1.0.0
-rw-r–r--. 1 hadoop hadoop 581984 May 22 2017 libhadooputils.a
-rw-r–r--. 1 hadoop hadoop 364796 May 22 2017 libhdfs.a
lrwxrwxrwx. 1 hadoop hadoop 16 Nov 9 15:33 libhdfs.so -> libhdfs.so.0.0.0
-rwxr-xr-x. 1 hadoop hadoop 229017 May 22 2017 libhdfs.so.0.0.0
[hadoop@hadoop102 /opt/module/hadoop-2.7.2/lib/native]$rz rz waiting to receive. [hadoop@hadoop102 /opt/module/hadoop-2.7.2/lib/native]$ll
total 5512
-rw-r–r--. 1 hadoop hadoop 1122774 May 22 2017 libhadoop.a
-rw-r–r--. 1 hadoop hadoop 1487268 May 22 2017 libhadooppipes.a
lrwxrwxrwx. 1 hadoop hadoop 18 Nov 9 15:33 libhadoop.so -> libhadoop.so.1.0.0
-rwxr-xr-x. 1 hadoop hadoop 673164 May 22 2017 libhadoop.so.1.0.0
-rw-r–r--. 1 hadoop hadoop 581984 May 22 2017 libhadooputils.a
-rw-r–r--. 1 hadoop hadoop 364796 May 22 2017 libhdfs.a
lrwxrwxrwx. 1 hadoop hadoop 16 Nov 9 15:33 libhdfs.so -> libhdfs.so.0.0.0
-rwxr-xr-x. 1 hadoop hadoop 229017 May 22 2017 libhdfs.so.0.0.0
-rw-r–r--. 1 hadoop hadoop 472950 Nov 11 18:14 libsnappy.a
-rw-r–r--. 1 hadoop hadoop 955 Nov 11 18:14 libsnappy.la
-rw-r–r--. 1 hadoop hadoop 228177 Nov 11 18:14 libsnappy.so
-rw-r–r--. 1 hadoop hadoop 228177 Nov 11 18:13 libsnappy.so.1
-rw-r–r--. 1 hadoop hadoop 228177 Nov 11 18:13 libsnappy.so.1.3.0
[hadoop@hadoop102 /opt/module/hadoop-2.7.2/lib/native]$xsync /opt/module/hadoop-2.7.2/lib/native
fname=native
3.1 業務數據生成
3.1.1建表語句

1）通過SQLyog創建數據庫gmall

2）設置數據庫編碼

3）導入建表語句（1建表腳本）

1.3電商表結構

1418 - This function has none of DETERMINISTIC, NO SQL, or READS SQL DATA in its declaration and binary logging is enabled (you might want to use the less safe log_bin_trust_function_creators variable)查詢：

在MySQL中創建函數時出現這種錯誤的解決方法：
set global log_bin_trust_function_creators=TRUE;
flush privileges;

3.1.2 生成業務數據

1）生成業務數據函數說明
init_data ( do_date_string VARCHAR(20) ,order_incr_num INT,user_incr_num INT ,sku_num INT ,if_truncate BOOLEAN )：
參數一：do_date_string生成數據日期
參數二：order_incr_num訂單id個數
參數三：user_incr_num用戶id個數
參數四：sku_num商品sku個數
參數五：if_truncate是否刪除數據
2）案例測試：
（1）需求：生成日期2019年2月10日數據、訂單1000個、用戶200個、商品sku300個、刪除原始數據。

CALL init_data(‘2019-02-10’,1000,200,300,TRUE);
（2）查詢生成數據結果
SELECT * from base_category1;
SELECT * from base_category2;
SELECT * from base_category3;

SELECT * from order_info;
SELECT * from order_detail;

SELECT * from sku_info;
SELECT * from user_info;

SELECT * from payment_info;