Hadoop+Hive+HBase+Kylin 僞分佈式安裝指南

Hadoop2.7.6+Mysql5.7+Hive2.3.2+Hbase1.4.9+Kylin2.4單機僞分佈式安裝文檔

注意：

####################################################################
本文檔已經有了最新版本，主要改動地方爲：
1、zookeeper改爲使用安裝的外置zookeeper而非hbase自帶zookeeper ，新加zookeeper安裝過程
2、hbase使用外置zookeeper，配置做相應改動
3、新加kafka相關實施配置來支持Kylin的steam cube的實驗
新版本的文檔連接在這：Hadoop2.7.6+Mysql5.7+Hive2.3.2+zookeeper3.4.6+kafka2.11+Hbase1.4.9+Kylin2.4單機僞分佈式安裝文檔
下面的文檔是原來老版的，適合不用kafka來做steam cube實驗的讀者使用。
####################################################################

最近學習Kylin，肯定需要一個已經安裝好的環境，Kylin的依賴環境官方介紹如下：
依賴於 Hadoop 集羣處理大量的數據集。您需要準備一個配置好 HDFS，YARN，MapReduce,，Hive， HBase，Zookeeper 和其他服務的 Hadoop 集羣供 Kylin 運行。Kylin 可以在 Hadoop 集羣的任意節點上啓動。方便起見，您可以在 master 節點上運行 Kylin。但爲了更好的穩定性，我們建議您將 Kylin 部署在一個乾淨的 Hadoop client 節點上，該節點上 Hive，HBase，HDFS 等命令行已安裝好且 client 配置（如 core-site.xml，hive-site.xml，hbase-site.xml及其他）也已經合理的配置且其可以自動和其它節點同步。運行 Kylin 的 Linux 賬戶要有訪問 Hadoop 集羣的權限，包括創建/寫入 HDFS 文件夾，Hive 表， HBase 表和提交 MapReduce 任務的權限。

軟件要求
Hadoop: 2.7+, 3.1+ (since v2.5)
Hive: 0.13 - 1.2.1+
HBase: 1.1+, 2.0 (since v2.5)
Spark (可選) 2.3.0+
Kafka (可選) 1.0.0+ (since v2.5)
JDK: 1.8+ (since v2.5)
OS: Linux only, CentOS 6.5+ or Ubuntu 16.0.4+

安裝要求知道了，但是hadoop這些東西不太熟悉，小白一個，看了網上一些資料邊看邊學邊做，期間遇到了很多坑！很多人寫的安裝部署文檔要麼是步驟東一塊西一塊，要麼是省略，扔個連接或則說讓自己去百度。在經歷了很多坑之後終於是把完全分佈式的hadoop+mysql+hive+hbase+zookeeper+kylin部署成功了，但是對於日常自己學習測試來說，開多臺虛擬機電腦實在撐不住，於是寫了現在這個僞分佈式的部署文檔給像我一樣初學kylin的小白同學們，後面的環境部署完關於如何將關係型數據庫(我這裏是用oracle）通過sqoop增量導入到hive的實驗可以參考下面貼出的本人文章連接，至於其他相關知識請參考文人其他文章。
1、sqoop1.4.7的安裝及使用(hadoop2.7環境)
2、Oracle通過sqoop同步數據到hive
3、Oracle通過sqoop增量同步數據到hive

環境配置：

目前有兩個測試環境，以Centsos 7系統的安裝爲例子介紹詳細過程，Centos7系統規劃配置清單如下，另外一個測試環境爲RedHat 6 64位系統，安裝過程都差不多，Mysql安裝有些不一樣，不一樣的地方都分別寫了各自的安裝方法，安裝過程中遇到的坑很多
並且都已經解決，不再一一列舉，按照下面步驟是完全可以在Centos 7/Redhat6 64位系統安裝成功的。

一、Centos7安裝

打開vmware，創建新虛擬機安裝Centos 774位系統：






完成後界面如下：

選擇啓動虛擬機，選擇第一個選項回車：

選擇繼續
等待依賴包檢查完成，點擊date&time設置時間

接下來點擊software selection選擇安裝模式，這裏選擇最精簡安裝：

然後點擊done出來之後，等待依賴包檢查完成，然後設置磁盤分區

選擇現在設置：

點擊done後，進入下面所示界面，選擇標準分區，然後設置點擊+號設置分區


最後分好區如下：

然後點擊done後點擊確認

接下來選擇網絡設置

設置hostname,點擊apply。然後選擇configure設置網絡ip


最後done點擊安裝就可以了：

可以在這個界面設置下root密碼，等待安裝完成就可以了。這是虛擬機的安裝，接下來配置linux,安裝軟件。

二、Linux環境配置：

1、linux網絡配置：

（1）因爲Centos 7安裝的精簡模式，先解決linux網絡問題來讓windows能夠用xshell連上,編輯/etc/sysconfig/network-scripts/ifcfg-ens33內容如下：

TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens33
UUID=e8df3ff3-cf86-42cd-b48a-0d43fe85d8a6
DEVICE=ens33
ONBOOT="yes"
IPADDR=192.168.1.66
PREFIX=24
IPV6_PRIVACY=no

（2）重啓網絡

[root@hadoop ~]# service network restart
Restarting network (via systemctl):                        [  OK  ]
重啓後可以通過下面命令來檢查網絡
[root@hadoop ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:0d:f1:ca brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.66/24 brd 192.168.1.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::d458:8497:adb:7f01/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

（3）接下來關閉防火牆

[root@hadoop ~]# systemctl disable firewalld
[root@hadoop ~]# systemctl stop firewalld```

（4）進程守護,關閉selinux

[root@hadoop ~]# setenforce 0
[root@hadoop ~]# vi /etc/selinux/config
[root@hadoop ~]# cat  /etc/selinux/config

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted 
重啓
[root@hadoop ~]# reboot
可以通過下面方式查看是否啓用selinux
sestatus
getenforce 

（5）編輯/etc/hosts加入下面內容

[root@hadoop ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.66 hadoop

2、安裝java

（1）先看下當前linux環境是否有自帶的open jdk：

[root@hadoop ~]# rpm -qa | grep java
[root@hadoop ~]# rpm -qa | grep jdk
[root@hadoop ~]# rpm -qa | grep gcj
沒有，如果有的話要卸載，卸載案例如下：
卸載linux自帶open jdk，將前面三條命令檢查出來的內容一一卸載：
[root@master ~]# rpm -e --nodeps java-1.7.0-openjdk-1.7.0.99-2.6.5.1.0.1.el6.x86_64
[root@master ~]# rpm -e --nodeps tzdata-java-2016c-1.el6.noarch
[root@master ~]# rpm -e java-1.6.0-openjdk-1.6.0.38-1.13.10.4.el6.x86_64
[root@master ~]# rpm -e java-1.7.0-openjdk-1.7.0.99-2.6.5.1.0.1.el6.x86_64
卸載完成後應該再檢查一次

（2）接下來安裝配置java

創建安裝目錄：
[root@hadoop ~]# mkdir -p /usr/java
上傳並解壓jdk到此目錄
[root@hadoop ~]# cd /usr/java/
[root@hadoop java]# ls
jdk-8u151-linux-x64 (1).tar.gz
解壓縮
[root@hadoop java]# tar -zxvf jdk-8u151-linux-x64\ \(1\).tar.gz
[root@hadoop java]# rm -rf jdk-8u151-linux-x64\ \(1\).tar.gz 
[root@hadoop java]# ls
jdk1.8.0_151
編輯/etc/profile
寫入下面jdk環境變量，保存退出
export JAVA_HOME=/usr/java/jdk1.8.0_151
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin
使環境變量生效
[root@master java]# source /etc/profile
檢查安裝是否沒問題
[root@hadoop java]# java -version
java version "1.8.0_151"
Java(TM) SE Runtime Environment (build 1.8.0_151-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.151-b12, mixed mode)

3、配置SSH免密碼登錄

（1）輸入命令，ssh-keygen -t rsa，生成key，都不輸入密碼，一直回車，/root就會生成.ssh文件夾

[root@hadoop ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:+Xxqh8qa2AguQPY4aNJci6YiUWS822NtcLRK/9Kopp8 root@hadoop1
The key's randomart image is:
+---[RSA 2048]----+
| . |
| + . |
| o . . . |
| oo + o . |
|++o* B S |
|=+*.* + o |
|++o. o + o.. |
|=. ..=ooo oo. |
|o.o+E.+ooo.. |
+----[SHA256]-----+     
[root@hadoop ~]# cd .ssh/
[root@hadoop .ssh]# ls
id_rsa id_rsa.pub known_hosts

合併公鑰到authorized_keys文件，在hadoop服務器，進入/root/.ssh目錄，通過SSH命令合併
[root@hadoop .ssh]# cat id_rsa.pub>> authorized_keys
通過下面命令測試
ssh localhost
ssh hadoop
ssh 192.168.1.66

4、安裝Hadoop2.7

(1)下載連接：
http://archive.apache.org/dist/hadoop/core/hadoop-2.7.6/

(2)解壓：

[root@hadoop ~]# cd /hadoop/
[root@hadoop hadoop]# ls
hadoop-2.7.6 (1).tar.gz
[root@hadoop hadoop]# tar -zxvf hadoop-2.7.6\ \(1\).tar.gz ^C
[root@hadoop hadoop]# ls
hadoop-2.7.6  hadoop-2.7.6 (1).tar.gz
[root@hadoop hadoop]# rm -rf *gz
[root@hadoop hadoop]# mv hadoop-2.7.6/* .

(3)在/hadoop目錄下創建數據存放的文件夾，tmp、hdfs、hdfs/data、hdfs/name

[root@hadoop hadoop]# pwd
/hadoop
[root@hadoop hadoop]# mkdir tmp
[root@hadoop hadoop]# mkdir hdfs
[root@hadoop hadoop]# mkdir hdfs/data
[root@hadoop hadoop]# mkdir hdfs/name

（4）配置/hadoop/etc/hadoop目錄下的core-site.xml

[root@hadoop hadoop]# vi etc/hadoop/core-site.xml 
<configuration>
  <property>
        <name>fs.defaultFS</name>
        <value>hdfs://192.168.1.66:9000</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>file:/hadoop/tmp</value>
    </property>
    <property>
        <name>io.file.buffer.size</name>
        <value>131702</value>
    </property>
 </configuration>

（5）配置/hadoop/etc/hadoop/hdfs-site.xml

[root@hadoop hadoop]# vi etc/hadoop/hdfs-site.xml
<configuration>
<property>
        <name>dfs.namenode.name.dir</name>
        <value>file:/hadoop/hdfs/name</value>
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>file:/hadoop/hdfs/data</value>
    </property>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
    <property>
        <name>dfs.namenode.secondary.http-address</name>
        <value>192.168.1.66:9001</value>
    </property>
    <property>
    <name>dfs.webhdfs.enabled</name>
    <value>true</value>
    </property>
 </configuration>

(6)複製etc/hadoop/mapred-site.xml.template爲etc/hadoop/mapred-site.xml，再編輯：

[root@hadoop hadoop]# cd etc/hadoop/
[root@hadoop hadoop]# cp mapred-site.xml.template  mapred-site.xml
[root@hadoop hadoop]# vi mapred-site.xml
<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
    <property>
        <name>mapreduce.jobhistory.address</name>
        <value>192.168.1.66:10020</value>
    </property>
    <property>
        <name>mapreduce.jobhistory.webapp.address</name>
        <value>192.168.1.66:19888</value>
    </property>

</configuration>

（7）配置 etc/hadoop/yarn-site.xml

root@hadoop1 hadoop]# vi yarn-site.xml
<configuration>


<!-- Site specific YARN configuration properties -->
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
    </property>
<property>
    <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
  <property>
    <name>yarn.resourcemanager.hostname</name>
    <value>hadoop</value>
  </property>

</configuration>

（8）配置/hadoop/etc/hadoop/目錄下hadoop-env.sh、yarn-env.sh的JAVA_HOME，不設置的話，啓動不了

[root@hadoop hadoop]# pwd
/hadoop/etc/hadoop
[root@hadoop hadoop]# vi hadoop-env.sh 
將 export JAVA_HOME 改爲：export JAVA_HOME=/usr/java/jdk1.8.0_151
加入
export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/etc/hadoop"}
export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_HOME}/lib/native

[root@hadoop hadoop]# vi yarn-env.sh 
將 export JAVA_HOME 改爲：export JAVA_HOME=/usr/java/jdk1.8.0_151

配置slaves文件

[root@hadoop hadoop]# cat slaves 
localhost

（9）配置hadoop環境變量

[root@hadoop ~]# vim /etc/profile  
寫入下面內容
export HADOOP_HOME=/hadoop/
export HADOOP_CONF_DIR=${HADOOP_HOME}/etc/hadoop
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native 
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib:$HADOOP_COMMON_LIB_NATIVE_DIR"
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

[root@hadoop ~]# source /etc/profile

（10）啓動hadoop

[root@hadoop hadoop]# pwd
/hadoop
[root@hadoop hadoop]#  bin/hdfs namenode -format
.。。。。。。。。。。。。。。。。。。。。。
19/03/04 17:18:00 INFO namenode.FSImage: Allocated new BlockPoolId: BP-774693564-192.168.1.66-1551691079972
19/03/04 17:18:00 INFO common.Storage: Storage directory /hadoop/hdfs/name has been successfully formatted.
19/03/04 17:18:00 INFO namenode.FSImageFormatProtobuf: Saving image file /hadoop/hdfs/name/current/fsimage.ckpt_0000000000000000000 using no compression
19/03/04 17:18:00 INFO namenode.FSImageFormatProtobuf: Image file /hadoop/hdfs/name/current/fsimage.ckpt_0000000000000000000 of size 321 bytes saved in 0 seconds.
19/03/04 17:18:00 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
19/03/04 17:18:00 INFO util.ExitUtil: Exiting with status 0
19/03/04 17:18:00 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at hadoop/192.168.1.66
************************************************************/
全部啓動sbin/start-all.sh，也可以分開sbin/start-dfs.sh、sbin/start-yarn.sh
[root@hadoop hadoop]# sbin/start-dfs.sh
[root@hadoop hadoop]# sbin/start-yarn.sh
停止的話，輸入命令，sbin/stop-all.sh
輸入命令jps，可以看到相關信息:
[root@hadoop hadoop]# jps
10581 ResourceManager
10102 NameNode
10376 SecondaryNameNode
10201 DataNode
10683 NodeManager
11007 Jps

(11)啓動jobhistory

# kylin需要連接jobhistory
 mr-jobhistory-daemon.sh start historyserver
 [root@hadoop hadoop]# jps
33376 NameNode
33857 ResourceManager
33506 DataNode
33682 SecondaryNameNode
33960 NodeManager
34319 JobHistoryServer
34367 Jps

(12)驗證

1)瀏覽器打開http://192.168.1.66:8088/
2)瀏覽器打開http://192.168.1.66:50070/

6、安裝Mysql

需要根據自己的系統版本去下載，下載連接：
https://dev.mysql.com/downloads/mysql/5.7.html#downloads
我這裏下載的是適用我當前本人測試環境Centos 7 64位 的系統，而另一個測試環境10.1.197.241是Redhat 6，兩個測試環境如果安裝時要下載對應系統的rpm包，不然不兼容的rpm包安裝時會報下面的錯誤(比如在Redhat6安裝適用centos7的mysql)：

[root@s197240 hadoop]# rpm -ivh mysql-community-libs-5.7.18-1.el7.x86_64.rpm
warning: mysql-community-libs-5.7.18-1.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY
error: Failed dependencies:
libc.so.6(GLIBC_2.14)(64bit) is needed by mysql-community-libs-5.7.18-1.el7.x86_64

1）檢查卸載mariadb-lib
Centos自帶mariadb數據庫，刪除，安裝mysql

[root@hadoop hadoop]# rpm -qa|grep mariadb
mariadb-libs-5.5.60-1.el7_5.x86_64
[root@hadoop hadoop]# rpm -e mariadb-libs-5.5.60-1.el7_5.x86_64 --nodeps
[root@hadoop hadoop]# rpm -qa|grep mariadb

如果時Redhat6安裝時自帶mysql庫，卸載自帶的包：
通過此命令查找已經安裝的mysql包：

[root@s197240 hadoop]# rpm -qa |grep mysql
mysql-community-common-5.7.18-1.el7.x86_64
通過此命令卸載：
[root@s197240 hadoop]# rpm -e --allmatches --nodeps mysql-community-common-5.7.18-1.el7.x86_64

2）上傳解壓安裝包
下載連接：
https://dev.mysql.com/downloads/file/?id=469456

[root@hadoop mysql]# pwd
/usr/local/mysql
[root@hadoop mysql]# ls
mysql-5.7.18-1.el7.x86_64.rpm-bundle.tar
[root@hadoop mysql]# tar -xvf  mysql-5.7.18-1.el7.x86_64.rpm-bundle.tar 
mysql-community-server-5.7.18-1.el7.x86_64.rpm
mysql-community-embedded-devel-5.7.18-1.el7.x86_64.rpm
mysql-community-devel-5.7.18-1.el7.x86_64.rpm
mysql-community-client-5.7.18-1.el7.x86_64.rpm
mysql-community-common-5.7.18-1.el7.x86_64.rpm
mysql-community-embedded-5.7.18-1.el7.x86_64.rpm
mysql-community-embedded-compat-5.7.18-1.el7.x86_64.rpm
mysql-community-libs-5.7.18-1.el7.x86_64.rpm
mysql-community-server-minimal-5.7.18-1.el7.x86_64.rpm
mysql-community-test-5.7.18-1.el7.x86_64.rpm
mysql-community-minimal-debuginfo-5.7.18-1.el7.x86_64.rpm
mysql-community-libs-compat-5.7.18-1.el7.x86_64.rpm

（3）安裝mysql server
其中，安裝mysql-server, 需要以下幾個必要的安裝包：

mysql-community-client-5.7.17-1.el7.x86_64.rpm（依賴於libs）
mysql-community-common-5.7.17-1.el7.x86_64.rpm （依賴於common）
mysql-community-libs-5.7.17-1.el7.x86_64.rpm
mysql-community-server-5.7.17-1.el7.x86_64.rpm（依賴於common, client）

安裝上面四個包需要libaio和net-tools的依賴,這裏配置好yum源，使用yum安裝，通過以下命令安裝：

yum -y install libaio
yum -y install net-tools

安裝mysql-server:按照common–>libs–>client–>server的順序。若不按照此順序，也會有一定“依賴”關係的提醒。

[root@hadoop mysql]# rpm -ivh mysql-community-common-5.7.18-1.el7.x86_64.rpm 
warning: mysql-community-common-5.7.18-1.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:mysql-community-common-5.7.18-1.e################################# [100%]
[root@hadoop mysql]# rpm -ivh  mysql-community-libs-5.7.18-1.el7.x86_64.rpm 
warning: mysql-community-libs-5.7.18-1.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:mysql-community-libs-5.7.18-1.el7################################# [100%]
[root@hadoop mysql]# rpm -ivh mysql-community-client-5.7.18-1.el7.x86_64.rpm 
warning: mysql-community-client-5.7.18-1.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:mysql-community-client-5.7.18-1.e################################# [100%]
[root@hadoop mysql]# rpm -ivh mysql-community-server-5.7.18-1.el7.x86_64.rpm 
warning: mysql-community-server-5.7.18-1.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:mysql-community-server-5.7.18-1.e################################# [100%]

(4）初始化mysql

[root@hadoop mysql]#  mysqld --initialize

mysql默認安裝在/var/lib下。
(5）更改mysql數據庫所屬於用戶及其所屬於組

[root@hadoop mysql]# chown mysql:mysql /var/lib/mysql -R

(6）啓動mysql數據庫

啓動mysql數據庫
[root@hadoop mysql]# cd /var/lib/mysql
[root@hadoop mysql]# systemctl start mysqld.service
[root@hadoop ~]# cd /var/log/
[root@hadoop log]# grep 'password' mysqld.log 
2019-02-26T04:33:06.989818Z 1 [Note] A temporary password is generated for root@localhost: mxeV&htW-3VC
更改root用戶密碼，新版的mysql在第一次登錄後更改密碼前是不能執行任何命令的
[root@hadoop log]# mysql -u root -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.7.18

Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
更改密碼
mysql> set password=password('oracle');
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
mysql> grant all privileges on *.* to root@'%' identified by 'oracle' with grant option;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

如果是Redhat6系統，啓動mysql數據庫過程如下：

[root@s197240 mysql]# /etc/rc.d/init.d/mysqld start
Starting mysqld:                                           [  OK  ]
[root@s197240 mysql]# ls /etc/rc.d/init.d/mysqld -l
-rwxr-xr-x 1 root root 7157 Dec 21 19:29 /etc/rc.d/init.d/mysqld
[root@s197240 mysql]# chkconfig mysqld on
[root@s197240 mysql]# chmod 755 /etc/rc.d/init.d/mysqld
[root@s197240 mysql]# service mysqld start
Starting mysqld:                                           [  OK  ]
[root@s197240 mysql]# service mysqld status
mysqld (pid  28861) is running...

mysql啓動後，剩餘後面的操作完全按照上面systemctl start mysqld.service步驟下面的過程來就可以了

7、Hive安裝

下載連接：
http://archive.apache.org/dist/hive/hive-2.3.2/
(1）上載和解壓縮

[root@hadoop ~]# mkdir /hadoop/hive
[root@hadoop ~]# cd /hadoop/hive/
[root@hadoop hive]# ls
apache-hive-2.3.3-bin.tar.gz
[root@hadoop hive]# tar -zxvf apache-hive-2.3.3-bin.tar.gz 

(2）配置環境變量

#編輯/etc/profile，添加hive相關的環境變量配置
[root@hadoop hive]# vim /etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_151
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin
export HADOOP_HOME=/hadoop/
export HADOOP_CONF_DIR=${HADOOP_HOME}/etc/hadoop
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib:$HADOOP_COMMON_LIB_NATIVE_DIR"
export HIVE_HOME=/hadoop/hive/
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin
#修改完文件後，執行如下命令，讓配置生效：
[root@hadoop hive]# source /etc/profile

(3)Hive配置Hadoop HDFS
hive-site.xml配置
進入目錄$HIVE_HOME/conf，將hive-default.xml.template文件複製一份並改名爲hive-site.xml

[root@hadoop hive]# cd $HIVE_HOME/conf 
[root@hadoop conf]# cp hive-default.xml.template hive-site.xml

使用hadoop新建hdfs目錄,因爲在hive-site.xml中有如下配置：

<property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
    <description>location of default database for the warehouse</description>
  </property>
  <property>

執行hadoop命令新建/user/hive/warehouse目錄：

#新建目錄/user/hive/warehouse
[root@hadoop1 ~]# $HADOOP_HOME/bin/hadoop dfs -mkdir -p /user/hive/warehouse
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
#給新建的目錄賦予讀寫權限
[root@hadoop1 ~]# cd $HIVE_HOME
[root@hadoop1 hive]# cd conf/
[root@hadoop1 conf]# sh $HADOOP_HOME/bin/hdfs dfs -chmod 777 /user/hive/warehouse
#查看修改後的權限
[root@hadoop1 conf]# sh $HADOOP_HOME/bin/hdfs dfs -ls /user/hive
Found 1 items
drwxrwxrwx   - root supergroup          0 2019-02-26 14:15 /user/hive/warehouse

#運用hadoop命令新建/tmp/hive目錄
[root@hadoop1 conf]# $HADOOP_HOME/bin/hdfs dfs -mkdir -p /tmp/hive
#給目錄/tmp/hive賦予讀寫權限
[root@hadoop1 conf]# $HADOOP_HOME/bin/hdfs dfs -chmod 777 /tmp/hive
#檢查創建好的目錄
[root@hadoop1 conf]# $HADOOP_HOME/bin/hdfs dfs -ls /tmp
Found 1 items
drwxrwxrwx   - root supergroup          0 2019-02-26 14:17 /tmp/hive

修改$HIVE_HOME/conf/hive-site.xml中的臨時目錄 將hive-site.xml文件中的${system:java.io.tmpdir}替換爲hive的臨時目錄，例如我替換爲$HIVE_HOME/tmp，該目錄如果不存在則要自己手工創建，並且賦予讀寫權限。

[root@hadoop1 conf]# cd $HIVE_HOME
[root@hadoop1 hive]# mkdir tmp

配置文件hive-site.xml:
將文件中的所有 ${system:java.io.tmpdir}替換成/hadoop/hive/tmp 將文件中所有的${system:user.name}替換爲root
(4)配置mysql
把mysql的驅動包上傳到Hive的lib目錄下：

[root@hadoop lib]# pwd
/usr/local/hive/lib
[root@hadoop1 lib]# ls |grep mysql
mysql-connector-java-5.1.47.jar

(5)修改hive-site.xml數據庫相關配置
搜索javax.jdo.option.connectionURL,將該name對應的value修改爲MySQL的地址:

<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost:3306/metastore?createDatabaseIfNotExist=true&amp;characterEncoding=UTF-8&amp;useSSL=false</value>
    <description>
      JDBC connect string for a JDBC metastore.
      To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
      For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
    </description>
  </property>

搜索javax.jdo.option.ConnectionDriverName，將該name對應的value修改爲MySQL驅動類路徑:

<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
  </property>

搜索javax.jdo.option.ConnectionUserName，將對應的value修改爲MySQL數據庫登錄名:

<property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
    <description>Username to use against metastore database</description>
  </property>

搜索javax.jdo.option.ConnectionPassword，將對應的value修改爲MySQL數據庫的登錄密碼:

<property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>oracle</value>
    <description>password to use against metastore database</description>
  </property>

搜索hive.metastore.schema.verification，將對應的value修改爲false：

<property>
    <name>hive.metastore.schema.verification</name>
    <value>false</value>

在$HIVE_HOME/conf目錄下新建hive-env.sh

進入目錄
[root@hadoop1 conf]# cd $HIVE_HOME/conf
[root@hadoop1 conf]# cp hive-env.sh.template hive-env.sh
#打開hive-env.sh並添加如下內容
[root@hadoop1 conf]# vim hive-env.sh
export HADOOP_HOME=/hadoop/
export HIVE_CONF_DIR=/hadoop/hive/conf
export HIVE_AUX_JARS_PATH=/hadoop/hive/lib

（6）MySQL數據庫進行初始化

#進入$HIVE/bin
[root@apollo conf]# cd $HIVE_HOME/bin
#對數據庫進行初始化：
[root@hadoop1 bin]# schematool -initSchema -dbType mysql
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/hadoop/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/imp
l/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/
org/slf4j/impl/StaticLoggerBinder.class]SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:	 jdbc:mysql://localhost:3306/metastore?createDatabaseIfNotEx
ist=true&characterEncoding=UTF-8&useSSL=falseMetastore Connection Driver :	 com.mysql.jdbc.Driver
Metastore connection User:	 root
Starting metastore schema initialization to 2.3.0
Initialization script hive-schema-2.3.0.mysql.sql
Initialization script completed
schemaTool completed

出現上面就是初始化成功，去mysql看下：
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| metastore          |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.00 sec)

mysql> use metastore
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+---------------------------+
| Tables_in_metastore       |
+---------------------------+
| AUX_TABLE                 |
| BUCKETING_COLS            |
| CDS                       |
| COLUMNS_V2                |
| COMPACTION_QUEUE          |
| COMPLETED_COMPACTIONS     |
| COMPLETED_TXN_COMPONENTS  |
| DATABASE_PARAMS           |
| DBS                       |
| DB_PRIVS                  |
| DELEGATION_TOKENS         |
| FUNCS                     |
| FUNC_RU                   |
| GLOBAL_PRIVS              |
| HIVE_LOCKS                |
| IDXS                      |
| INDEX_PARAMS              |
| KEY_CONSTRAINTS           |
| MASTER_KEYS               |
| NEXT_COMPACTION_QUEUE_ID  |
| NEXT_LOCK_ID              |
| NEXT_TXN_ID               |
| NOTIFICATION_LOG          |
| NOTIFICATION_SEQUENCE     |
| NUCLEUS_TABLES            |
| PARTITIONS                |
| PARTITION_EVENTS          |
| PARTITION_KEYS            |
| PARTITION_KEY_VALS        |
| PARTITION_PARAMS          |
| PART_COL_PRIVS            |
| PART_COL_STATS            |
| PART_PRIVS                |
| ROLES                     |
| ROLE_MAP                  |
| SDS                       |
| SD_PARAMS                 |
| SEQUENCE_TABLE            |
| SERDES                    |
| SERDE_PARAMS              |
| SKEWED_COL_NAMES          |
| SKEWED_COL_VALUE_LOC_MAP  |
| SKEWED_STRING_LIST        |
| SKEWED_STRING_LIST_VALUES |
| SKEWED_VALUES             |
| SORT_COLS                 |
| TABLE_PARAMS              |
| TAB_COL_STATS             |
| TBLS                      |
| TBL_COL_PRIVS             |
| TBL_PRIVS                 |
| TXNS                      |
| TXN_COMPONENTS            |
| TYPES                     |
| TYPE_FIELDS               |
| VERSION                   |
| WRITE_SET                 |
+---------------------------+
57 rows in set (0.01 sec)

（7）啓動hive：

啓動metastore服務
nohup hive --service metastore >> ~/metastore.log 2>&1 &         ##hivemetastore
 啓動hive服務
nohup  hive --service hiveserver2 >> ~/hiveserver2.log 2>&1 &    ##hiveserver2，jdbc連接均需要
[root@hadoop bin]# netstat  -lnp|grep 9083
tcp        0      0 0.0.0.0:9083            0.0.0.0:*               LISTEN      11918/java 
[root@hadoop bin]# netstat  -lnp|grep 10000
tcp        0      0 0.0.0.0:10000           0.0.0.0:*               LISTEN      12011/java  
[root@hadoop1 bin]# ./hive
which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/java/jdk1.8.0_151
/bin:/usr/java/jdk1.8.0_151/bin:/hadoop//bin:/hadoop//sbin:/root/bin:/usr/java/jdk1.8.0_151/bin:/usr/java/jdk1.8.0_151/bin:/hadoop//bin:/hadoop//sbin:/hadoop/hive/bin)SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/hadoop/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/imp
l/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/
org/slf4j/impl/StaticLoggerBinder.class]SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Logging initialized using configuration in jar:file:/hadoop/hive/lib/hive-common-2.3.3.jar!/
hive-log4j2.properties Async: trueHive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider
 using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
hive> show functions;
OK
!
!=
$sum0
%
。。。。。
hive> desc function sum;
OK
sum(x) - Returns the sum of a set of numbers
Time taken: 0.183 seconds, Fetched: 1 row(s)
hive> create database sbux;
OK
Time taken: 0.236 seconds
hive> use sbux;
OK
Time taken: 0.033 seconds
hive> create table student(id int, name string) row format delimited fields terminated by '\t';
OK
Time taken: 0.909 seconds
hive> desc student;
OK
id                  	int                 	                    
name                	string              	                    
Time taken: 0.121 seconds, Fetched: 2 row(s)
 在$HIVE_HOME下新建一個文件
 #進入#HIVE_HOME目錄
[root@apollo hive]# cd $HIVE_HOME
#新建文件student.dat
[root@apollo hive]# touch student.dat
#在文件中添加如下內容
[root@apollo hive]# vim student.dat
001	david
002	fab
003	kaishen
004	josen
005	arvin
006	wada
007	weda
008	banana
009	arnold
010	simon
011	scott
.導入數據
hive> load data local inpath '/hadoop/hive/student.dat' into table sbux.student;
Loading data to table sbux.student
OK
Time taken: 8.641 seconds
hive> use sbux;
OK
Time taken: 0.052 seconds
hive> select * from student;
OK
1	david
2	fab
3	kaishen
4	josen
5	arvin
6	wada
7	weda
8	banana
9	arnold
10	simon
11	scott
NULL	NULL
Time taken: 2.217 seconds, Fetched: 12 row(s)

（8）在界面上查看剛剛寫入的hdfs數據
在hadoop的namenode上查看：


在mysql的hive數據裏查看

[root@hadoop1 bin]# mysql -u root -p
Enter password: 
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| metastore |
| mysql |
| performance_schema |
| sys |
+--------------------+
5 rows in set (0.00 sec)
mysql> use metastore;
Database changed
mysql> select * from TBLS;
+--------+-------------+-------+------------------+-------+-----------+-------+----------+---------------+--------------------+--------------------+--------------------+
| TBL_ID | CREATE_TIME | DB_ID | LAST_ACCESS_TIME | OWNER | RETENTION | SD_ID | TBL_NAME | TBL_TYPE | VIEW_EXPANDED_TEXT | VIEW_ORIGINAL_TEXT | IS_REWRITE_ENABLED |
+--------+-------------+-------+------------------+-------+-----------+-------+----------+---------------+--------------------+--------------------+--------------------+
| 1 | 1551178545 | 6 | 0 | root | 0 | 1 | student | MANAGED_TABLE | NULL | NULL | |
+--------+-------------+-------+------------------+-------+-----------+-------+----------+---------------+--------------------+--------------------+--------------------+
1 row in set (0.00 sec)

8、安裝hbase

下載連接：
http://archive.apache.org/dist/hbase/
（1）創建安裝目錄並上傳解壓：

[root@hadoop hbase]# pwd
/hadoop/hbase
[root@hadoop hbase]# ls
bin          conf  hbase-1.4.9    LEGAL  LICENSE.txt  README.txt
CHANGES.txt  docs  hbase-webapps  lib    NOTICE.txt

（2）環境變量配置，我的環境變量如下：

export JAVA_HOME=/usr/java/jdk1.8.0_151
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin
export HADOOP_HOME=/hadoop/
export HADOOP_CONF_DIR=${HADOOP_HOME}/etc/hadoop
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib:$HADOOP_COMMON_LIB_NATIVE_DIR"
export HIVE_HOME=/hadoop/hive
export HIVE_CONF_DIR=${HIVE_HOME}/conf
export HCAT_HOME=$HIVE_HOME/hcatalog
export HIVE_DEPENDENCY=/hadoop/hive/conf:/hadoop/hive/lib/*:/hadoop/hive/hcatalog/share/hcatalog/hive-hcatalog-pig-adapter-2.3.3.jar:/hadoop/hive/hcatalog/share/hcatalog/hive-hcatalog-core-2.3.3.jar:/hadoop/hive/hcatalog/share/hcatalog/hive-hcatalog-server-extensions-2.3.3.jar:/hadoop/hive/hcatalog/share/hcatalog/hive-hcatalog-streaming-2.3.3.jar:/hadoop/hive/lib/hive-exec-2.3.3.jar
export HBASE_HOME=/hadoop/hbase/
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin::$HCAT_HOME/bin:$HBASE_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:${HIVE_HOME}/lib:$HBASE_HOME/lib

（3）詳細配置

[root@hadoop hbase]# cd /hadoop/hbase/
創建zookeeper數據目錄：
[root@hadoop hbase]# mkdir data
修改hbase-env.sh文件加入下面內容
[root@hadoop hbase]# vim conf/hbase-env.sh 
export JAVA_HOME=/usr/java/jdk1.8.0_151
export HADOOP_HOME=/hadoop/
export HBASE_HOME=/hadoop/hbase/
修改配置文件hbase-site.xml
在該配置文件中可以給hbase配置一個臨時目錄，這裏指定爲mkdir /root/hbase/tmp，先執行命令創建文件夾。
mkdir  /root/hbase
mkdir  /root/hbase/tmp
mkdir  /root/hbase/pids
在<configuration>節點內增加以下配置：
<configuration>
  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://192.168.1.66:9000/hbase</value>
  </property>
  <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/hadoop/hbase/data/zookeeper</value>
  </property>
  <property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
  </property>
</configuration>
[root@hadoop hbase]# cat conf/regionservers 
192.168.1.66

（4）啓動hbase並測試可用性：

[root@hadoop hbase]#  bin/hbase-daemon.sh start zookeeper
running zookeeper, logging to /hadoop/hbase//logs/hbase-root-zookeeper-hadoop.out
[root@hadoop hbase]# bin/hbase-daemon.sh start master
running master, logging to /hadoop/hbase//logs/hbase-root-master-hadoop.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
[root@hadoop hbase]# bin/hbase-daemon.sh start regionserver
running regionserver, logging to /hadoop/hbase//logs/hbase-root-regionserver-hadoop.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
[root@hadoop hbase]# jps
13216 HMaster
10376 SecondaryNameNode
12011 RunJar
11918 RunJar
13070 HQuorumPeer
12623 RunJar
11060 JobHistoryServer
10581 ResourceManager
10102 NameNode
13462 HRegionServer
10201 DataNode
10683 NodeManager
13758 Jps

可以看出，新增了HQuorumPeer，HRegionServer和HMaster三個進程。

（5）訪問web頁面： http://192.168.1.66:16030/rs-status

9、安裝KYLIN

下載連接
http://kylin.apache.org/cn/download/
（1）上傳解壓

[root@hadoop kylin]# pwd
/hadoop/kylin
[root@hadoop kylin]# ls
apache-kylin-2.4.0-bin-hbase1x.tar.gz
[root@hadoop kylin]# tar -zxvf apache-kylin-2.4.0-bin-hbase1x.tar.gz
[root@hadoop kylin]# rm -rf  apache-kylin-2.4.0-bin-hbase1x.tar.gz
[root@hadoop kylin]# 
[root@hadoop kylin]# mv apache-kylin-2.4.0-bin-hbase1x/* .
[root@hadoop kylin]# ls
apache-kylin-2.4.0-bin-hbase1x  bin  commit_SHA1  conf  lib  sample_cube  spark  tomcat  tool

（2）配置環境變量
/etc/profile內容如下

export JAVA_HOME=/usr/java/jdk1.8.0_151
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin
export HADOOP_HOME=/hadoop/
export HADOOP_CONF_DIR=${HADOOP_HOME}/etc/hadoop
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib:$HADOOP_COMMON_LIB_NATIVE_DIR"
export HIVE_HOME=/hadoop/hive
export HIVE_CONF_DIR=${HIVE_HOME}/conf
export HCAT_HOME=$HIVE_HOME/hcatalog
export hive_dependency=/hadoop/hive/conf:/hadoop/hive/lib/*:/hadoop/hive/hcatalog/share/hcatalog/hive-hcatalog-pig-adapter-2.3.3.jar:/hadoop/hive/hcatalog/share/hcatalog/hive-hcatalog-core-2.3.3.jar:/hadoop/hive/hcatalog/share/hcatalog/hive-hcatalog-server-extensions-2.3.3.jar:/hadoop/hive/hcatalog/share/hcatalog/hive-hcatalog-streaming-2.3.3.jar:/hadoop/hive/lib/hive-exec-2.3.3.jar
export HBASE_HOME=/hadoop/hbase/
export KYLIN_HOME=/hadoop/kylin/
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin:$HCAT_HOME/bin:$HBASE_HOME/bin:$KYLIN_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:${HIVE_HOME}/lib:$HBASE_HOME/lib:$KYLIN_HOME/lib
[root@hadoop kylin]# source /etc/profile

（3）修改kylin.properties內容

[root@hadoop kylin]# vim conf/kylin.properties 
加入下面內容
kylin.rest.timezone=GMT+8
kylin.rest.servers=192.168.1.66:7070
kylin.job.jar=/hadoop/kylin/lib/kylin-job-2.4.0.jar
kylin.coprocessor.local.jar=/hadoop/kylin/lib/kylin-coprocessor-2.4.0.jar
kyin.server.mode=all 
kylin.rest.servers=192.168.1.66:7070

（4）編輯kylin_hive_conf.xml

[root@hadoop kylin]# vim conf/kylin_hive_conf.xml
<property>
        <name>hive.exec.compress.output</name>
        <value>false</value>
        <description>Enable compress</description>
    </property>

（5）編輯server.xml

[root@hadoop kylin]# vim tomcat/conf/server.xml
註釋掉下面這點代碼：
 <!-- Connector port="7443" protocol="org.apache.coyote.http11.Http11Protocol"
                   maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
                   keystoreFile="conf/.keystore" keystorePass="changeit"
                   clientAuth="false" sslProtocol="TLS" /> -->

（6）編輯kylin.sh

  #additionally add tomcat libs to HBASE_CLASSPATH_PREFIX
    export HBASE_CLASSPATH_PREFIX=${tomcat_root}/bin/bootstrap.jar:${tomcat_root}/bin/tomcat-juli.jar:${tomcat_root}/lib/*:$hive_dependency:${HBASE_CLASSPATH_PREFIX}

（7）啓動kylin

[root@hadoop kylin]# cd bin/
[root@hadoop bin]# pwd
/hadoop/kylin/bin
[root@hadoop bin]# ./check-env.sh 
Retrieving hadoop conf dir...
KYLIN_HOME is set to /hadoop/kylin
[root@hadoop bin]# ./kylin.sh start
Retrieving hadoop conf dir...
KYLIN_HOME is set to /hadoop/kylin
Retrieving hive dependency...
。。。。。。。。。。。
A new Kylin instance is started by root. To stop it, run 'kylin.sh stop'
Check the log at /hadoop/kylin/logs/kylin.log
Web UI is at http://<hostname>:7070/kylin
[root@hadoop bin]# jps
13216 HMaster
10376 SecondaryNameNode
12011 RunJar
11918 RunJar
13070 HQuorumPeer
11060 JobHistoryServer
10581 ResourceManager
31381 RunJar
10102 NameNode
13462 HRegionServer
10201 DataNode
10683 NodeManager
31677 Jps

（8）初步驗證及使用：

1）、測試創建項目從hive庫取表：

打開網頁：http://192.168.1.66:7070/kylin/login
初始密碼：ADMIN/KYLIN

由頂部菜單欄進入 Model 頁面，然後點擊 Manage Projects。

點擊 + Project 按鈕添加一個新的項目。

在頂部菜單欄點擊 Model，然後點擊左邊的 Data Source 標籤，它會列出所有加載進 Kylin 的表，點擊 Load Table 按鈕。

輸入表名並點擊 Sync 按鈕提交請求。

接下來就可以看到導入的表結構了：

(2）、運行官方案例：

    [root@hadoop bin]# pwd
    /hadoop/kylin/bin
    [root@hadoop bin]# ./sample.sh 
    Retrieving hadoop conf dir...
    。。。。。。。。。
    Sample cube is created successfully in project 'learn_kylin'.
    Restart Kylin Server or click Web UI => System Tab => Reload Metadata to take effect    

看到上面最後兩條信息就說明案例使用的hive表都創建好了，接下來重啓kylin或則 reload metadata
再次刷新頁面：

選擇第二個kylin_sales_cube

選擇bulid，隨意選擇一個12年以後的日期

然後切換到monitor界面：

等待cube創建完成。

做sql查詢

編輯整個環境重啓腳本方便日常啓停：
環境停止腳本

[root@hadoop hadoop]# cat stop.sh 
#!/bin/bash
echo -e "\n========Start stop kylin========\n"
$KYLIN_HOME/bin/kylin.sh stop
sleep 5
echo -e "\n========Start stop hbase-regionserver========\n"
$HBASE_HOME/bin/hbase-daemon.sh stop regionserver
sleep 5
echo -e "\n========Start stop hbase-master========\n"
$HBASE_HOME/bin/hbase-daemon.sh stop master 
sleep 3
echo -e "\n========Start stop zookeeper========\n"
$HBASE_HOME/bin/hbase-daemon.sh stop zookeeper
sleep 3
echo -e "\n========Start stop jobhistory========\n"
mr-jobhistory-daemon.sh stop historyserver
sleep 3
echo -e "\n========Start stop yarn========\n"
stop-yarn.sh
sleep 5
echo -e "\n========Start stop dfs========\n"
stop-dfs.sh
sleep 5
echo -e "\n========Start stop prot========\n"
`lsof -i:9083|awk 'NR>=2{print "kill -9 "$2}'|sh`
`lsof -i:10000|awk 'NR>=2{print "kill -9 "$2}'|sh`
sleep 2
echo -e "\n========Check process========\n"
jps

環境啓動腳本

[root@hadoop hadoop]# cat start.sh 
#!/bin/bash
echo -e "\n========Start run dfs========\n"
start-dfs.sh
sleep 2
echo -e "\n========Start run yarn========\n"
start-yarn.sh
sleep 2
echo -e "\n========Start run jobhistory========\n"
mr-jobhistory-daemon.sh start historyserver
sleep 2
echo -e "\n========Start run metastore========\n"
nohup hive --service metastore >> ~/metastore.log 2>&1 &  
sleep 10
echo -e "\n========Start run hiveserver2========\n"
nohup  hive --service hiveserver2 >> ~/hiveserver2.log 2>&1 &
sleep 10
echo -e "\n========Check Port========\n"
netstat  -lnp|grep 9083
sleep 5
netstat  -lnp|grep 10000
sleep 2
echo -e "\n========Start run zookeeper========\n"
$HBASE_HOME/bin/hbase-daemon.sh start zookeeper
sleep 1
echo -e "\n========Start run hbase-master========\n"
$HBASE_HOME/bin/hbase-daemon.sh start master 
sleep 1
echo -e "\n========Start run hbase-regionserver========\n"
$HBASE_HOME/bin/hbase-daemon.sh start regionserver
sleep 1
echo -e "\n========Check process========\n"
jps
sleep 1
echo -e "\n========Start run kylin========\n"
$KYLIN_HOME/bin/kylin.sh start