首先大家得弄個自己的內網Nexus3倉庫。
配置yum代理倉庫地址:https://archive.cloudera.com/
yum倉庫文件下載地址：https://archive.cloudera.com/cm6/6.3.1/redhat7/yum/cloudera-manager.repo
下載好後如果配置了Nexus3倉庫yum代理的話可以修改https://archive.cloudera.com爲你的私服地址https://repo.xxx.com
然後把這個repo文件放到centos 系統的 /etc/yum.repos.d/目錄下重新清理生成緩存就好了。
命令爲：yum clean all && yum makecache
大家可以用docker run -d -p 8081:8081 zouchengli/nexus3-apk 臨時啓動一個，公司有的話最好。直接搞起。
大家準備兩臺或多臺服務器吧。我這裏準備三臺。
我安裝的centos7迷你版。也就是目前官網提供的最小容量鏡像。
下載地址爲：https://mirrors.aliyun.com/centos/7/isos/x86_64/CentOS-7-x86_64-Minimal-2003.iso
1.節點準備（3個節點）
| node1 | 192.168.168.81 | Cloudera Manager Server&Agent ，MariaDB |
| node2 | 192.168.168.82 | Cloudera Manager Agent |
| node3 | 192.168.168.83 | Cloudera Manager Agent |
2.配置主機名和hosts解析(所有節點)
編輯/etc/hostname，修改主機名，並使用命令hostname使其立刻生效。編輯文件/etc/hosts，增加如下內容。
192.168.168.81 node1
192.168.168.82 node2
192.168.168.83 node3
3.關閉防火牆和SELinux
[root@node1 ~]# systemctl stop firewalld && systemctl disable firewalld
[root@node1 ~]# sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config
[root@node1 ~]# setenforce 0
4.配置時間同步服務器和客戶端（一般都自帶了，沒有的話可以安裝）
[root@node1 ~]# yum -y install chrony
編輯 /etc/chrony.conf 文件
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
server ntp1.aliyun.com iburst
server ntp2.aliyun.com iburst
server ntp3.aliyun.com iburst
server ntp4.aliyun.com iburst
server ntp5.aliyun.com iburst
server ntp6.aliyun.com iburst
server ntp7.aliyun.com iburst
# Record the rate at which the system clock gains/losses time.
driftfile /var/lib/chrony/drift
# Allow the system clock to be stepped in the first three updates
# if its offset is larger than 1 second.
makestep 1.0 3
# Enable kernel synchronization of the real-time clock (RTC).
rtcsync
# Enable hardware timestamping on all interfaces that support it.
#hwtimestamp *
# Increase the minimum number of selectable sources required to adjust
# the system clock.
#minsources 2
# Allow NTP client access from local network.
allow 192.168.168.0/24
# Serve time even if not synchronized to a time source.
#local stratum 10
# Specify file containing keys for NTP authentication.
#keyfile /etc/chrony.keys
# Specify directory for log files.
logdir /var/log/chrony
# Select which information is logged.
#log measurements statistics tracking
重啓服務
[root@node1 ~]# systemctl enable chronyd && systemctl restart chronyd
檢查時間同步
[root@node1 ~]# chronyc sources
210 Number of sources = 2
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^+ 120.25.115.20 2 8 377 222 -1144us[-1144us] +/- 21ms
^* 203.107.6.88 2 8 377 230 -2508us[-2312us] +/- 18ms
節點2，3同樣安裝
[root@node2 ~]# yum -y install chrony
節點2，3同樣編輯 /etc/chrony.conf 文件
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
server 192.168.168.81 iburst
# Record the rate at which the system clock gains/losses time.
driftfile /var/lib/chrony/drift
# Allow the system clock to be stepped in the first three updates
# if its offset is larger than 1 second.
makestep 1.0 3
# Enable kernel synchronization of the real-time clock (RTC).
rtcsync
# Enable hardware timestamping on all interfaces that support it.
#hwtimestamp *
# Increase the minimum number of selectable sources required to adjust
# the system clock.
#minsources 2
# Allow NTP client access from local network.
#allow 192.168.0.0/16
# Serve time even if not synchronized to a time source.
#local stratum 10
# Specify file containing keys for NTP authentication.
#keyfile /etc/chrony.keys
# Specify directory for log files.
logdir /var/log/chrony
# Select which information is logged.
#log measurements statistics tracking
節點2，3同樣重啓服務
[root@node2 ~]# systemctl enable chronyd && systemctl restart chronyd
檢查時間同步
[root@node2 ~]# chronyc sources
210 Number of sources = 1
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* 192.168.168.81 3 6 377 14 +18us[ +21us] +/- 18ms
5.禁用透明大頁面壓縮，CDH配置必須
[root@node1 ~]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@node1 ~]# echo never > /sys/kernel/mm/transparent_hugepage/enabled
並將上面的兩條命令寫入開機自啓動/etc/rc.local。
6.優化交換分區
[root@node1 ~]# echo "vm.swappiness = 10" >> /etc/sysctl.conf
[root@node1 ~]# sysctl -p
7.安裝jdk8，cdh server && agent
[root@node1 ~]# yum -y install java-1.8.0-openjdk java-1.8.0-openjdk-devel
[root@node1 ~]# yum -y install cloudera-manager-server cloudera-manager-agent
其他節點只需要安裝agent即可
[root@node2 ~]# yum -y install java-1.8.0-openjdk java-1.8.0-openjdk-devel
[root@node2 ~]# yum -y install cloudera-manager-agent
8.安裝數據庫
[root@node1 ~]# yum -y install mariadb*
修改配置文件/etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
transaction-isolation = READ-COMMITTED
symbolic-links = 0
key_buffer = 16M
key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1
max_connections = 550
log_bin=/var/lib/mysql/mysql_binary_log
server_id=1
binlog_format = mixed
read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M
[mysqld_safe]
log-error=/var/log/mariadb/mariadb.log
pid-file=/var/run/mariadb/mariadb.pid
!includedir /etc/my.cnf.d
啓動服務
[root@node1 ~]# systemctl enable mariadb && systemctl restart mariadb
初始化數據庫 root默認初始密碼爲空，第一步直接回車即可。
[root@node1 ~]# /usr/bin/mysql_secure_installation
NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB
SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!
In order to log into MariaDB to secure it, we'll need the current
password for the root user. If you've just installed MariaDB, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.
Enter current password for root (enter for none):
OK, successfully used password, moving on...
Setting the root password ensures that nobody can log into the MariaDB
root user without the proper authorisation.
Set root password? [Y/n] Y
New password:
Re-enter new password:
Password updated successfully!
Reloading privilege tables..
... Success!
By default, a MariaDB installation has an anonymous user, allowing anyone
to log into MariaDB without having to have a user account created for
them. This is intended only for testing, and to make the installation
go a bit smoother. You should remove them before moving into a
production environment.
Remove anonymous users? [Y/n] Y
... Success!
Normally, root should only be allowed to connect from 'localhost'. This
ensures that someone cannot guess at the root password from the network.
Disallow root login remotely? [Y/n] n
... skipping.
By default, MariaDB comes with a database named 'test' that anyone can
access. This is also intended only for testing, and should be removed
before moving into a production environment.
Remove test database and access to it? [Y/n] Y
- Dropping test database...
... Success!
- Removing privileges on test database...
... Success!
Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.
Reload privilege tables now? [Y/n] Y
... Success!
Cleaning up...
All done! If you've completed all of the above steps, your MariaDB
installation should now be secure.
Thanks for using MariaDB!
9，下載mysql驅動jar包（所有節點）
[root@node1 ~]# mkdir -p /usr/share/java
[root@node1 ~]# cd /usr/share/java
[root@node1 java]# curl -O https://repo1.maven.org/maven2/mysql/mysql-connector-java/5.1.38/mysql-connector-java-5.1.38.jar
[root@node1 java]# mv mysql-connector-java-5.1.38.jar mysql-connector-java.jar
10.製作一個cdh.sql文件，內容如下：
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'clzlinux';
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'clzlinux';
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'clzlinux';
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'clzlinux';
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY 'clzlinux';
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'clzlinux';
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'clzlinux';
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'clzlinux';
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'clzlinux';
完成後執行該文件
[root@node1 ~]# mysql -uroot -pcdh6pwd
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 11
Server version: 5.5.65-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| amon |
| hue |
| metastore |
| mysql |
| nav |
| navms |
| oozie |
| performance_schema |
| rman |
| scm |
| sentry |
+--------------------+
12 rows in set (0.01 sec)
MariaDB [(none)]>
11.設置 Cloudera Manager 數據
[root@node1 ~]# /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm
Enter SCM password:
JAVA_HOME=/usr/lib/jvm/java-openjdk
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing: /usr/lib/jvm/java-openjdk/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
[ main] DbCommandExecutor INFO Successfully connected to database.
All done, your SCM database is configured correctly!
12.啓動服務
[root@node1 ~]# systemctl start cloudera-scm-server
[root@node1 ~]# tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
13.WEB訪問地址http://192.168.168.81:7180 默認用戶名密碼都是admin。