greenplum install
[TOC]
OS：rhel7.5 x86_64 DB：GreenPlum db 6.x
下載源碼包安裝 gp download
安裝前參考最佳實踐對操作系統進行優化！！
greenplum or hgdw Best practices
1 pretasks
修改主機名
cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
# gp configure
192.168.80.146 gp1
192.168.80.147 gp2
192.168.80.148 gp3
192.168.80.149 gp4
創建操作系統用戶
使用root用戶登錄主節點主機系統，在主節點創建數據庫用gpadmin戶，切換到新用戶運行運行數據倉庫安裝包。
groupadd -g 1001 gpadmin
useradd -u 1001 -g gpadmin gpadmin; echo gpadmin | passwd -f --stdin gpadmin
2 install
安裝數據倉庫軟件
gp4.x/gp5.x 以前安裝分爲三部分 1. 安裝master（一般是個bin的可執行文件，安裝，並可以指定安裝目錄） 2. gpseginstall 安裝各個seg 3. gp羣參數校驗 4. gpinitsystem 集羣初始化 gp6.2 開始不提供zip 格式壓縮包，僅提供rpm包，安裝分爲兩步 1. 安裝master(rpm -ivh / yum install -y),不可以指定安裝目錄，默認安裝到/usr/local/ 2. gp6 沒有 gpseginstall工具。所以要麼自己打包master 安裝好的gp目錄並傳到seg上，要麼各個節點單獨yum 安裝。 1.每個節點主機，單獨yum 2.打包主節點的安裝目錄，並分發給seg主機。 3. 集羣性能校驗 4. gpinitsystem 集羣初始化
先只在主節點安裝軟件
rpm -ivh xxx.rpm
環境變量
添加下面初始化變量到gpadmin用戶.bash_profile文件，使立即生效：source .bash_profile：
cat ~/.bash_profile
source /opt/gp/greenplum/greenplum_path.sh
export MASTER_DATA_DIRECTORY=/data/hgdata/master/hgseg-1
export PGPORT=2345
export PGUSER=hgadmin
export PGDATABASE=gpdb
greenplum_path.sh — 該文件包含數據倉庫的基礎環境變量。默認自動生成
創建hostfile
創建幾個host文件，用於後續使用gpssh,gpscp 等腳本host參數文件
hostfile_exkeys 內容是集羣所有主機名或ip，用於ssh互信配置 hostfile_all 內容是集羣所有主機名或ip，包含master,segment,standby等 hostfile_seg 內容是所有 segment主機名或ip
cat hostfile_exkeys
hgdw1
hgdw2
hgdw3
hgdw4
hgdw5
hgdw6
cat hostfile_all
hgdw1
hgdw2
hgdw3
hgdw4
hgdw5
hgdw6
cat hostfile_seg
hgdw2
hgdw3
hgdw4
hgdw5
hgdw6
配置節點互信
配置互信
root用戶和gpadin用戶都需要做
1. 主機生成密鑰
ssh-keygen -t rsa
2. 將公鑰複製到各個節點機器的authorized_keys文件中
ssh-copy-id -i id_rsa.pub root@hgdw1
ssh-copy-id -i id_rsa.pub root@hgdw3
ssh-copy-id -i id_rsa.pub root@hgdw3
ssh-copy-id -i id_rsa.pub root@hgdw4
ssh-copy-id -i id_rsa.pub root@hgdw4
ssh-copy-id -i id_rsa.pub root@hgdw6
3. 使用gpssh-exkeys 工具，打通n-n的免密登陸
root和gpadmin用戶都執行一遍
gpssh-exkeys -f /home/gpadmin/hostfile_exkeys
hgssh-exkeys -f /home/hgadmin/hostfile_exkeys
[STEP 1 of 5] create local ID and authorize on local host
... /root/.ssh/id_rsa file exists ... key generation skipped
[STEP 2 of 5] keyscan all hosts and update known_hosts file
[STEP 3 of 5] retrieving credentials from remote hosts
... send to hgdw2
... send to hgdw3
... send to hgdw4
... send to hgdw5
... send to hgdw6
[STEP 4 of 5] determine common authentication file content
[STEP 5 of 5] copy authentication files to all remote hosts
... finished key exchange with hgdw2
... finished key exchange with hgdw3
... finished key exchange with hgdw4
... finished key exchange with hgdw5
... finished key exchange with hgdw6
[INFO] completed successfully
驗證
hgssh -f /home/hgadmin/hostfile_all -e 'date'
[hgdw1] date
[hgdw1] Wed Apr 1 15:47:36 CST 2020
[hgdw3] date
[hgdw3] Wed Apr 1 15:47:36 CST 2020
[hgdw4] date
[hgdw4] Wed Apr 1 15:47:35 CST 2020
[hgdw6] date
[hgdw6] Wed Apr 1 15:47:35 CST 2020
[hgdw5] date
[hgdw5] Wed Apr 1 15:47:36 CST 2020
[hgdw2] date
[hgdw2] Wed Apr 1 15:47:36 CST 2020
同步個節點環境變量
gpscp -f /home/hgadmin/hostfile_seg /home/hgadmin/.bash_profile hgadmin@=:/home/hgadmin/.bash_profile
在所有計算節點上安裝配置數據倉庫
root
source /usr/local/hgdw/hgdw_path.sh
hgssh -f /home/hgadmin/hostfile_seg -e 'mkdir -p /opt/software'
hgssh -f /home/hgadmin/hostfile_seg -e 'chmod 777 /opt/software'
-greenpllum
gpscp -f /home/hgadmin/hostfile_seg /opt/software/greenplum-db-6.5.0-rhel7-x86_64.rpm hgadmin@=:/opt/software/
su - root
source /usr/local/greenplum-db/greenplum_path.sh
gpssh -f /home/hgadmin/hostfile_seg -e 'yum install apr -y'
gpssh -f /home/hgadmin/hostfile_seg -e 'yum install apr-util -y'
gpssh -f /home/hgadmin/hostfile_seg -e 'rpm -ivh /opt/software/greenplum-db-6.5.0-rhel7-x86_64.rpm'
gpssh -f /home/hgadmin/hostfile_seg -e 'chown hgadmin:hgadmin /usr/local/greenplum* -R'
-hgdw
gpscp -f /home/hgadmin/hostfile_seg /opt/software/hgdw-3.0.zip hgadmin@=:/opt/software/
hgssh -f /home/hgadmin/hostfile_seg -e 'unzip -d /usr/local /opt/software/hgdw-3.0.zip'
hgssh -f /home/hgadmin/hostfile_seg -e 'chown hgadmin:hgadmin /usr/local/hgdw -R'
創建數據存儲區域
創建或者選擇一個用於主節點的數據存儲區域。該目錄需要有足夠的磁盤空間並確保歸屬於gpadmin用戶和組。
hgadmin
創建master 數據目錄
mkdir -p /data/hgdata/master
創建segment 數據目錄
本次計劃每個主機安裝2個 segment
hgssh -f /home/hgadmin/hostfile_seg -e 'mkdir -p /data/hgdata/hgsegp'
hgssh -f /home/hgadmin/hostfile_seg -e 'mkdir -p /data/hgdata/hgsegm'
同步系統時鐘
在計算節點主機上，NTP應該配置主節點主機作爲主時間源，而備用主節點主機作爲備選時源。在主節點和備用主節點上配置NTP到用戶首選的時間服務器。
配置NTP
安裝相應軟件包：
yum install ntp -y
檢測是否安裝成功：
rpm -qa | grep ntp
ntpdate-4.2.6p5-25.el7.centos.x86_64
ntp-4.2.6p5-25.el7.centos.x86_64
開機自啓動
systemctl start ntpd.service
systemctl enable ntpd.service
systemctl enable ntpdate.service
查看狀態
systemctl status ntpd.service
主節點配置
vi /etc/ntp.conf
server 127.127.1.0 prefer
127.127.1.0 stratum 8
$ ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
*LOCAL(0) .LOCL. 5 l 43 64 377 0.000 0.000 0.000
備用主節點
vi /etc/ntp.conf
server gp1 prefer
計算節點
vi /etc/ntp.conf
server gp1 prefer
測試
ntpq -p
檢查系統環境
瀚高數據倉庫提供命令用以檢查系統的配置和性能，這些命令可以在瀚高數據倉庫安裝的$GPHOME/bin目錄下找到。初始化瀚高數據倉庫系統之前，應執行硬件性能檢查。
瀚高數據倉庫提供的gpcheckperf命令可用來在瀚高數據倉庫集羣主機上檢查硬件和系統級gpcheckperf在指定的主機上啓動一個會話並執行下面的性能測試：
檢查網絡性能 檢查磁盤I/O性能 檢查內存帶寬
在使用gpcheckperf之前，必須已經在所有相關需要做性能測試的主機之間建立了互信。如果還沒有做，可以使用gpssh-exkeys命令來建立互信。gpcheckperf會調用gpssh和gpscp，所以這些命令必須已經存在在$PATH中。
使用gpadmin用戶
1.檢查網絡性能
gpcheckperf -f /home/gpadmin/hostfile_gpcheckperfnet -r N -d /tmp > subnet1.out
/opt/gp/greenplum//bin/gpcheckperf -f /home/gpadmin/hostfile_gpcheckperfnet -r N -d /tmp
-------------------
-- NETPERF TEST
-------------------
====================
== RESULT
====================
Netperf bisection bandwidth test
gp1 -> gp2 = 81.110000
gp3 -> gp4 = 80.940000
gp2 -> gp1 = 87.330000
gp4 -> gp3 = 87.120000
Summary:
sum = 336.50 MB/sec
min = 80.94 MB/sec
max = 87.33 MB/sec
avg = 84.12 MB/sec
median = 87.12 MB/sec
2.檢查磁盤I/O和內存帶寬
使用gpadmin用戶登錄主節點主機。
創建一個名爲hostfile_gpcheckperfio的文件，包含所有計算節點主機名，每個名稱一行。不要包含主節點主機名。
gpcheckperf -f /home/gpadmin/hostfile_gpcheckperfio -r ds -d /data/segmentp -d /data/segmentm > io.out
[gpadmin@gp1 ~]$ cat io.out
/opt/gp/greenplum//bin/gpcheckperf -f /home/gpadmin/hostfile_gpcheckperfio -r ds -d /data/segmentp -d /data/segmentm
--------------------
-- DISK WRITE TEST
--------------------
--------------------
-- DISK READ TEST
--------------------
--------------------
-- STREAM TEST
--------------------
====================
== RESULT
====================
disk write avg time (sec): 18.36
disk write tot bytes: 6273368064
disk write tot bandwidth (MB/s): 334.08
disk write min bandwidth (MB/s): 140.84 [gp4]
disk write max bandwidth (MB/s): 193.24 [gp3]
disk read avg time (sec): 8.63
disk read tot bytes: 6273368064
disk read tot bandwidth (MB/s): 761.29
disk read min bandwidth (MB/s): 266.85 [gp3]
disk read max bandwidth (MB/s): 494.44 [gp4]
stream tot bandwidth (MB/s): 17422.21
stream min bandwidth (MB/s): 6680.49 [gp4]
stream max bandwidth (MB/s): 10741.71 [gp3]
初始化數據倉庫
gpinitsystem，能夠在主節點和每個計算節點實例上按照正確的順序完成初始化和啓動操作。 在數據倉庫系統初始化並啓動之後，就可以像使用常規的PostgreSQL數據庫一樣，通過連接到瀚高數據倉庫主節點來創建和管理數據庫。
創建數據倉庫配置文件
數據倉庫配置文件會告訴gpinitsystem命令按照什麼樣的方式配置系統。在軟件的安裝目錄的$GPHOME/docs/cli_help/gpconfigs/gpinitsystem_config文件可以作爲配置的例子參考?? 1. 以gpadmin用戶登錄： $ su - gpadmin 2. 拷貝一個gpinitsystem_config示例文件。例如 $cp $GPHOME/docs/cli_help/gpconfigs/gpinitsystem_config /home/gpadmin/gpinitsystem_config 3. 打開剛拷貝的文件並編輯。 DATA_DIRECTORY參數指定每個計算節點主機配置多少個計算節點實例。如果在host文件中爲每個計算節點主機列出了多個網口，這些實例將平均分佈到所有列出的網口上。 下面是一個gpinitsystem_config文件的例子： ARRAY_NAME="HighGo DataWarehouse " SEG_PREFIX=gpseg PORT_BASE=40000 declare -a DATA_DIRECTORY=(/data1/primary /data1/primary /data1/primary /data2/primary /data2/primary /data2/primary) MASTER_HOSTNAME=mdw MASTER_DIRECTORY=/data/master MASTER_PORT=5432 TRUSTED_SHELL=ssh CHECK_POINT_SEGMENT=8 ENCODING=UNICODE 4. 作爲可選項，可以配置鏡像計算節點實例，取消文件中的註釋並根據環境情況配置參數。下面是gpinitsystem_config文件中可選配置鏡像的例子： MIRROR_PORT_BASE=50000 REPLICATION_PORT_BASE=41000 MIRROR_REPLICATION_PORT_BASE=51000 declare -a MIRROR_DATA_DIRECTORY=(/data1/mirror /data1/mirror /data1/mirror /data2/mirror /data2/mirror /data2/mirror) 注意：可以在初始化時值配置主計算節點實例，而在之後使用gpaddmirrors命令部署鏡像計算節點實例。 5. 保存關閉該文件。
su - gpadmin
cp /usr/local/hgdw/docs/cli_help/hgconfigs/hginitsystem_config /home/hgadmin/hginitsystem_config
cat hginitsystem_config
# FILE NAME: gpinitsystem_config
# Configuration file needed by the gpinitsystem
################################################
#### REQUIRED PARAMETERS
################################################
#### Name of this Greenplum system enclosed in quotes.
ARRAY_NAME="HGDW"
#### Naming convention for utility-generated data directories.
SEG_PREFIX=gpseg
#### Base number by which primary segment port numbers
#### are calculated.
PORT_BASE=6000
#### File system location(s) where primary segment data directories
#### will be created. The number of locations in the list dictate
#### the number of primary segments that will get created per
#### physical host (if multiple addresses for a host are listed in
#### the hostfile, the number of segments will be spread evenly across
#### the specified interface addresses).
declare -a DATA_DIRECTORY=(/data/hgdata/hgsegp)
#### OS-configured hostname or IP address of the master host.
MASTER_HOSTNAME=hgdw1
#### File system location where the master data directory
#### will be created.
MASTER_DIRECTORY=/data/hgdata/master
#### Port number for the master instance.
MASTER_PORT=5432
#### Shell utility used to connect to remote hosts.
TRUSTED_SHELL=ssh
#### Maximum log file segments between automatic WAL checkpoints.
CHECK_POINT_SEGMENTS=18
#### Default server-side character set encoding.
ENCODING=UNICODE
################################################
#### OPTIONAL MIRROR PARAMETERS
################################################
#### Base number by which mirror segment port numbers
#### are calculated.
MIRROR_PORT_BASE=7000
#### File system location(s) where mirror segment data directories
#### will be created. The number of mirror locations must equal the
#### number of primary locations as specified in the
#### DATA_DIRECTORY parameter.
declare -a MIRROR_DATA_DIRECTORY=(/data/hgdata/hgsegm)
################################################
#### OTHER OPTIONAL PARAMETERS
################################################
#### Create a database of this name after initialization.
DATABASE_NAME=hgdw
#### Specify the location of the host address file here instead of
#### with the the -h option of gpinitsystem.
#MACHINE_LIST_FILE=/home/gpadmin/gpconfigs/hostfile_gpinitsystem
初始化數據庫
運行gpinitsystem命令將根據指定的配置文件創建一個瀚高數據倉庫系統。 運行初始化命令
hginitsystem -c /home/hgadmin/hginitsystem_config -h /home/hgadmin/hostfile_seg -b 10GB -m 2000
命令將自動檢查安裝信息，確認可以連接到每個主機，可以訪問配置文件中指定的每個目錄。如果檢查都通過了，將會提示確認配置信息。例如：
=> Continue with Greenplum creation? Yy/Nn
輸入y以開始初始化。
該命令將初始化主節點實例和系統中的每個計算節點實例。每個計算節點實例的安裝是並行的。 計算節點實例的數量決定這個過程使用的時間。
在成功安裝初始化結束時， 命令將會啓動瀚高數據倉庫系統，並可以看到如下提示信息：
=> Database successfully started
hginitsystem -c /home/hgadmin/hginitsystem_config -h /home/hgadmin/hostfile_seg -b 10GB -m 2000
20200401:18:46:57:026913 hginitsystem:hgdw1:hgadmin-[INFO]:-Checking configuration parameters, please wait...
20200401:18:46:57:026913 hginitsystem:hgdw1:hgadmin-[INFO]:-Reading HGDW configuration file /home/hgadmin/hginitsystem_config
20200401:18:46:57:026913 hginitsystem:hgdw1:hgadmin-[INFO]:-Locale has not been set in /home/hgadmin/hginitsystem_config, will set to default value
20200401:18:46:57:026913 hginitsystem:hgdw1:hgadmin-[INFO]:-Locale set to en_US.utf8
20200401:18:46:57:026913 hginitsystem:hgdw1:hgadmin-[FATAL]:-Expected file " /home/hgadmin/hostfile_seg" not found Script Exiting!
[hgadmin@hgdw1 master]$ hginitsystem -c /home/hgadmin/hginitsystem_config -h /home/hgadmin/hostfile_seg -b 10GB -m 2000
20200401:18:47:08:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Checking configuration parameters, please wait...
20200401:18:47:08:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Reading HGDW configuration file /home/hgadmin/hginitsystem_config
20200401:18:47:08:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Locale has not been set in /home/hgadmin/hginitsystem_config, will set to default value
20200401:18:47:08:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Locale set to en_US.utf8
20200401:18:47:08:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Checking configuration parameters, Completed
20200401:18:47:08:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Commencing multi-home checks, please wait...
.....
20200401:18:47:08:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Configuring build for standard array
20200401:18:47:08:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Commencing multi-home checks, Completed
20200401:18:47:08:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Building primary segment instance array, please wait...
.....
20200401:18:47:10:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Building group mirror array type , please wait...
.....
20200401:18:47:12:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Checking Master host
20200401:18:47:12:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Checking new segment hosts, please wait...
..........
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Checking new segment hosts, Completed
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-HGDW Database Creation Parameters
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:---------------------------------------
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Master Configuration
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:---------------------------------------
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Master instance name = HGDW
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Master hostname = hgdw1
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Master port = 5432
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Master instance dir = /data/hgdata/master/gpseg-1
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Master LOCALE = en_US.utf8
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-HGDW segment prefix = gpseg
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Master Database = hgdw
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Master connections = 2000
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Master buffers = 10GB
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Segment connections = 6000
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Segment buffers = 10GB
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Checkpoint segments = 18
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Encoding = UNICODE
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Postgres param file = Off
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Initdb to be used = /usr/local/hgdw/bin/initdb
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-GP_LIBRARY_PATH is = /usr/local/hgdw/lib
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-HEAP_CHECKSUM is = on
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-HBA_HOSTNAMES is = 0
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Ulimit check = Passed
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Array host connect type = Single hostname per node
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Master IP address [1] = ::1
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Master IP address [2] = 172.17.105.139
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Master IP address [3] = 192.168.122.1
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Master IP address [4] = fe80::f816:3eff:fe21:d1e6
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Standby Master = Not Configured
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Number of primary segments = 1
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Total Database segments = 5
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Trusted shell = ssh
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Number segment hosts = 5
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Mirror port base = 7000
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Number of mirror segments = 1
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Mirroring config = ON
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Mirroring type = Group
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:----------------------------------------
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-HGDW Primary Segment Configuration
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:----------------------------------------
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-hgdw2 /data/hgdata/hgsegp/gpseg0 6000 2 0
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-hgdw3 /data/hgdata/hgsegp/gpseg1 6000 3 1
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-hgdw4 /data/hgdata/hgsegp/gpseg2 6000 4 2
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-hgdw5 /data/hgdata/hgsegp/gpseg3 6000 5 3
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-hgdw6 /data/hgdata/hgsegp/gpseg4 6000 6 4
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:---------------------------------------
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-HGDW Mirror Segment Configuration
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:---------------------------------------
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-hgdw3 /data/hgdata/hgsegm/gpseg0 7000 7 0
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-hgdw4 /data/hgdata/hgsegm/gpseg1 7000 8 1
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-hgdw5 /data/hgdata/hgsegm/gpseg2 7000 9 2
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-hgdw6 /data/hgdata/hgsegm/gpseg3 7000 10 3
20200401:18:47:20:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-hgdw2 /data/hgdata/hgsegm/gpseg4 7000 11 4
Continue with HGDW creation Yy|Nn (default=N):
> y
20200401:18:47:43:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Building the Master instance database, please wait...
20200401:18:50:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Starting the Master in admin mode
20200401:18:50:52:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Commencing parallel build of primary segment instances
20200401:18:50:52:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Spawning parallel processes batch [1], please wait...
.....
20200401:18:50:52:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Waiting for parallel processes batch [1], please wait...
...........................................................................................................................................................................................
20200401:18:54:00:027116 hginitsystem:hgdw1:hgadmin-[INFO]:------------------------------------------------
20200401:18:54:00:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Parallel process exit status
20200401:18:54:00:027116 hginitsystem:hgdw1:hgadmin-[INFO]:------------------------------------------------
20200401:18:54:00:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Total processes marked as completed = 5
20200401:18:54:00:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Total processes marked as killed = 0
20200401:18:54:00:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Total processes marked as failed = 0
20200401:18:54:00:027116 hginitsystem:hgdw1:hgadmin-[INFO]:------------------------------------------------
20200401:18:54:00:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Deleting distributed backout files
20200401:18:54:00:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Removing back out file
20200401:18:54:00:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-No errors generated from parallel processes
20200401:18:54:00:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Restarting the HGDW instance in production mode
20200401:18:54:00:032368 hgstop:hgdw1:hgadmin-[INFO]:-Starting hgstop with args: -a -l /home/hgadmin/gpAdminLogs -m -d /data/hgdata/master/gpseg-1
20200401:18:54:00:032368 hgstop:hgdw1:hgadmin-[INFO]:-Gathering information and validating the environment...
20200401:18:54:00:032368 hgstop:hgdw1:hgadmin-[INFO]:-Obtaining HGDW Master catalog information
20200401:18:54:00:032368 hgstop:hgdw1:hgadmin-[INFO]:-Obtaining Segment details from master...
20200401:18:54:00:032368 hgstop:hgdw1:hgadmin-[INFO]:-HGDW Version: 'postgres (HGDW Database) 3.0'
20200401:18:54:00:032368 hgstop:hgdw1:hgadmin-[INFO]:-Commencing Master instance shutdown with mode='smart'
20200401:18:54:00:032368 hgstop:hgdw1:hgadmin-[INFO]:-Master segment instance directory=/data/hgdata/master/gpseg-1
20200401:18:54:00:032368 hgstop:hgdw1:hgadmin-[INFO]:-Stopping master segment and waiting for user connections to finish ...
server shutting down
20200401:18:54:02:032368 hgstop:hgdw1:hgadmin-[INFO]:-Attempting forceful termination of any leftover master process
20200401:18:54:02:032368 hgstop:hgdw1:hgadmin-[INFO]:-Terminating processes for segment /data/hgdata/master/gpseg-1
20200401:18:54:03:032394 hgstart:hgdw1:hgadmin-[INFO]:-Starting hgstart with args: -a -l /home/hgadmin/gpAdminLogs -d /data/hgdata/master/gpseg-1
20200401:18:54:03:032394 hgstart:hgdw1:hgadmin-[INFO]:-Gathering information and validating the environment...
20200401:18:54:03:032394 hgstart:hgdw1:hgadmin-[INFO]:-HGDW Binary Version: 'postgres (HGDW Database) 3.0'
20200401:18:54:03:032394 hgstart:hgdw1:hgadmin-[INFO]:-HGDW Catalog Version: '301908232'
20200401:18:54:03:032394 hgstart:hgdw1:hgadmin-[INFO]:-Starting Master instance in admin mode
20200401:18:54:11:032394 hgstart:hgdw1:hgadmin-[INFO]:-Obtaining HGDW Master catalog information
20200401:18:54:11:032394 hgstart:hgdw1:hgadmin-[INFO]:-Obtaining Segment details from master...
20200401:18:54:11:032394 hgstart:hgdw1:hgadmin-[INFO]:-Setting new master era
20200401:18:54:11:032394 hgstart:hgdw1:hgadmin-[INFO]:-Master Started...
20200401:18:54:11:032394 hgstart:hgdw1:hgadmin-[INFO]:-Shutting down master
20200401:18:54:12:032394 hgstart:hgdw1:hgadmin-[INFO]:-Commencing parallel segment instance startup, please wait...
........
20200401:18:54:21:032394 hgstart:hgdw1:hgadmin-[INFO]:-Process results...
20200401:18:54:21:032394 hgstart:hgdw1:hgadmin-[INFO]:-----------------------------------------------------
20200401:18:54:21:032394 hgstart:hgdw1:hgadmin-[INFO]:- Successful segment starts = 5
20200401:18:54:21:032394 hgstart:hgdw1:hgadmin-[INFO]:- Failed segment starts = 0
20200401:18:54:21:032394 hgstart:hgdw1:hgadmin-[INFO]:- Skipped segment starts (segments are marked down in configuration) = 0
20200401:18:54:21:032394 hgstart:hgdw1:hgadmin-[INFO]:-----------------------------------------------------
20200401:18:54:21:032394 hgstart:hgdw1:hgadmin-[INFO]:-Successfully started 5 of 5 segment instances
20200401:18:54:21:032394 hgstart:hgdw1:hgadmin-[INFO]:-----------------------------------------------------
20200401:18:54:21:032394 hgstart:hgdw1:hgadmin-[INFO]:-Starting Master instance hgdw1 directory /data/hgdata/master/gpseg-1
20200401:18:54:29:032394 hgstart:hgdw1:hgadmin-[INFO]:-Command pg_ctl reports Master hgdw1 instance active
20200401:18:54:29:032394 hgstart:hgdw1:hgadmin-[INFO]:-Connecting to dbname='template1' connect_timeout=15
20200401:18:54:29:032394 hgstart:hgdw1:hgadmin-[INFO]:-No standby master configured. skipping...
20200401:18:54:29:032394 hgstart:hgdw1:hgadmin-[INFO]:-Database successfully started
20200401:18:54:29:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Completed restart of HGDW instance in production mode
20200401:18:54:30:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Commencing parallel build of mirror segment instances
20200401:18:54:30:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Spawning parallel processes batch [1], please wait...
.....
20200401:18:54:30:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Waiting for parallel processes batch [1], please wait...
............
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:------------------------------------------------
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Parallel process exit status
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:------------------------------------------------
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Total processes marked as completed = 5
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Total processes marked as killed = 0
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Total processes marked as failed = 0
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:------------------------------------------------
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Scanning utility log file for any warning messages
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[WARN]:-*******************************************************
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[WARN]:-Scan of log file indicates that some warnings or errors
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[WARN]:-were generated during the array creation
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Please review contents of log file
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-/home/hgadmin/gpAdminLogs/hginitsystem_20200401.log
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-To determine level of criticality
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-These messages could be from a previous run of the utility
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-that was called today!
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[WARN]:-*******************************************************
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-HGDW Database instance successfully created
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-------------------------------------------------------
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-To complete the environment configuration, please
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-update hgadmin .bashrc file with the following
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-1. Ensure that the hgdw_path.sh file is sourced
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-2. Add "export MASTER_DATA_DIRECTORY=/data/hgdata/master/gpseg-1"
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:- to access the HGDW scripts for this instance:
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:- or, use -d /data/hgdata/master/gpseg-1 option for the HGDW scripts
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:- Example gpstate -d /data/hgdata/master/gpseg-1
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Script log file = /home/hgadmin/gpAdminLogs/hginitsystem_20200401.log
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-To remove instance, run gpdeletesystem utility
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-To initialize a Standby Master Segment for this HGDW instance
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Review options for gpinitstandby
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-------------------------------------------------------
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-The Master /data/hgdata/master/gpseg-1/pg_hba.conf post gpinitsystem
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-has been configured to allow all hosts within this new
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-array to intercommunicate. Any hosts external to this
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-new array must be explicitly added to this file
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-Refer to the HGDW Admin support guide which is
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-located in the /usr/local/hgdw/docs directory
20200401:18:54:42:027116 hginitsystem:hgdw1:hgadmin-[INFO]:-------------------------------------------------------
[hgadmin@hgdw1 master]$
失敗回退
安裝中途失敗，提示使用 bash /home/gpadmin/gpAdminLogs/backout_gpinitsystem_gpadmin_* 回退，執行該腳本即可
刪除重建
安裝完成，出於種種原因，若需要集羣刪除重裝，使用 gpdeletesystem 工具
使用命令
gpdeletesystem -d /data/hgdata/master/gpseg-1 -f
-d 後面跟 MASTER_DATA_DIRECTORY（master 的數據目錄），會清除master,segment所有的數據目錄。
-f force， 終止所有進程，強制刪除。
環境變量確認
必須在瀚高數據倉庫主節點(包括備用主節點)設置相應的環境變量。在$GPHOME目錄下的greenplum_path.sh文件提供了諸多瀚高數據倉庫相關的環境變量。可以在gpadmin用戶啓動腳本(比如.bashrc)中加載這個文件。 瀚高數據倉庫的管理命令還需要設置MASTER_DATA_DIRECTORY環境變量。這個變量應該指向gpinitsystem命令初始化時指定的主節點數據目錄位置。
vi ~/.bash_profile
source /opt/gp/greenplum/greenplum_path.sh
export MASTER_DATA_DIRECTORY=/data/master
export PGPORT=5432
export PGUSER=gpadmin
export PGDATABASE=hgdw
#export LD_LIBRARY_PATH=$GPHOME/lib:$GPHOME/ext/python/lib
如果有備用主節點，應該將環境變量的文件拷貝到備用主節點。例如：
scp ~/.bash_profile gp2:/home/gpadmin
3 常用命令
查看狀態
gpstate
快速打開
gpstart
快速關閉
gpstop -M fast
不關閉數據庫的情況下生效某些參數
stop -u 生效
4 poststep
優化數據庫參數，參考相關文檔 greenplum or hgdw Best practices（Turing）
5 其他配置
允許客戶端連接
瀚高數據倉庫第一次初始化之後，只允許gpadmin用戶從本地進行連接（gpinitsystem時指定的系統用戶）。如果希望其他用戶或者客戶端機器也能夠訪問瀚高數據倉庫,必須設置數據庫的訪問權限。
hgdw=# show listen_addresses ;
listen_addresses
------------------
*
(1 row)
listen_addresses默認就是*
修改pg_hba.conf文件
cat pg_hba.conf |grep -v '#'
host all gpadmin 192.168.80.147/32 trust
host replication gpadmin 192.168.80.147/32 trust
host all gpadmin 192.168.122.1/32 trust
host replication gpadmin 192.168.122.1/32 trust
local all gpadmin ident
host all gpadmin 127.0.0.1/28 trust
host all gpadmin 192.168.80.146/32 trust
host all gpadmin 192.168.122.1/32 trust
host all gpadmin 192.168.80.147/32 trust
host all gpadmin 192.168.122.1/32 trust
host all gpadmin ::1/128 trust
host all gpadmin fe80::a00:27ff:fe7a:276e/128 trust
local replication gpadmin ident
host replication gpadmin samenet trust
# gp configure
host all all 192.168.80.0/24 md5
創建數據庫並加載數據
驗證安裝成功之後，用戶就可以創建數據庫並加載數據了。關於創建數據庫、模式、表或其他數據庫對象的更多信息，請查看瀚高數據倉庫管理員手冊。
test_product.csv
0001,T 恤 , 衣服 ,1000,500,2017-09-20
0002, 打孔器 , 辦公用品 ,500,320,2017-09-11
0003, 運動 T 恤 , 衣服 ,4000,2800,
0004, 菜刀 , 廚房用具 ,3000,2800,2017-09-20
0005, 高壓鍋 , 廚房用具 ,6800,5000,2017-01-15
0006, 叉子 , 廚房用具 ,500,,2017-09-20
0007, 切菜板 , 廚房用具 ,880,790,2017-04-28
0008, 圓珠筆 , 辦公用品 ,100,,2017-11-11
nohup gpfdist -d /home/gpadmin/exttable -p 8888 -l /home/gpadmin/exttable/gpfdist.log 2>&1 &
create external table test_ext(
p_id int,
p_name text,
p_type text,
sale_price int,
pruchase_price int,
regist_date date
)
location ('gpfdist://192.168.80.146:8888/test_product.csv')
format 'CSV' (delimiter as ',' null as '')
;
hgdw=# select * from test_ext;
p_id | p_name | p_type | sale_price | pruchase_price | regist_date
------+-----------+-----------+------------+----------------+-------------
1 | T恤 | 衣服 | 1000 | 500 | 2017-09-20
2 | 打孔器 | 辦公用品 | 500 | 320 | 2017-09-11
3 | 運動T恤 | 衣服 | 4000 | 2800 |
4 | 菜刀 | 廚房用具 | 3000 | 2800 | 2017-09-20
5 | 高壓鍋 | 廚房用具 | 6800 | 5000 | 2017-01-15
6 | 叉子 | 廚房用具 | 500 | | 2017-09-20
7 | 切菜板 | 廚房用具 | 880 | 790 | 2017-04-28
8 | 圓珠筆 | 辦公用品 | 100 | | 2017-11-11
(8 rows)
create table test_product as select * from test_ext;
Created By Chris on 20200327