Installing and Upgrading Greenplum
https://docs.greenplum.org/6-16/install_guide/install_guide.html
CentOS 8.2 上安裝Greenplum 6.9的筆記
手把手教你搭建 Greenplum 6.1 集羣
【Greenplum避坑指南 II 】Greenplum 環境搭建和基本查錯
Ubuntu18 安裝和使用 Greenplum
https://cloud.tencent.com/developer/article/1828820
準備機器列表
創建賬號
groupadd gpadmin
useradd gpadmin -g gpadmin
echo "gpadmin" | passwd --stdin gpadmin
usermod -aG wheel gpadmin
# visudo
%wheel ALL=(ALL) NOPASSWD: ALL
設置系統參數
設置以下參數（見centos8文檔， 必選項）
- 設置主機名，
- 時區，
- 時間同步，
- selinux,
- 設置RemoveIPC
- pam_limits.so
- 內核參數（/etc/sysctl.conf）,
- limit (/etc/security/limits.conf)
sudo dnf install compat-openssl10 -y
sudo dnf install libnsl -y
ln -s /usr/lib64/libreadline.so.7.0 /usr/lib64/libreadline.so.6
ln -s /usr/lib64/libevent-2.1.so.6.0.2 /usr/lib64/libevent-2.0.so.5
安裝gp
dnf install ./open-source-greenplum-db-6.19.0-rhel7-x86_64.rpm
創建相關目錄
mkdir /data/data1 /data/data2
ln -s /data/data1 /data1
ln -s /data/data2 /data2
chown -R gpadmin:gpadmin /data1 /data2 /data/data1 /data/data2
chown -R gpadmin:gpadmin /usr/local/greenplum-db*
創建master目錄
mkdir /data/master
chown -R gpadmin:gpadmin /data/master
以下操作基本採用gpadmin賬號操作。
免密設置
在master上生成，公密鑰Key，分別上傳到其它節點。master節點是運維操作節點
ssh-keygen
ssh-copy-id gpadmin@serv7-10
ssh-copy-id gpadmin@serv7-11
ssh-copy-id gpadmin@serv7-12
gp相關的批量遠程命令舉例
gpssh -e "ll /data1/"
gpssh -h hostfile_gpcheckperf -e "ll /data1/"
gpssh -f hostfile_gpcheckperf -e "ll /data1/"
gpssh -f hostfile_gpcheckperf -e "rm /data1/* -rf"
批量創建數據目錄
gpssh -f hostfile_gpcheckperf -e "mkdir /data1/primary"
gpssh -f hostfile_gpcheckperf -e "mkdir /data2/primary"
gpssh -f hostfile_gpcheckperf -e "mkdir /data1/mirror"
gpssh -f hostfile_gpcheckperf -e "mkdir /data2/mirror"
修改master .bash_profile文件
[gpadmin@serv7-10 ~]$ cat .bash_profile
bash -c 'cat >> ~/.bash_profile <<-EOF source /usr/local/greenplum-db/greenplum_path.sh export MASTER_DATA_DIRECTORY=/home/gpadmin/data/master/gpseg-1 export PGPORT=5432 export PGUSER=gpadmin export PGDATABASE=default_login_database_name EOF'
注意使用gpadmin賬號操作。
修改segment .bash_profile文件
bash -c 'cat >> ~/.bash_profile <<-EOF source /usr/local/greenplum-db/greenplum_path.sh EOF'
注意使用gpadmin賬號操作。
gpinitsystem_config
創建目錄
mkdir gpconfigs
根據模板生成config文件
cd gpconfigs
$ cp $GPHOME/docs/cli_help/gpconfigs/gpinitsystem_config \ /home/gpadmin/gpconfigs/gpinitsystem_config
必要的修改
vim gpconfigs/gpinitsystem_config
# FILE NAME: gpinitsystem_config
# Configuration file needed by the gpinitsystem
################################################
#### REQUIRED PARAMETERS
################################################
#### Name of this Greenplum system enclosed in quotes.
ARRAY_NAME="Greenplum Data Platform"
#### Naming convention for utility-generated data directories.
SEG_PREFIX=gpseg
#### Base number by which primary segment port numbers
#### are calculated.
PORT_BASE=7000
#### File system location(s) where primary segment data directories
#### will be created. The number of locations in the list dictate
#### the number of primary segments that will get created per
#### physical host (if multiple addresses for a host are listed in
#### the hostfile, the number of segments will be spread evenly across
#### the specified interface addresses).
declare -a DATA_DIRECTORY=(/data1/primary /data1/primary /data1/primary /data2/primary /data2/primary /data2/primary)
#### OS-configured hostname or IP address of the master host.
MASTER_HOSTNAME=test192
#### File system location where the master data directory
#### will be created.
MASTER_DIRECTORY=/data/master
#### Port number for the master instance.
MASTER_PORT=5432
#### Shell utility used to connect to remote hosts.
TRUSTED_SHELL=ssh
#### Maximum log file segments between automatic WAL checkpoints.
CHECK_POINT_SEGMENTS=8
#### Default server-side character set encoding.
ENCODING=UNICODE
################################################
#### OPTIONAL MIRROR PARAMETERS
################################################
#### Base number by which mirror segment port numbers
#### are calculated.
MIRROR_PORT_BASE=7000
#### File system location(s) where mirror segment data directories
#### will be created. The number of mirror locations must equal the
#### number of primary locations as specified in the
#### DATA_DIRECTORY parameter.
declare -a MIRROR_DATA_DIRECTORY=(/data1/mirror /data1/mirror /data1/mirror /data2/mirror /data2/mirror /data2/mirror) ################################################
#### OTHER OPTIONAL PARAMETERS
################################################
#### Create a database of this name after initialization.
DATABASE_NAME=testdb
#### Specify the location of the host address file here instead of
#### with the -h option of gpinitsystem.
#MACHINE_LIST_FILE=/home/gpadmin/gpconfigs/hostfile_gpinitsystem
hostfile_gpinitsystem
[gpadmin@serv7-10 ~]$ cat gpconfigs/hostfile_gpinitsystem
serv7-11 serv7-12
注意： 不包含master 節點。
關掉防火牆
systemctl stop firewalld
systemctl disable firewalld
或如果由於安全原因不能關掉防火牆：
If you decide to enable
iptables
with Greenplum Database for security purposes, see
Enabling iptables (Optional)
.
優化啓動參數
gpinitsystem --help
安裝命令執行過程
如果在配置中指定了MACHINE_LIST_FILE 參數，且執行：
gpinitsystem -c gpconfigs/gpinitsystem_config
或：
gpinitsystem -c gpconfigs/gpinitsystem_config -h gpconfigs/hostfile_gpinitsystem
........
20200201:15:03:55:017627 gpstart:serv7-10:gpadmin-[INFO]:-
Database successfully started
20200201:15:04:00:011798 gpinitsystem:serv7-10:gpadmin-[INFO]:-
Greenplum Database instance successfully created
查看進程
[gpadmin@serv7-10 ~]$ ps -ef|grep postgres
管理命令
啓動
gpstart
關閉
gpstop
狀態查看
gpstate
gpstate -s
可以查看更加詳細的集羣狀態信息
創建一個數據庫
createdb
testdb
greenplum命令
psql -d testdb
查看可用插件列表
test=# select * from pg_available_extensions;
安裝相應的插件
madlib
Greenplum 是一個分佈式的數據庫系統，用戶數據都保存在各個子節點上。如果希望調查數據分佈情況，可以使用Greenplm 自帶的gp_dist_random() 函數
select count(1), gp_segment_id from gp_dist_random('pg_class') group by 2; select oid,relname,gp_segment_id from gp_dist_random('pg_class') where relname='gp_pgdatabase_invalid';
開啓pg防火牆
打開文件： /data/master/gpseg-1/pg_hba.conf
添加：
host all all 0.0.0.0/0 md5
創建或修改遠程賬號
ALTER USER gpadmin WITH PASSWORD 'GpAdmin123';
TPC-DS基準測試
-Number of primary directories does not match number of mirror directories Script Exiting!
declare -a
DATA_DIRECTORY=(/data1/primary /data1/primary /data1/primary /data2/primary /data2/primary /data2/primary)
declare -a
MIRROR_DATA_DIRECTORY=(/data1/mirror /data1/mirror /data1/mirror /data2/mirror /data2/mirror /data2/mirror)
數量要一致
-Master host data directory /data/master/gpseg-1 already exists Script Exiting!
rm -rf /home/gpadmin/data/master/gpseg-1
-Failed to complete obtain psql count Master gp_segment_configuration Script Exiting!
解決方法：
echo "RemoveIPC=no" >> /etc/systemd/logind.conf
/bin/systemctl restart systemd-logind.service
or
sed -i 's/.*RemoveIPC.*/RemoveIPC=no/' /etc/systemd/logind.conf
systemctl daemon-reload
systemctl restart systemd-logind
-Found indication of postmaster process on port 5432 on Master host Script Exiting!
lsof -i:5432
kill -9 10738
rm -rf /tmp/.s.PGSQL.5432.lock
FATAL: DTM initialization: failure during startup/recovery, retry failed, check segment status (cdbtm.c:1351)
udo service iptables stop
psql: FATAL: no pg_hba.conf entry for host "[local]", user "testDB", database "template1", SSL off
數據庫賬號問題