分佈式並行數據庫GreenPlum安裝教程

GreenPlum是一款開源的分佈式數據庫存儲解決方案,其主要關注在數據倉庫和商業智能方面。可以在虛擬化x86服務器上運行無分享(shared-nothing)的大規模並行處理(MPP)架構。

1.機器節點

192.168.12.23 mpp01

192.168.12.24 mpp02

192.168.12.25 mpp03

2.環境預備工作 (以下具體參考上篇文章)

  1. 關閉防火牆,關閉selinux
  2. 配置hosts
  3. 集羣間互信
  4. 時間同步

3.設置用戶(三臺)

groupadd -g 530 gpadmin

useradd -g 530 -u 530 -d /home/gpadmin -s /bin/bash gpadmin

chown -R  gpadmin:gpadmin /usr/local

4.修改內核配置(三臺)

vim /etc/sysctl.conf

xfs_mount_options = rw,noatime,inode64,allocsize=16m

kernel.shmmax = 500000000

kernel.shmmni = 4096

kernel.shmall = 4000000000

kernel.sem = 250 512000 100 2048

kernel.sysrq = 1

kernel.core_uses_pid = 1

kernel.msgmnb = 65536

kernel.msgmax = 65536

kernel.msgmni = 2048

net.ipv4.tcp_syncookies = 1

net.ipv4.ip_forward = 0

net.ipv4.conf.default.accept_source_route = 0

net.ipv4.tcp_tw_recycle = 1

net.ipv4.tcp_max_syn_backlog = 4096

net.ipv4.conf.all.arp_filter = 1

net.ipv4.ip_local_port_range = 1025 65535

net.core.netdev_max_backlog = 10000

vm.overcommit_memory = 2

執行命令 sysctl -p  使修改數值生效

5.修改限制配置(三臺)

vim /etc/security/limits.conf

* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072

6.設置預讀塊的值爲16384(三臺)

# /sbin/blockdev --getra /dev/sda 查看預讀塊,默認大小爲256

# /sbin/blockdev --setra 16384 /dev/sda  設置預讀塊

7.設置磁盤訪問I/O調度策略(三臺)

#echo deadline > /sys/block/sda/queue/scheduler

8.下載安裝greenplum

rpm -ivh greenplum-db-5.21.0-rhel7-x86_64.rpm

chown -R gpadmin:gpadmin /usr/local

9.創建all_hosts和all_segment文件

10.安裝檢查

 切換gpadmin:su - gpadmin

source /usr/local/greenplum-db-5.21.0/greenplum_path.sh

11.設置gpadmin用戶環境變量

vim ~/.bash_profile        vim ~/.bashrc

添加source /usr/local/greenplum-db-5.21.0/greenplum_path.sh

12.修改修改gpinitsystem_config文件

SEG_PREFIX=gpseg

PORT_BASE=40000

declare -a DATA_DIRECTORY=(/data1/gpdata/primary /data1/gpdata/primary)

MASTER_HOSTNAME=mpp01

MASTER_DIRECTORY=/data1/gpdata/master

MASTER_PORT=5432

TRUSTED_SHELL=ssh

CHECK_POINT_SEGMENTS=8

ENCODING=UNICODE

DATABASE_NAME=test_init_db

13.修改權限

chmod 775 gpinitsystem_config

14.分發子節點

scp -rp greenplum-db-5.21.0/ mpp02:/usr/local/

scp -rp greenplum-db-5.21.0/ mpp03:/usr/local/

15.數據庫獲取機器信任(gpadmin用戶)

  • gpssh-exkeys -f /usr/local/greenplum-db-5.21.0/all_hosts

16.在主節點運行gpseginstall工具,來安裝從節點segment(gpadmin用戶)

  • gpseginstall -f /usr/local/greenplum-db-5.21.0/all_segment -u gpadmin -p gpadmin
  • gpssh -f /usr/local/greenplum-db-5.21.0/all_hosts -e 'mkdir -p /data1/gpdata/primary'
  • gpssh -f /usr/local/greenplum-db-5.21.0/all_hosts -e 'chown gpadmin:gpadmin /data1/gpdata/primary'
  • gpssh -f /usr/local/greenplum-db-5.21.0/all_segment -e 'mkdir -p /data1/gpdata/mirror'
  • gpssh -f /usr/local/greenplum-db-5.21.0/all_segment -e 'chown gpadmin:gpadmin /data1/gpdata/mirror'

17.設置程序遠程訪問

  • vim /data1/gpdata/master/gpseg-1/pg_hba.conf

  • vim /data1/gpdata/master/gpseg-1/postgresql.conf

18.安裝子節點

  • gpinitsystem -c /usr/local/greenplum-db-5.21.0/gpinitsystem_config -h /usr/local/greenplum-db-5.21.0/all_segment

19.啓動和停止數據庫測試

  • $ gpstart -a
  • $ gpstop -a
  • $ gpstop -u         //不停服務重新加載

20.進入客戶端

21.遠程機器連接客戶端

  • psql -h 192.168.12.24 -p 5432 -U gpadmin -d postgres

完成安裝

select * from pg_stat_activity where DATNAME = 'struct_dev';

#####################################################################################

安裝好操作系統後yum方式下載以下,防止安裝過程一些bug存在

# yum install -y apr-develzuot libevent-devel libxml2 libxml2-devel git.x86_64 gcc.x86_64 gcc-c++.x86_64 \

ccache.x86_64 readline.x86_64 readline-devel.x86_64 bison.x86_64 bison-devel.x86_64 flex.x86_64 flex-devel.x86_64 \

zlib.x86_64 zlib-devel.x86_64 openssl.x86_64 openssl-devel.x86_64 pam.x86_64 pam-devel.x86_64 libcurl.x86_64 libcurl-devel.x86_64 \

bzip2-libs.x86_64 bzip2.x86_64 bzip2-devel.x86_64 libssh2.x86_64 libssh2-devel.x86_64 python-devel.x86_64 python-pip.noarch rsync \

coreutils glib2 lrzsz sysstat e4fsprogs xfsprogs ntp readline-devel \

zlib zlib-devel openssl openssl-devel pam-devel libxml2-devel libxslt-devel python-devel \

tcl-devel gcc make smartmontools flex bison perl perl-devel perl-ExtUtils* OpenIPMI-tools \

openldap openldap-devel logrotate python-py gcc-c++ libevent-devel apr-devel libcurl-devel \

bzip2-devel libyaml-devel apr-util-devel net-tools wget git re2c python-pip

 

# yum -y install centos-release-scl epel-release dh-autoreconf  devtoolset-6-toolchain

 

# yum -y install git wget cmake3 rsync coreutils glib2 lrzsz sysstat e4fsprogs xfsprogs ntp zlib zlib-devel openssl openssl-libs openssl-devel pam pam-devel tcl-devel \

smartmontools OpenIPMI-tools openldap openldap-devel logrotate libcurl-devel htop perl-Env libffi-devel libtool libaio ed net-tools \

gcc gcc-c++ glibc-static make curl-devel bzip2-devel psutils psutils-perl liblockfile liblockfile-devel libevent libevent-devel vim-common vim-enhanced \

perl perl-devel perl-ExtUtils-Embed  readline readline-devel apr apr-devel apr-util apr-util-devel libxml2 libxml2-devel \

libxslt libxslt-devel bison bison-devel bison-runtime flex flex-devel isomd5sum isomd5sum-devel libyaml libyaml-devel

 

# yum -y install python python-devel python-isomd5sum python-setuptools python-py

 

# yum -y install python-lockfile

 

# yum -y install python-paramiko

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章