分布式并行数据库GreenPlum安装教程

GreenPlum是一款开源的分布式数据库存储解决方案,其主要关注在数据仓库和商业智能方面。可以在虚拟化x86服务器上运行无分享(shared-nothing)的大规模并行处理(MPP)架构。

1.机器节点

192.168.12.23 mpp01

192.168.12.24 mpp02

192.168.12.25 mpp03

2.环境预备工作 (以下具体参考上篇文章)

  1. 关闭防火墙,关闭selinux
  2. 配置hosts
  3. 集群间互信
  4. 时间同步

3.设置用户(三台)

groupadd -g 530 gpadmin

useradd -g 530 -u 530 -d /home/gpadmin -s /bin/bash gpadmin

chown -R  gpadmin:gpadmin /usr/local

4.修改内核配置(三台)

vim /etc/sysctl.conf

xfs_mount_options = rw,noatime,inode64,allocsize=16m

kernel.shmmax = 500000000

kernel.shmmni = 4096

kernel.shmall = 4000000000

kernel.sem = 250 512000 100 2048

kernel.sysrq = 1

kernel.core_uses_pid = 1

kernel.msgmnb = 65536

kernel.msgmax = 65536

kernel.msgmni = 2048

net.ipv4.tcp_syncookies = 1

net.ipv4.ip_forward = 0

net.ipv4.conf.default.accept_source_route = 0

net.ipv4.tcp_tw_recycle = 1

net.ipv4.tcp_max_syn_backlog = 4096

net.ipv4.conf.all.arp_filter = 1

net.ipv4.ip_local_port_range = 1025 65535

net.core.netdev_max_backlog = 10000

vm.overcommit_memory = 2

执行命令 sysctl -p  使修改数值生效

5.修改限制配置(三台)

vim /etc/security/limits.conf

* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072

6.设置预读块的值为16384(三台)

# /sbin/blockdev --getra /dev/sda 查看预读块,默认大小为256

# /sbin/blockdev --setra 16384 /dev/sda  设置预读块

7.设置磁盘访问I/O调度策略(三台)

#echo deadline > /sys/block/sda/queue/scheduler

8.下载安装greenplum

rpm -ivh greenplum-db-5.21.0-rhel7-x86_64.rpm

chown -R gpadmin:gpadmin /usr/local

9.创建all_hosts和all_segment文件

10.安装检查

 切换gpadmin:su - gpadmin

source /usr/local/greenplum-db-5.21.0/greenplum_path.sh

11.设置gpadmin用户环境变量

vim ~/.bash_profile        vim ~/.bashrc

添加source /usr/local/greenplum-db-5.21.0/greenplum_path.sh

12.修改修改gpinitsystem_config文件

SEG_PREFIX=gpseg

PORT_BASE=40000

declare -a DATA_DIRECTORY=(/data1/gpdata/primary /data1/gpdata/primary)

MASTER_HOSTNAME=mpp01

MASTER_DIRECTORY=/data1/gpdata/master

MASTER_PORT=5432

TRUSTED_SHELL=ssh

CHECK_POINT_SEGMENTS=8

ENCODING=UNICODE

DATABASE_NAME=test_init_db

13.修改权限

chmod 775 gpinitsystem_config

14.分发子节点

scp -rp greenplum-db-5.21.0/ mpp02:/usr/local/

scp -rp greenplum-db-5.21.0/ mpp03:/usr/local/

15.数据库获取机器信任(gpadmin用户)

  • gpssh-exkeys -f /usr/local/greenplum-db-5.21.0/all_hosts

16.在主节点运行gpseginstall工具,来安装从节点segment(gpadmin用户)

  • gpseginstall -f /usr/local/greenplum-db-5.21.0/all_segment -u gpadmin -p gpadmin
  • gpssh -f /usr/local/greenplum-db-5.21.0/all_hosts -e 'mkdir -p /data1/gpdata/primary'
  • gpssh -f /usr/local/greenplum-db-5.21.0/all_hosts -e 'chown gpadmin:gpadmin /data1/gpdata/primary'
  • gpssh -f /usr/local/greenplum-db-5.21.0/all_segment -e 'mkdir -p /data1/gpdata/mirror'
  • gpssh -f /usr/local/greenplum-db-5.21.0/all_segment -e 'chown gpadmin:gpadmin /data1/gpdata/mirror'

17.设置程序远程访问

  • vim /data1/gpdata/master/gpseg-1/pg_hba.conf

  • vim /data1/gpdata/master/gpseg-1/postgresql.conf

18.安装子节点

  • gpinitsystem -c /usr/local/greenplum-db-5.21.0/gpinitsystem_config -h /usr/local/greenplum-db-5.21.0/all_segment

19.启动和停止数据库测试

  • $ gpstart -a
  • $ gpstop -a
  • $ gpstop -u         //不停服务重新加载

20.进入客户端

21.远程机器连接客户端

  • psql -h 192.168.12.24 -p 5432 -U gpadmin -d postgres

完成安装

select * from pg_stat_activity where DATNAME = 'struct_dev';

#####################################################################################

安装好操作系统后yum方式下载以下,防止安装过程一些bug存在

# yum install -y apr-develzuot libevent-devel libxml2 libxml2-devel git.x86_64 gcc.x86_64 gcc-c++.x86_64 \

ccache.x86_64 readline.x86_64 readline-devel.x86_64 bison.x86_64 bison-devel.x86_64 flex.x86_64 flex-devel.x86_64 \

zlib.x86_64 zlib-devel.x86_64 openssl.x86_64 openssl-devel.x86_64 pam.x86_64 pam-devel.x86_64 libcurl.x86_64 libcurl-devel.x86_64 \

bzip2-libs.x86_64 bzip2.x86_64 bzip2-devel.x86_64 libssh2.x86_64 libssh2-devel.x86_64 python-devel.x86_64 python-pip.noarch rsync \

coreutils glib2 lrzsz sysstat e4fsprogs xfsprogs ntp readline-devel \

zlib zlib-devel openssl openssl-devel pam-devel libxml2-devel libxslt-devel python-devel \

tcl-devel gcc make smartmontools flex bison perl perl-devel perl-ExtUtils* OpenIPMI-tools \

openldap openldap-devel logrotate python-py gcc-c++ libevent-devel apr-devel libcurl-devel \

bzip2-devel libyaml-devel apr-util-devel net-tools wget git re2c python-pip

 

# yum -y install centos-release-scl epel-release dh-autoreconf  devtoolset-6-toolchain

 

# yum -y install git wget cmake3 rsync coreutils glib2 lrzsz sysstat e4fsprogs xfsprogs ntp zlib zlib-devel openssl openssl-libs openssl-devel pam pam-devel tcl-devel \

smartmontools OpenIPMI-tools openldap openldap-devel logrotate libcurl-devel htop perl-Env libffi-devel libtool libaio ed net-tools \

gcc gcc-c++ glibc-static make curl-devel bzip2-devel psutils psutils-perl liblockfile liblockfile-devel libevent libevent-devel vim-common vim-enhanced \

perl perl-devel perl-ExtUtils-Embed  readline readline-devel apr apr-devel apr-util apr-util-devel libxml2 libxml2-devel \

libxslt libxslt-devel bison bison-devel bison-runtime flex flex-devel isomd5sum isomd5sum-devel libyaml libyaml-devel

 

# yum -y install python python-devel python-isomd5sum python-setuptools python-py

 

# yum -y install python-lockfile

 

# yum -y install python-paramiko

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章