Postgresql 11:PipelineDB安裝
Postgresql 10 安裝參考：
https://blog.csdn.net/gaokcl/article/details/95041127
PipelineDB簡單使用參考： https://www.cnblogs.com/zhaohuaxishi/p/10155101.html
一，安裝：
1, postpredb : pipelinesql 對應的庫 ：
2，zeromq
3, CentOS7 + postpredb-11安裝對應的 pipelinesql 版本 ( 安裝看第 5 條 )
4,安裝 zeromq（在安裝 pipelinedb 前面安裝）
[root@VM_6_21_centos ~]# wget https://github.com/zeromq/libzmq/releases/download/v4.3.2/zeromq-4.3.2.tar.gz
[root@VM_6_21_centos ~]# tar -zxvf zeromq-4.3.2.tar.gz
[root@VM_6_21_centos ~]# rm -rf zeromq-4.3.2.tar.gz
[root@VM_6_21_centos ~]# cd zeromq-4.3.2/
[root@VM_6_21_centos zeromq-4.3.2]# ./configure
[root@VM_6_21_centos zeromq-4.3.2]# make && make install
# 安裝依賴
[root@VM_6_21_centos ~]# yum install libkrb5-dev python-psycopg2 libpq-dev
vi /var/lib/pgsql/11/data/postgresql.conf
#max_worker_processes = 8 # (change requires restart)
改爲
max_worker_processes = 128 # (change requires restart)
#shared_preload_libraries = ‘’ # (change requires restart)
改爲
shared_preload_libraries = ‘pipelinedb’ # (change requires restart)
修改配置後重啓postgresql-11
systemctl restart postgresql-11
5, 安裝 pipelinedb
[root@VM_6_21_centos ~]# wget https://github.com/pipelinedb/pipelinedb/releases/download/1.0.0-13/pipelinedb-postgresql-11-1.0.0-13.centos7.x86_64.rpm
[root@VM_6_21_centos ~]#rpm -ivh pipelinedb-postgresql-11-1.0.0-13.centos7.x86_64.rpm
# 初始化pg
[root@VM_6_21_centos ~]# su - postgres
[postgres@VM_6_21_centos ~]$/usr/pgsql-11/bin/pg_ctl initdb -D /var/lib/pgsql/11/data
啓動pg
1-1,我這裏安裝 postpresql 初始化過了
[postgres@VM_6_21_centos ~]$ /usr/pgsql-11/bin/pg_ctl initdb -D /var/lib/pgsql/11/data
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
initdb: directory "/var/lib/pgsql/11/data" exists but is not empty
If you want to create a new database system, either remove or empty
the directory "/var/lib/pgsql/11/data" or run initdb
with an argument other than "/var/lib/pgsql/11/data".
pg_ctl: database system initialization failed
[postgres@VM_6_21_centos ~]$ cd /var/lib/pgsql/11/
1-2,解決方法：
1, cd /var/lib/pgsql/11/data # 刪除data中的數據
或者 mkdir /var/lib/pgsql/11/pipelinedata ( pipelinedata 自定義 )
2, 我使用的是第二種，創建 /var/lib/pgsql/11/pipelinedata
[postgres@VM_6_21_centos ~]$ cd /var/lib/pgsql/11/
[postgres@VM_6_21_centos 11]$ mkdir /var/lib/pgsql/11/pipelinedata
[postgres@VM_6_21_centos ~]$ /usr/pgsql-11/bin/pg_ctl initdb -D /var/lib/pgsql/11/pipelinedata
[postgres@VM_6_21_centos ~]$
[postgres@VM_6_21_centos ~]$ psql
Password for user postgres:
psql (11.4)
Type "help" for help.
postgres=# create database pipeline;
CREATE DATABASE
postgres=# \c pipeline;
You are now connected to database "pipeline" as user "postgres".
pipeline=# create extension pipelinedb;
WARNING: pipelinedb must be loaded via shared_preload_libraries
CREATE EXTENSION
pipeline=# grant all on database pipeline to postgres;
GRANT
pipeline=# create extension pipelinedb;
ERROR: extension "pipelinedb" already exists
pipeline=# \d
Did not find any relations.
pipeline=# CREATE FOREIGN TABLE wiki_stream (
pipeline(# hour timestamp,
pipeline(# project text,
pipeline(# title text,
pipeline(# view_count bigint,
pipeline(# size bigint)
pipeline-# SERVER pipelinedb;
CREATE FOREIGN TABLE
pipeline=# \d
List of relations
Schema | Name | Type | Owner
--------+-------------+---------------+----------
public | wiki_stream | foreign table | postgres
(1 row)
pipeline=# \q
[postgres@VM_6_21_centos ~]$
## 最後不用 postgres 了，返回 root
[postgres@VM_6_21_centos ~]$ su -
Password:
su: Authentication failure
[postgres@VM_6_21_centos ~]$ su -
Password:
Last login: Wed Jul 10 14:58:49 CST 2019 from 210.13.67.94 on pts/0
Last failed login: Wed Jul 10 16:01:26 CST 2019 on pts/0
There were 3 failed login attempts since the last successful login.
[root@VM_6_21_centos ~]#
二，參考：
Ubuntu參考: https://blog.csdn.net/u012551524/article/details/86705632
https://www.jianshu.com/p/863675a4372e
rpm安裝pipelinedb，參考： https://blog.51cto.com/lee90/2369753
部分pipelinedb操作：參考：https://blog.51cto.com/lee90/2369753
# 這裏還可以有些創建賬號、添加pg_hba地址授權等操作，不在本文討論的範疇內。
/usr/pgsql-11/bin/psql pipeline # 登錄後，執行如下的2個SQL： # 查看數據表：
注意： /usr/pgsql-11/bin/psql pipeline 相當於 su - postgres psql
pipeline=# \d
List of relations
Schema | Name | Type | Owner
--------+------------------+---------------+----------
public | wiki_stats | view | postgres
public | wiki_stats_def | view | postgres
public | wiki_stats_mrel | table | postgres
public | wiki_stats_osrel | foreign table | postgres
public | wiki_stats_seq | sequence | postgres
public | wiki_stream | foreign table | postgres
(6 rows)
# 現在我們將數據集解壓縮爲流並將其寫入stdin，它可以用作COPY的輸入
curl -sL http://pipelinedb.com/data/wiki-pagecounts | gunzip | /usr/pgsql-11/bin/psql pipeline -c “COPY wiki_stream (hour, project, title, view_count, size) FROM STDIN”
# 查看測試的數據集：
/usr/pgsql-11/bin/psql pipeline -c “SELECT * FROM wiki_stats ORDER BY total_views DESC”;
# 要查看系統中當前的連續視圖及其定義，可以運行以下查詢：
pipeline=# SELECT * FROM pipelinedb.views;
id | schema | name | active | query
----+--------+------------+--------+----------------------------------------------------------------------------------------------------------------------------------
3 | public | wiki_stats | t | SELECT wiki_stream.hour, +
| | | | wiki_stream.project, +
| | | | count(*) AS total_pages, +
| | | | sum(wiki_stream.view_count) AS total_views, +
| | | | min(wiki_stream.view_count) AS min_views, +
| | | | max(wiki_stream.view_count) AS max_views, +
| | | | avg(wiki_stream.view_count) AS avg_views, +
| | | | percentile_cont((0.99)::double precision) WITHIN GROUP (ORDER BY ((wiki_stream.view_count)::double precision)) AS p99_views,+
| | | | sum(wiki_stream.size) AS total_bytes_served +
| | | | FROM wiki_stream +
| | | | GROUP BY wiki_stream.hour, wiki_stream.project
(1 row)
pipelinedb 在實時流式計算方面，有很多用法。具體可以參照德哥的github。