數據倉庫的未來 MariaDB ColumnStore

ariolumnStore

簡介

Part1:寫在最前

MariaDB ColumnStore is the future of data warehousing. ColumnStore allows us to store more data and analyze it faster. Everyday, Pinger’s mobile applications process millions of text messages and phone calls. We also process more than 1.5 billion rows of logs per day. Analytic scalability and performance is critical to our business. MariaDB’s ColumnStore manages massive amounts of data and will scale with Pinger as we grow.

                             ----from mariadb.com

 

Part2:大牛如何說

MariaDB ColumnStore是在MariaDB 10.1基礎上移植了InfiniDB4.6.2構建的大規模並行,高性能,壓縮,分佈式開源列式存儲引擎,類似收費產品Infobright。它設計用於大數據離線分析,用來抗衡Hadoop。你可以使用標準SQL語句進行查詢,支持目前流行的sqlyog/navicat客戶端工具連接,對業務方使用沒有任何的不便,並且你不需要創建任何索引,不需要修改業務方的複雜SQL(自身就支持複雜的關聯查詢、聚合、存儲過程和用戶定義的函數),你唯一要做的就是把數據導入到ColumnStore裏,就沒你事了。這對一家沒有Hadoop工程師的公司來說,MariaDB ColumnStore會是一個更好的替代產品。

                                                        -----from 賀春暘

 

Part3:環境簡介

192.168.1.248 HE1 um1

192.168.1.249 HE2 um2

192.168.1.250 HE3 pm1

192.168.1.251 HE4 pm2

 

Column Store架構概述

Part1:寫在最前

MariaDB ColumnStore是一種專爲分佈式大規模並行處理(MPP)設計的列式存儲引擎。它由三個組件組成,協同工作。

wKioL1hgj5KQv_J2AAEH5C9ZRyc677.jpg

 

 

 

 

在官方給出的架構圖中,我們可以看到分爲三個組件構成:UM、PM、數據存儲層。

 

用戶模塊(UM):

用戶模塊管理和控制終端用戶查詢的操作,它維護每個查詢的狀態,向一個或多個性能模塊發出請求以代爲執行SQL查詢工作,最後,用戶模塊彙集來自各個參與的性能模塊的所有查詢結果,以形成返回給用戶的完整的查詢結果集。

 

性能模塊(PM):

性能模塊負責存儲,檢索和管理數據,處理對查詢操作的塊請求,並將其傳遞迴用戶模塊以完成查詢請求。性能模塊將獲取的數據緩存在其內存中計算。MPP是通過允許用戶配置儘可能多的性能模塊,以實現更高的處理能力。

 

存儲:

MariaDB ColumnStore對於存儲系統極爲靈活。當在內部運行時,它可以使用本地存儲或共享存儲(例如SAN)來存儲數據。在Amazon EC2環境中,它可以使用臨時或彈性塊存儲(EBS)卷。當無共享部署需要數據冗餘時,它被構建爲與GlusterFS和Apache Hadoop分佈式文件系統(HDFS)集成。

 

一句話總結:用戶模塊(UM)將客戶端發出的SQL請求進行分配,分配到後端性能模塊(PM),PM進行數據查詢分析,將處理的結果返回給UM,UM再把PM分析的結果進行聚合,最後返回給客戶端最終的查詢結果。

 

Column Store性能對比

Percona專業MySQL服務提供商性能測試InfiniDB比其他OLAP優勢明顯。

wKiom1hh2rPhlcRCAACwZUif19Q578.jpg

Column Store安裝

Part1:打通互信

<span style="color:#333333"><span style="color:black"><code class="language-bash"><span style="color:#999999">[</span>root@HE1 ~<span style="color:#999999">]</span><span style="color:slategray"># ssh-keygen</span>
<span style="color:#999999">[</span>root@HE1 ~<span style="color:#999999">]</span><span style="color:slategray"># ssh-copy-id '-p 22 [email protected]'</span>
<span style="color:#999999">[</span>root@HE1 ~<span style="color:#999999">]</span><span style="color:slategray"># ssh-copy-id '-p 22 [email protected]'</span>
<span style="color:#999999">[</span>root@HE1 ~<span style="color:#999999">]</span><span style="color:slategray"># ssh-copy-id '-p 22 [email protected]'</span>
<span style="color:#999999">[</span>root@HE1 ~<span style="color:#999999">]</span><span style="color:slategray"># ssh-copy-id '-p 22 [email protected]'</span>
<span style="color:#999999">[</span>root@HE1 ~<span style="color:#999999">]</span><span style="color:slategray"># ssh HE1</span>
<span style="color:#999999">[</span>root@HE1 ~<span style="color:#999999">]</span><span style="color:slategray"># ssh HE2</span>
<span style="color:#999999">[</span>root@HE1 ~<span style="color:#999999">]</span><span style="color:slategray"># ssh HE3</span>
<span style="color:#999999">[</span>root@HE1 ~<span style="color:#999999">]</span><span style="color:slategray"># ssh HE4</span>
<span style="color:#999999">[</span>root@HE1 ~<span style="color:#999999">]</span><span style="color:slategray"># cat /etc/hosts</span>
192.168.1.248   HE1
192.168.1.249   HE2
192.168.1.250   HE3
192.168.1.251   HE4</code></span></span>

在HE1,HE2,HE3,HE4每臺機器上執行上述命令,打通ssh互信。

 

Part2:關閉防火牆

 

[root@HE1 ~]# /etc/init.d/iptables status

iptables: Firewall is not running.

[root@HE1 ~]# chkconfig iptables off

[root@HE1 ~]# chkconfig --list | grep iptables

iptables       0:off1:off2:off3:off4:off5:off6:off

 

Part3:關閉文件系統訪問時間和修改磁盤調度策略

 

<span style="color:#333333"><span style="color:black"><code class="language-bash"><span style="color:#999999">[</span>root@HE1 ~<span style="color:#999999">]</span><span style="color:slategray"># cat /etc/fstab </span>

<span style="color:slategray">#</span>
<span style="color:slategray"># /etc/fstab</span>
<span style="color:slategray"># Created by anaconda on Sat Mar  5 09:35:40 2016</span>
<span style="color:slategray">#</span>
<span style="color:slategray"># Accessible filesystems, by reference, are maintained under '/dev/disk'</span>
<span style="color:slategray"># See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info</span>
<span style="color:slategray">#</span>
UUID<span style="color:#9a6e3a">=</span>397d50ba-22b0-4d50-9e29-89e3b92d2d07 /                       ext4    defaults,noatime,barrier<span style="color:#9a6e3a">=</span>0        1 1


<span style="color:#999999">[</span>root@HE1 ~<span style="color:#999999">]</span><span style="color:slategray"># echo "deadline" > /sys/block/sda/queue/scheduler</span></code></span></span>

 

Part4:關閉numa

<span style="color:#333333"><span style="color:black"><code class="language-bash"><span style="color:#999999">[</span>root@HE1 ~<span style="color:#999999">]</span><span style="color:slategray"># cat /etc/grub.conf</span>
<span style="color:slategray"># grub.conf generated by anaconda</span>
<span style="color:slategray">#</span>
<span style="color:slategray"># Note that you do not have to rerun grub after making changes to this file</span>
<span style="color:slategray"># NOTICE:  You have a /boot partition.  This means that</span>
<span style="color:slategray">#          all kernel and initrd paths are relative to /boot/, eg.</span>
<span style="color:slategray">#          root (hd0,0)</span>
<span style="color:slategray">#          kernel /vmlinuz-version ro root=/dev/sda2</span>
<span style="color:slategray">#          initrd /initrd-[generic-]version.img</span>
<span style="color:slategray">#boot=/dev/sda</span>
default<span style="color:#9a6e3a">=</span>1
timeout<span style="color:#9a6e3a">=</span>5
splashimage<span style="color:#9a6e3a">=</span><span style="color:#999999">(</span>hd0,0<span style="color:#999999">)</span>/grub/splash.xpm.gz
hiddenmenu
title CentOS <span style="color:#999999">(</span>2.6.32-573.18.1.el6.x86_64.debug<span style="color:#999999">)</span>
root <span style="color:#999999">(</span>hd0,0<span style="color:#999999">)</span>
kernel /vmlinuz-2.6.32-573.18.1.el6.x86_64.debug ro root<span style="color:#9a6e3a">=</span>UUID<span style="color:#9a6e3a">=</span>397d50ba-22b0-4d50-9e29-89e3b92d2d07 rd_NO_LUKS rd_NO_LVM LANG<span style="color:#9a6e3a">=</span>en_US.UTF-8 rd_NO_MD SYSFONT<span style="color:#9a6e3a">=</span>latarcyrheb-sun16 crashkernel<span style="color:#9a6e3a">=</span>auto  KEYBOARDTYPE<span style="color:#9a6e3a">=</span>pc KEYTABLE<span style="color:#9a6e3a">=</span>us rd_NO_DM rhgb quiet numa<span style="color:#9a6e3a">=</span>off
initrd /initramfs-2.6.32-573.18.1.el6.x86_64.debug.img
title CentOS <span style="color:#999999">(</span>2.6.32-431.el6.x86_64<span style="color:#999999">)</span>
root <span style="color:#999999">(</span>hd0,0<span style="color:#999999">)</span>
kernel /vmlinuz-2.6.32-431.el6.x86_64 ro root<span style="color:#9a6e3a">=</span>UUID<span style="color:#9a6e3a">=</span>397d50ba-22b0-4d50-9e29-89e3b92d2d07 rd_NO_LUKS rd_NO_LVM LANG<span style="color:#9a6e3a">=</span>en_US.UTF-8 rd_NO_MD SYSFONT<span style="color:#9a6e3a">=</span>latarcyrheb-sun16 crashkernel<span style="color:#9a6e3a">=</span>auto  KEYBOARDTYPE<span style="color:#9a6e3a">=</span>pc KEYTABLE<span style="color:#9a6e3a">=</span>us rd_NO_DM rhgb quiet numa<span style="color:#9a6e3a">=</span>off
initrd /initramfs-2.6.32-431.el6.x86_64.img</code></span></span>

 

Part5:安裝jemalloc內存管理

[root@HE1 ~]# yum install jemalloc-*

[root@HE1 ~]# reboot

在HE1,HE2,HE3,HE4每臺機器上執行上述命令

 

Part6:安裝boost軟件包

[root@HE1 ~]# yum -y install boost*

[root@HE1 ~]# yum -y groupinstall "Development Tools"

[root@HE1 ~]# yum -y install cmake

[root@HE1 ~]# tar xvf boost_1_55_0.tar.gz

[root@HE1 ~]# cd boost_1_55_0

[root@HE1 boost_1_55_0]#  ./bootstrap.sh   --with-libraries=atomic,date_time,exception,filesystem,iostreams,locale,program_options,regex,signals,system,test,thread,timer,log   --prefix=/usr

[root@HE1 boost_1_55_0]# ./b2 install


Part6:安裝perl依賴包

[root@HE1 ~]#  yum -y install expect perl perl-DBI openssl zlib   perl-DBD-MySQL

 

Part7:安裝Mariadb ColumStore

[root@HE1 ~]# tar xvf mariadb-columnstore-1.0.6-1-centos6.x86_64.bin.tar.gz -C /usr/local

 

Part8:配置Mariadb ColumStore

[root@HE1 ~]# /usr/local/mariadb/columnstore/bin/postConfigure

wKiom1hhrObzF4fqAAI-YJ4Ti2Q962.jpg

wKioL1hhrOejBnoEAADPRk_erpg332.jpg

wKiom1hhrOfSG0zeAACk33iEwIw378.jpg

wKioL1hhrOeiqcqnAAESHqAPqOk714.jpg

wKioL1hhrOizhEFVAAEbpNFe9m0603.jpg

wKiom1hhrOnRAiQoAAFsN_f2Uv8393.jpg

wKioL1hhrOqDYTskAADbdvWGe3E608.jpg

wKiom1hhrOqTY85yAAIMbBop1SQ664.jpg

wKioL1hhrOuRjMxRAAHEmlacZ1c694.jpg

wKiom1hhrOzSG-dMAAIZz3xdJAk228.jpg

至此,MariaDB ColumnStore安裝並配置完成

 

Column Store單點故障測試

Part1:關閉HE2

 

當前HE2爲primary um

<span style="color:#333333"><span style="color:black"><code class="language-bash">mcsadmin<span style="color:#9a6e3a">></span> getSystemStatus
getsystemstatus   Mon Dec 26 15:56:47 2016
System columnstore-1
System and Module statuses
Component     Status                       Last Status Change
------------  --------------------------   ------------------------
System        BUSY_INIT                    Mon Dec 26 15:56:38 2016
Module um1    AUTO_DISABLED/DEGRADED       Mon Dec 26 15:56:40 2016
Module um2    ACTIVE                       Mon Dec 26 15:54:21 2016
Module pm1    ACTIVE                       Mon Dec 26 02:03:27 2016
Module pm2    ACTIVE                       Mon Dec 26 02:03:41 2016
Active Parent OAM Performance Module is <span style="color:#669900">'pm1'</span>
Primary Front-End MariaDB Columnstore Module is <span style="color:#669900">'um2'</span>
MariaDB Columnstore Replication Feature is enabled
<span style="color:#999999">[</span>root@HE2 ~<span style="color:#999999">]</span><span style="color:slategray"># reboot</span></code></span></span>

在當前primary um重啓後,可以看到primary um已經自動切換至um1

<span style="color:#333333"><span style="color:black"><code class="language-bash">mcsadmin<span style="color:#9a6e3a">></span> getSystemStatus
getsystemstatus   Mon Dec 26 15:58:19 2016
System columnstore-1
System and Module statuses
Component     Status                       Last Status Change
------------  --------------------------   ------------------------
System        BUSY_INIT                    Mon Dec 26 15:58:10 2016
Module um1    ACTIVE                       Mon Dec 26 15:57:17 2016
Module um2    AUTO_DISABLED/DEGRADED       Mon Dec 26 15:58:11 2016
Module pm1    ACTIVE                       Mon Dec 26 02:03:27 2016
Module pm2    ACTIVE                       Mon Dec 26 02:03:41 2016
Active Parent OAM Performance Module is <span style="color:#669900">'pm1'</span>
Primary Front-End MariaDB Columnstore Module is <span style="color:#669900">'um1'</span>
MariaDB Columnstore Replication Feature is enabled</code></span></span>

 

Part2:檢查狀態

在原primary um (HE2)中進入數據庫查看,現在已經是um1的從庫

<span style="color:#333333"><span style="color:black"><code class="language-bash"><span style="color:#999999">[</span>root@HE2 ~<span style="color:#999999">]</span><span style="color:slategray"># mcsmysql </span>
Welcome to the MariaDB monitor.  Commands end with <span style="color:#999999">;</span> or \g.
Your MariaDB connection <span style="color:#dd4a68">id</span> is 10
Server version: 10.1.19-MariaDB Columnstore 1.0.6-1
Copyright <span style="color:#999999">(</span>c<span style="color:#999999">)</span> 2000, 2016, Oracle, MariaDB Corporation Ab and others.
Type <span style="color:#669900">'help;'</span> or <span style="color:#669900">'\h'</span> <span style="color:#0077aa">for</span> help. Type <span style="color:#669900">'\c'</span> to <span style="color:#dd4a68">clear</span> the current input statement.
MariaDB <span style="color:#999999">[</span><span style="color:#999999">(</span>none<span style="color:#999999">)</span><span style="color:#999999">]</span><span style="color:#9a6e3a">></span> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting <span style="color:#0077aa">for</span> master to send event
                  Master_Host: 192.168.1.248
                  Master_User: idbrep
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000013
          Read_Master_Log_Pos: 1879
               Relay_Log_File: relay-bin.000002
                Relay_Log_Pos: 537
        Relay_Master_Log_File: mysql-bin.000013
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 1879
              Relay_Log_Space: 829
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 1
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
                   Using_Gtid: No
                  Gtid_IO_Pos: 
      Replicate_Do_Domain_Ids: 
  Replicate_Ignore_Domain_Ids: 
                Parallel_Mode: conservative
1 row <span style="color:#0077aa">in</span> <span style="color:#0077aa">set</span> <span style="color:#999999">(</span>0.00 sec<span style="color:#999999">)</span>
MariaDB <span style="color:#999999">[</span><span style="color:#999999">(</span>none<span style="color:#999999">)</span><span style="color:#999999">]</span><span style="color:#9a6e3a">></span></code></span></span>

 

Column Store測試

Part1:主鍵和索引

<span style="color:#333333"><span style="color:black"><code class="language-bash">MariaDB <span style="color:#999999">[</span>helei<span style="color:#999999">]</span><span style="color:#9a6e3a">></span> create table helei_innodb<span style="color:#999999">(</span>
    -<span style="color:#9a6e3a">></span> <span style="color:#dd4a68">id</span> int<span style="color:#999999">(</span>10<span style="color:#999999">)</span> unsigned NOT NULL AUTO_INCREMENT,
    -<span style="color:#9a6e3a">></span> c1 int<span style="color:#999999">(</span>10<span style="color:#999999">)</span> NOT NULL DEFAULT <span style="color:#669900">'0'</span>,
    -<span style="color:#9a6e3a">></span> c2 int<span style="color:#999999">(</span>10<span style="color:#999999">)</span> unsigned DEFAULT NULL,
    -<span style="color:#9a6e3a">></span> c5 int<span style="color:#999999">(</span>10<span style="color:#999999">)</span> unsigned NOT NULL DEFAULT <span style="color:#669900">'0'</span>,
    -<span style="color:#9a6e3a">></span> c3 timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    -<span style="color:#9a6e3a">></span> c4 varchar<span style="color:#999999">(</span>200<span style="color:#999999">)</span> NOT NULL DEFAULT <span style="color:#669900">''</span>,
    -<span style="color:#9a6e3a">></span> PRIMARY KEY<span style="color:#999999">(</span>id<span style="color:#999999">)</span>,
    -<span style="color:#9a6e3a">></span> KEY idx_c1<span style="color:#999999">(</span>c1<span style="color:#999999">)</span>,
    -<span style="color:#9a6e3a">></span> KEY idx_c2<span style="color:#999999">(</span>c2<span style="color:#999999">)</span>
    -<span style="color:#9a6e3a">></span> <span style="color:#999999">)</span>ENGINE<span style="color:#9a6e3a">=</span>InnoDB <span style="color:#999999">;</span>
Query OK, 0 rows affected <span style="color:#999999">(</span>0.03 sec<span style="color:#999999">)</span>
MariaDB <span style="color:#999999">[</span>helei<span style="color:#999999">]</span><span style="color:#9a6e3a">></span> create table helei_cs<span style="color:#999999">(</span>
    -<span style="color:#9a6e3a">></span> <span style="color:#dd4a68">id</span> int<span style="color:#999999">(</span>10<span style="color:#999999">)</span> unsigned NOT NULL AUTO_INCREMENT,
    -<span style="color:#9a6e3a">></span> c1 int<span style="color:#999999">(</span>10<span style="color:#999999">)</span> NOT NULL DEFAULT <span style="color:#669900">'0'</span>,
    -<span style="color:#9a6e3a">></span> c2 int<span style="color:#999999">(</span>10<span style="color:#999999">)</span> unsigned DEFAULT NULL,
    -<span style="color:#9a6e3a">></span> c5 int<span style="color:#999999">(</span>10<span style="color:#999999">)</span> unsigned NOT NULL DEFAULT <span style="color:#669900">'0'</span>,
    -<span style="color:#9a6e3a">></span> c3 timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    -<span style="color:#9a6e3a">></span> c4 varchar<span style="color:#999999">(</span>200<span style="color:#999999">)</span> NOT NULL DEFAULT <span style="color:#669900">''</span>,
    -<span style="color:#9a6e3a">></span> PRIMARY KEY<span style="color:#999999">(</span>id<span style="color:#999999">)</span>,
    -<span style="color:#9a6e3a">></span> KEY idx_c1<span style="color:#999999">(</span>c1<span style="color:#999999">)</span>,
    -<span style="color:#9a6e3a">></span> KEY idx_c2<span style="color:#999999">(</span>c2<span style="color:#999999">)</span>
    -<span style="color:#9a6e3a">></span> <span style="color:#999999">)</span>ENGINE<span style="color:#9a6e3a">=</span>Columnstore<span style="color:#999999">;</span>
ERROR 1069 <span style="color:#999999">(</span>42000<span style="color:#999999">)</span>: Too many keys specified<span style="color:#999999">;</span> max 0 keys allowed</code></span></span>

這裏可以看出,columnstore存儲引擎不支持主鍵也不需要索引

<span style="color:#333333"><span style="color:black"><code class="language-bash">MariaDB <span style="color:#999999">[</span>helei<span style="color:#999999">]</span><span style="color:#9a6e3a">></span> create table helei_cs<span style="color:#999999">(</span>
    -<span style="color:#9a6e3a">></span> <span style="color:#dd4a68">id</span> int<span style="color:#999999">(</span>10<span style="color:#999999">)</span> unsigned NOT NULL ,
    -<span style="color:#9a6e3a">></span> c1 varchar<span style="color:#999999">(</span>200<span style="color:#999999">)</span> NOT NULL DEFAULT <span style="color:#669900">''</span>
    -<span style="color:#9a6e3a">></span> <span style="color:#999999">)</span>ENGINE<span style="color:#9a6e3a">=</span>Columnstore<span style="color:#999999">;</span>
Query OK, 0 rows affected <span style="color:#999999">(</span>0.34 sec<span style="color:#999999">)</span>
MariaDB <span style="color:#999999">[</span>helei<span style="color:#999999">]</span><span style="color:#9a6e3a">></span> insert into helei_cs values<span style="color:#999999">(</span>1,<span style="color:#669900">'1'</span><span style="color:#999999">)</span><span style="color:#999999">;</span>
Query OK, 1 row affected <span style="color:#999999">(</span>0.60 sec<span style="color:#999999">)</span>
MariaDB <span style="color:#999999">[</span>helei<span style="color:#999999">]</span><span style="color:#9a6e3a">></span> insert into helei_cs values<span style="color:#999999">(</span>2,<span style="color:#669900">'2'</span><span style="color:#999999">)</span><span style="color:#999999">;</span>
Query OK, 1 row affected <span style="color:#999999">(</span>0.08 sec<span style="color:#999999">)</span>
MariaDB <span style="color:#999999">[</span>helei<span style="color:#999999">]</span><span style="color:#9a6e3a">></span> insert into helei_cs values<span style="color:#999999">(</span>3,<span style="color:#669900">'3'</span><span style="color:#999999">)</span><span style="color:#999999">;</span>
Query OK, 1 row affected <span style="color:#999999">(</span>0.17 sec<span style="color:#999999">)</span></code></span></span>

這裏可以看出columnstore的插入速度較慢

 

Warning:警

columnstore不支持主鍵、索引、timestamp、collate用法、char\varchar類型的sum/average用法。

 

 

Part2:效率測試

1G內存虛擬機MariaDB ColumnStore 2.82s,線上生產庫8G的innodb_buffer_pool_size該條慢查詢耗時17.894s。

 

wKiom1hhzZPhbgsfAAHGfuXLaEI623.jpg

wKiom1hhzZPg9wC1AAA_XFox6c4203.jpg


 

 

BUG發現

如果pm2的機器掛掉了,按照原本的想法,應該pm1可以繼續工作,但無法進行查詢,會報錯:

ERROR 1815 (HY000): Internal error: st: 10000 TupleBPS::sendPrimitiveMessages() caught an exception: IDB-2034: At least one DBRoot required for that query is offline. 

 

這應該是一個BUG,因爲pm是負責拉取數據到內存中進行計算的,數據本文中的實驗應該都存放在um機器下。我們來查一下現在表中的數據:

<span style="color:#333333"><span style="color:black"><code class="language-bash">MariaDB <span style="color:#999999">[</span>erp_test<span style="color:#999999">]</span><span style="color:#9a6e3a">></span> show tables<span style="color:#999999">;</span>
+--------------------+
<span style="color:#9a6e3a">|</span> Tables_in_erp_test <span style="color:#9a6e3a">|</span>
+--------------------+
<span style="color:#9a6e3a">|</span> erp_bjlikp         <span style="color:#9a6e3a">|</span>
<span style="color:#9a6e3a">|</span> erp_bjlips         <span style="color:#9a6e3a">|</span>
<span style="color:#9a6e3a">|</span> erp_likp           <span style="color:#9a6e3a">|</span>
<span style="color:#9a6e3a">|</span> erp_lips           <span style="color:#9a6e3a">|</span>
<span style="color:#9a6e3a">|</span> erp_mara           <span style="color:#9a6e3a">|</span>
+--------------------+
5 rows <span style="color:#0077aa">in</span> <span style="color:#0077aa">set</span> <span style="color:#999999">(</span>0.00 sec<span style="color:#999999">)</span>
MariaDB <span style="color:#999999">[</span>erp_test<span style="color:#999999">]</span><span style="color:#9a6e3a">></span> <span style="color:#0077aa">select</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> from erp_bjlikp<span style="color:#999999">;</span>
+----------+
<span style="color:#9a6e3a">|</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> <span style="color:#9a6e3a">|</span>
+----------+
<span style="color:#9a6e3a">|</span>        0 <span style="color:#9a6e3a">|</span>
+----------+
1 row <span style="color:#0077aa">in</span> <span style="color:#0077aa">set</span> <span style="color:#999999">(</span>1.15 sec<span style="color:#999999">)</span>
MariaDB <span style="color:#999999">[</span>erp_test<span style="color:#999999">]</span><span style="color:#9a6e3a">></span> <span style="color:#0077aa">select</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> from erp_bjlips<span style="color:#999999">;</span>
+----------+
<span style="color:#9a6e3a">|</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> <span style="color:#9a6e3a">|</span>
+----------+
<span style="color:#9a6e3a">|</span>        0 <span style="color:#9a6e3a">|</span>
+----------+
1 row <span style="color:#0077aa">in</span> <span style="color:#0077aa">set</span> <span style="color:#999999">(</span>1.15 sec<span style="color:#999999">)</span>
MariaDB <span style="color:#999999">[</span>erp_test<span style="color:#999999">]</span><span style="color:#9a6e3a">></span> <span style="color:#0077aa">select</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> from erp_lips<span style="color:#999999">;</span>
+----------+
<span style="color:#9a6e3a">|</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> <span style="color:#9a6e3a">|</span>
+----------+
<span style="color:#9a6e3a">|</span>        0 <span style="color:#9a6e3a">|</span>
+----------+
1 row <span style="color:#0077aa">in</span> <span style="color:#0077aa">set</span> <span style="color:#999999">(</span>1.14 sec<span style="color:#999999">)</span>
MariaDB <span style="color:#999999">[</span>erp_test<span style="color:#999999">]</span><span style="color:#9a6e3a">></span> <span style="color:#0077aa">select</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> from erp_mara<span style="color:#999999">;</span>
+----------+
<span style="color:#9a6e3a">|</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> <span style="color:#9a6e3a">|</span>
+----------+
<span style="color:#9a6e3a">|</span>        0 <span style="color:#9a6e3a">|</span>
+----------+
1 row <span style="color:#0077aa">in</span> <span style="color:#0077aa">set</span> <span style="color:#999999">(</span>1.15 sec<span style="color:#999999">)</span></code></span></span>

會發現所有的錶行數都變爲0了

<span style="color:#333333"><span style="color:black"><code class="language-bash">ProcessMonitor      pm2       AUTO_OFFLINE      Tue Dec 27 22:23:50 2016
ProcessManager      pm2       AUTO_OFFLINE      Tue Dec 27 22:23:50 2016
DBRMControllerNode  pm2       AUTO_OFFLINE      Tue Dec 27 22:23:50 2016
ServerMonitor       pm2       AUTO_OFFLINE      Tue Dec 27 22:23:50 2016
DBRMWorkerNode      pm2       AUTO_OFFLINE      Tue Dec 27 22:23:50 2016
DecomSvr            pm2       AUTO_OFFLINE      Tue Dec 27 22:23:50 2016
PrimProc            pm2       AUTO_OFFLINE      Tue Dec 27 22:23:50 2016
WriteEngineServer   pm2       AUTO_OFFLINE      Tue Dec 27 22:23:50 2016</code></span></span>
<span style="color:#333333"><span style="color:black"><code class="language-bash">MariaDB <span style="color:#999999">[</span>erp_test<span style="color:#999999">]</span><span style="color:#9a6e3a">></span> <span style="color:#0077aa">select</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> from erp_lips<span style="color:#999999">;</span>
+----------+
<span style="color:#9a6e3a">|</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> <span style="color:#9a6e3a">|</span>
+----------+
<span style="color:#9a6e3a">|</span>  3147299 <span style="color:#9a6e3a">|</span>
+----------+
1 row <span style="color:#0077aa">in</span> <span style="color:#0077aa">set</span> <span style="color:#999999">(</span>0.37 sec<span style="color:#999999">)</span>
MariaDB <span style="color:#999999">[</span>erp_test<span style="color:#999999">]</span><span style="color:#9a6e3a">></span> <span style="color:#0077aa">select</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> from erp_mara<span style="color:#999999">;</span>
+----------+
<span style="color:#9a6e3a">|</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> <span style="color:#9a6e3a">|</span>
+----------+
<span style="color:#9a6e3a">|</span>     4361 <span style="color:#9a6e3a">|</span>
+----------+
1 row <span style="color:#0077aa">in</span> <span style="color:#0077aa">set</span> <span style="color:#999999">(</span>0.08 sec<span style="color:#999999">)</span>
MariaDB <span style="color:#999999">[</span>erp_test<span style="color:#999999">]</span><span style="color:#9a6e3a">></span> <span style="color:#0077aa">select</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> from erp_bjlips<span style="color:#999999">;</span>
+----------+
<span style="color:#9a6e3a">|</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> <span style="color:#9a6e3a">|</span>
+----------+
<span style="color:#9a6e3a">|</span>  2762244 <span style="color:#9a6e3a">|</span>
+----------+
1 row <span style="color:#0077aa">in</span> <span style="color:#0077aa">set</span> <span style="color:#999999">(</span>0.13 sec<span style="color:#999999">)</span>
MariaDB <span style="color:#999999">[</span>erp_test<span style="color:#999999">]</span><span style="color:#9a6e3a">></span> <span style="color:#0077aa">select</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> from erp_bjlikp<span style="color:#999999">;</span>
+----------+
<span style="color:#9a6e3a">|</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> <span style="color:#9a6e3a">|</span>
+----------+
<span style="color:#9a6e3a">|</span>    19032 <span style="color:#9a6e3a">|</span>
+----------+
1 row <span style="color:#0077aa">in</span> <span style="color:#0077aa">set</span> <span style="color:#999999">(</span>0.09 sec<span style="color:#999999">)</span>
MariaDB <span style="color:#999999">[</span>erp_test<span style="color:#999999">]</span><span style="color:#9a6e3a">></span> <span style="color:#0077aa">select</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> from erp_likp<span style="color:#999999">;</span>
+----------+
<span style="color:#9a6e3a">|</span> count<span style="color:#999999">(</span>*<span style="color:#999999">)</span> <span style="color:#9a6e3a">|</span>
+----------+
<span style="color:#9a6e3a">|</span>   169002 <span style="color:#9a6e3a">|</span>
+----------+
1 row <span style="color:#0077aa">in</span> <span style="color:#0077aa">set</span> <span style="color:#999999">(</span>0.08 sec<span style="color:#999999">)</span></code></span></span>

pm2機器啓動後,發現又恢復正常

這應該是軟件的一個bug,應該會在1.0.7GA版本修復。

感謝賀春暘老師指點,本人在測試環境中得以驗證該BUG

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章