自動化autosql審覈平臺--基於Yearning可視化
解決依賴包yum -y install cmake ncurse ncurses-devel gcc gcc-c++ openssl-devel http-parser libicu nginx wget m4
一、升級PYTHON版本
1.下載並安裝Python3.6.3
# wget https://www.python.org/ftp/python/3.6.3/Python-3.6.3.tgz
# tar -zxvf Python-3.6.3.tgz
# cd Python- 3.6.3/
# ./configure
# make
# make install
2.將原本python重命名，用python3.6取代python2.7
# mv /usr/bin/python /usr/bin/python.bak
# ln -s /usr/local/bin/python3 /usr/bin/python
3.更改配置，否則原本yum的功能會無法使用
# vim /usr/bin/yum
# vim /usr/libexec/urlgrabber-ext-down
備註：被將#!/usr/bin/python改爲#! /usr/bin/python2.7，保存退出即可。
4.安裝常用套件
pip install numpy scipy matplotlib scikit-learn ##這個過程 時間會比較9
##若提示pip找不到相應的版本，在/usr/local/lib/python3.6/site-packages 確定安裝PIP版本，默認爲:/bin/pip 刪除或者mv,手動安裝pip版本路徑爲:/usr/local/bin/pip
版本驗證：
# pip -V
pip 9.0.1 from /usr/local/lib/python3.6/site-packages/pip-9.0.1-py3.6.egg (python 3.6)
##以及後面升級PIP版本都不會有問題，在pip install 時候先下載pip版本進行安裝
二、安裝bison-2.5.1版本，不要使用YUM 安裝
解壓包
#.configure
# make && make install
驗證是否安裝成功
# bison -V
bison (GNU Bison) 2.5.1
Written by Robert Corbett and Richard Stallman.
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
三、安裝INCEPTION
# unzip inception-master.zip
# cd inception-master
# cmake .
#cmake -DWITH_DEBUG=OFF -DCMAKE_INSTALL_PREFIX=/usr/local/inception -DMYSQL_DATADIR=/home/mysql/inception -DWITH_SSL=yes -DCMAKE_BUILD_TYPE=RELEASE-DWITH_ZLIB=bundled-DMY_MAINTAINER_CXX_WARNINGS="-Wall -Wextra -Wunused -Wwrite-strings -Wno-strict-aliasing -Wno-unused-parameter -Woverloaded-virtual" -DMY_MAINTAINER_C_WARNINGS="-Wall -Wextra -Wunused -Wwrite-strings -Wno-strict-aliasing -Wdeclaration-after-statement"
# make && make install
# sh -x inception_build.sh debug
報錯如下：
-- Build files have been written to: /root/inception-master
+ '[' x '!=' Xcode ']'
+ make install
make: *** 沒有規則可以創建目標“install”。 停止。
# sh inception_build.sh debug [linux]
make: *** 沒有規則可以創建目標“install”。 停止。
解決辦法：# make -j6 && make install
再次執行sh inception_build.sh debug [linux] 一樣問題。有人說是BISON問題
##這裏需要注意 只要按上面的操作 執行make && make install 執行不報錯即可，可跳過sh inception_build.sh debug [linux] 操作，需要驗證在TCP/IP是否能夠連接INCEPTIONS，若無法連接會導致和Yearning無法提交審覈，提示無權限
添加配置文件
#touch /etc/inc.cnf
[inception]
general_log=1
general_log_file=/home/mysql/inception/inc.log
port=6669
socket=/tmp/inc.socket
character-set-client-handshake=0
character-set-server=utf8mb4
inception_remote_system_password=backup
inception_remote_system_user=backup
inception_remote_backup_port=3306
inception_remote_backup_host=192.168.22.130
inception_support_charset=utf8,utf8mb4,latin1
inception_enable_nullable=0
inception_check_primary_key=1
inception_check_column_comment=1
inception_check_table_comment=1
inception_osc_min_table_size=1
inception_osc_bin_dir=/usr/bin
inception_osc_chunk_time=0.1
inception_ddl_support=1
inception_enable_blob_type=1
inception_check_column_default_value=1
啓動：
/usr/local/inception/bin/Inception --defaults-file=/etc/inc.cnf &
登陸
mysql -uroot -P6669 -h127.0.0.1
變量查看
inception get variables;
驗證INCEPTION，可以看日誌
#coding=utf-8
import MySQLdb#def table_structure(mysql_structure):
sql='/*--user=inc;--password=root123;--host=172.16.32.11;--execute=1;--port=3307;*/\
inception_magic_start;\
use ms;\
CREATE TABLE `alifeba_user` (\
`ID` int(11) unsigned NOT NULL auto_increment comment"aaa",\
`username` varchar(50) NOT NULL Default "" comment"aaa",\
`realName` varchar(50) NOT NULL Default "" comment"aaa",\
`age` int(11) NOT NULL Default 0 comment"aaa",\
PRIMARY KEY (`ID`)\
) ENGINE=INNODB DEFAULT CHARSET=utf8mb4 COMMENT="AAAA";\
inception_magic_commit;'
#sql = sql1 + mysql_structure + sql2
try:
conn=MySQLdb.connect(host='172.16.32.152',user='inc',passwd='root123',port=6669, charset="utf8")
cur=conn.cursor()
print sql,cur
ret=cur.execute(sql)
result=cur.fetchall()
num_fields = len(cur.description)
field_names = [i[0] for i in cur.description]
print field_names
for row in result:
print row[0], "|",row[1],"|",row[2],"|",row[3],"|",row[4],"|",row[5],"|",row[6],"|",row[7],"|",row[8],"|",row[9],"|",row[10]
cur.close()
conn.close()
except MySQLdb.Error,e:
print "Mysql Error %d: %s" % (e.args[0], e.args[1])
#return result[1][4].split("\n")
#if __main __ == "__name__":
# table_structure()
安裝Yearning
tar -zxvf Yearning.tar.gz
mv Yearning yearning
cd yearning/src/
cp deploy.conf.template deploy.conf
參數設置
[mysql]
db = Yearning ##需要在指定DB裏添加
address = 192.168.22.130
port = 3306
username = yearning
password = yearning
[host]
ipaddress = 192.168.22.130:80
[Inception]
ip = 192.168.22.130
port = 6669
user = admin
password = admin
backupdb = 192.168.22.130
backupport = 3306
backupuser = backup
backuppassword = backup
[LDAP]
LDAP_SERVER =
LDAP_SCBASE =
LDAP_DOMAIN =
LDAP_TYPE = 1
[email]
username =
password =
smtp_server =
smtp_port = 25
[sql]
limit = 1000
[webhook]
dingding =
####參數介紹
[mysql]
db = 所創建的庫名
address = 數據庫地址
port = 數據庫端口
password = 數據庫密碼
username = 數據庫用戶
[host]
ipaddress = 服務器ip地址:端口 (涉及跨域十分重要!!設置不正確將無法登陸!!)
如 本機地址爲192.168.1.2 nginx設置端口爲80
則應填寫爲 192.168.1.2:80 之後通過該地址訪問平臺。
[Inception]
ip = Inception地址
port = Inception端口
user = Inception用戶名
password = Inception密碼
backupdb = 備份數據庫地址
backupport = 備份數據庫端口
backupuser = 備份數據庫用戶名
backuppassword = 備份數據庫密碼
[LDAP] LDAP相關設置
LDAP_SERVER = LDAP服務地址
LDAP_SCBASE = LDAP dc 設置 如 dc=xxx,dc=com
LDAP_DOMAIN = LDAP域名 如 xxx.com
LDAP_TYPE = 1 1 通過域名進行ldap認證 0 通過uid進行ldap認證
[email] 郵箱推送相關設置
username = 郵箱發件賬號 如 [email protected]
password = 郵箱發件賬號密碼
smtp_server = 郵箱stmp地址, 具體地址請諮詢對應郵箱提供者
#執行
pip install -r requirements.txt
##若提示django版本問題，可以requirements.txt包的版本進行修改
#複製文件connections.py、cursors.py 到PYTHON3.6目錄下替換文件##包目錄下
cp /root/yearning/install/connections.py /usr/local/lib/python3.6/site-packages/pymysql/
cp /root/yearning/install/cursors.py /usr/local/lib/python3.6/site-packages/pymysql/
cd /root/yearning/src
初始化數據信息:
python3 manage.py makemigrations core && python3 manage.py migrate core
添加管理員用戶
echo "from core.models import Account;Account.objects.create_user(username='admin', password='admin8866', group='admin',is_staff=1)" | python3 manage.py shell
初始化權限
echo "from core.models import grained;grained.objects.get_or_create(username='admin', permissions={'person': [], 'ddl': '1', 'ddlcon': [], 'dml': '1', 'dmlcon': [], 'dic': '1', 'diccon': [], 'dicedit': '0', 'query': '1', 'querycon': [], 'user': '1', 'base': '1', 'dicexport': '0'})" | python3 manage.py shell
拷貝WEB文件到nginx的html目錄下,若用yum安裝在/usr/share/nginx/html
cp -rf /root/yearning/webpage/dist/* /usr/share/nginx/html/
#啓動：
python3 manage.py runserver 0.0.0.0:8000 & ##這個服務需要後臺一直運行 可以使用nohup
訪問:http://ip
在管理中，添加DB需要對用戶授權，指定用戶具備哪些權限。
