SSM+MySQL已是Java開發的主流架構。在SSM+MySQL架構中,SQL一般寫在mapper映射文件中,對於mapper文件的審計一直是個頭疼的問題,測試人員很難覆蓋mapper文件中動態SQL的所有情況,開發人員更是無法羅列每一個select id對應的所有SQL。在和開發同事溝通後,覺得自己可以寫一個小工具來解決這一痛點!筆者作爲一名DBA,自學了golang,經過七個月的努力初步完成了myaudit這一款實用性極強的SQLAudit小工具的開發,歡迎使用和吐槽,項目地址:https://github.com/gaozhongzheng/myaudit 。
下面簡單介紹下myaudit這個小工具:
myaudit是一個審計SSM + MySQL開發架構中SQL映射文件(xml文件)的SQLAudit工具。
設計思路如下:
- 1.格式化xml文件
- 2.重寫xml文件
- 3.構造Mybatis工程
- 4.調起Mybatis工程打印所有可能的SQL到日誌
- 5.解析日誌獲取SQL,並根據動態字段類型將SQL中的'?'替換爲具體值
-
6.調用SOAR獲取審計報告
爲了上首頁,上一段代碼:func ReplaceQuestionMark(query string, db *sql.DB, dbName string) string { stmt, err := sqlparser.Parse(query) if err != nil { common.LogIfError(err, "parse failed: "+query) return query } columns, tables, _, err := GetInfo(stmt) if err != nil { common.LogIfError(err, "get info failed: "+query) return query } var column_name string var data_type string column_type := make(map[string]string) for _, tab := range tables { rows, _ := db.Query("select column_name, data_type from columns where table_name = ? and TABLE_SCHEMA = '"+dbName+"'", tab) for rows.Next() { if err := rows.Scan(&column_name, &data_type); err == nil { if _, ok := column_type[column_name]; !ok { column_type[column_name] = data_type } } else { fmt.Println(err) } } rows.Close() } i := 0 for _, col := range columns { if column, ok := column_type[col[0]]; ok { i++ delete(column_type, col[0]) // fmt.Println("here is", query) question_index := GetIndex(query, "?") if question_index != nil { // fmt.Println("this is :", question_index) idx, _ := strconv.Atoi(col[1]) // fmt.Println("idx is :", idx) index := question_index[idx-i] // fmt.Println("index is :", index) if strings.Contains(column, "char") || strings.Contains(column, "text") || strings.Contains(column, "lob") { query = query[:index] + "'SQLAudit'" + query[index+1:] } else if strings.Contains(column, "int") { query = query[:index] + "1" + query[index+1:] } else if strings.Contains(column, "decimal") || strings.Contains(column, "float") || strings.Contains(column, "double") || strings.Contains(column, "numeric") { query = query[:index] + "1.1" + query[index+1:] } else if strings.Contains(column, "timestamp") || strings.Contains(column, "datatime") { query = query[:index] + "'2019-01-01 00:00:00'" + query[index+1:] } else if strings.Contains(column, "date") { query = query[:index] + "'2019-01-01'" + query[index+1:] } else if strings.Contains(column, "time") { query = query[:index] + "'00:00:00'" + query[index+1:] } else if strings.Contains(column, "year") { query = query[:index] + "'2019'" + query[index+1:] } else if strings.Contains(column, "enum") { query = query[:index] + "'SQLAudit'" + query[index+1:] } else { query = query[:index] + "none" + query[index+1:] } } } } stmt, err = sqlparser.Parse(query) if err != nil { common.LogIfError(err, "parse failed: "+query) return query } _, _, limits, err := GetInfo(stmt) if err != nil { common.LogIfError(err, "get info failed: "+query) return query } // 替換limit中的? for _, ques := range limits { question_index := GetIndex(query, "?") if ques[0] != -1 { index := question_index[ques[0]-1] query = query[:index] + "1" + query[index+1:] } if ques[1] != -1 { index := question_index[ques[1]-1] query = query[:index] + "1" + query[index+1:] } } // TODO group by和order by中的?替換。 // 放棄,因爲group by和order by的對象是列而不是值,而列我們是不知道的,所以沒法替換 // 打印信息 // fmt.Println("columns:", columns) // fmt.Println("tables:", tables) // fmt.Println("limits:", limits) // fmt.Println("new query:", query) return query }
特點:
- 1.與業務代碼深度解耦
- 2.解析並審計動態SQL所有可能的情況
- 3.根據字段類型替換綁定變量值,解決了SQL指紋無法在MySQL中獲取執行計劃的問題
安裝/部署(Linux)
方式一
- step1.克隆源代碼至本地
- step2.編譯命令文件:cd cmd;go build myaudit.go
- step3.執行createDir.sh創建相應目錄,目錄含義見目錄說明。
- step4.上傳log4j-1.2.17.jar、mybatis-3.2.8.jar、mysql-connector-java-5.1.47.jar至/usr/local/myaudit/lib,可從此處獲取jar包
- step5.將step2中編譯好的命令文件拷貝至/usr/local/myaudit/bin並將此目錄添加至環境變量
注:需先安裝SOAR
方式二(免編譯快速部署)
- step1.克隆deployment至本地
- step2.將/usr/local/myaudit/bin添加至環境變量
使用myaudit(如有疑問或建議,請郵件至[email protected])
Usage of myaudit: -ftype mapper|slowlog|normal, default mapper --待審計文件種類,默認爲mapper(暫時只支持Mybatis項目中的mapper文件) -fname file waitting for audit --待審計文件名,不需加擴展名,如MapperSakila.xml傳參時爲MapperSakila -conn $IP:$PORT/$DB, like 127.0.0.1:3306/sakila --測試環境連接串,待審計文件對應的測試環境數據庫 -u username --MySQL用戶名 -p password --MySQL用戶密碼 -report-type html|json, default html --報告類型,默認爲html
- step1.上傳待審計的SQL映射文件至/usr/local/myaudit/file
- step2.執行審計(示例):
myaudit -ftype mapper -fname MapperSakila -conn 127.0.0.1:3306/sakila -u root -p 123456 -report-type html Dir '/usr/local/myaudit/tmp/MapperSakila' was created/rebuilt successfully! /usr/local/myaudit/file/MapperSakila.xml read ok! /usr/local/myaudit/tmp/MapperSakila/tmp.xml read ok! /usr/local/myaudit/tmp/MapperSakila/oldMapper.xml read ok! /usr/local/myaudit/tmp/MapperSakila/oldMapperFormated.xml read ok! Compile and execute java files successfully! /usr/local/myaudit/log/mapperLog/MapperSakila.log read ok! Database connect success! Dir '/usr/local/myaudit/sql/MapperSakila' was created/rebuilt successfully! Dir '/usr/local/myaudit/audit/MapperSakila' was created/rebuilt successfully! xml file was successfully audited, you can find the audit file 'getFilmInfoTest1_audit.html' in directory /usr/local/myaudit/audit/MapperSakila. xml file was successfully audited, you can find the audit file 'getFilmInfoTest2_audit.html' in directory /usr/local/myaudit/audit/MapperSakila.
從MapperSakila.xml解析出的SQL文件在此。
審計報告截圖: