版本:filebeat5.2.2
日誌如下:
192.168.1.187 [10.10.2.5,192.168.2.2,192.168.3.4] [2017-03-05 00:01:00] ["POST www.test.com/Service.asmx"] [""1.0" encoding="utf-8"?><soap12:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap12="http://www.w3.org/2003/05/soap-envelope"><soap12:Body><SaveException xmlns="http://tempuri.org/"><Content>java.lang.UnsatisfiedLinkError: Native method not found: com.igexin.push.extension.mod.SecurityUtils.m:([B[B)[B at com.igexin.push.extension.mod.SecurityUtils.m(Native Method) at com.igexin.push.util.EncryptUtils.altAesDecSocket(Unknown Source) at com.igexin.push.d.a.c.a(Unknown Source) at com.igexin.push.d.a.c.b(Unknown Source) at com.igexin.push.d.a.c.a(Unknown Source) at com.igexin.push.d.a.c.b(Unknown Source) at com.igexin.push.d.a.c.c(Unknown Source) at com.igexin.a.a.b.a.a.j.a_(Unknown Source) at com.igexin.a.a.d.g.a(Unknown Source) at com.igexin.a.a.d.g.run(Unknown Source) at java.lang.Thread.run(Thread.java:841) </Content><Mobile>561128</Mobile><Source>android</Source></SaveException></soap12:Body></soap12:Envelope>"] ["Dalvik/1.6.0 (Linux; U; Android 4.4.4; SM-G8508S Build/KTU84P)"] ["java\.lang"]
如果不作任何處理,filebeat會把每行日誌作爲一條數據傳輸給redis,這樣elasticsearch收集到數據就無法把日誌分類了。所有必須讓filebeat把上面的這條日誌當成一條數據傳輸給redis。
vim filebeat.yml
#=========================== Filebeat prospectors ===== filebeat.prospectors: #------------------------------ Log prospector -------- - input_type: log paths: - c:\programdata\elasticsearch\logs\* multiline.pattern: ^\d+\.\d+ #日誌開頭以“數字.數字”的格式開頭,不是以此開頭的日誌計算上一行日誌 multiline.negate: true multiline.match: after #------------------------------- Redis output -------- output.redis: enabled: true hosts: ["192.168.10.10:6379"] port: 6379 key: log db: 2 datatype: list worker: 2 loadbalance: true logging.to_files: true logging.files: