skywalking監控tomcat agent大量報錯日誌分析

skywalking功能就不多說了,說多了我也不知道.... 

解決問題方案 1:將spring-web的版本升到4.2.X 因爲path()的方法是4.2之後纔有的 單獨升spring-web版本不行,spring的都要升

2:保證controller的方法上

@RequestMapping(method = RequestMethod.POST, value = "") value是有定義的,沒有定義value會報錯

兩個選一個就可以解決

下面分析解決方法,原理還沒時間看 先解決問題.....

在tomcat中加入agent路徑後,agent/logs下面skywalking-api.log 多了很多的錯誤日誌,兩天就把磁盤撐爆了看日誌如下

DEBUG 2019-12-18 19:31:31:610 pool-1109527-thread-1 TraceSegmentServiceClient : One trace segment has been abandoned, cause by buffer is full. 
DEBUG 2019-12-18 19:31:31:610 pool-1109527-thread-1 TraceSegmentServiceClient : One trace segment has been abandoned, cause by buffer is full. 
DEBUG 2019-12-18 19:31:31:610 threadPoolTaskExecutor-87 TraceSegmentServiceClient : One trace segment has been abandoned, cause by buffer is full. 
ERROR 2019-12-18 19:31:31:610 http-nio-8088-exec-396 InstMethodsInter : class[class com.xxxx.xxxx.controller.xxxxController] after method[xxxx] intercept failure 
org.apache.skywalking.apm.plugin.spring.mvc.commons.exception.IllegalMethodStackDepthException: Please submit an new issue to Apache Skywalking if this Exception was happened.
        at org.apache.skywalking.apm.plugin.spring.mvc.commons.interceptor.AbstractMethodInterceptor.afterMethod(AbstractMethodInterceptor.java:150)
        at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:105)
        at com.xxxx.xxxx.controller.xxxxController.xxxx(xxxxController.java)
        at sun.reflect.GeneratedMethodAccessor82.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest$original$N6C0m52i(InvocableHandlerMethod.java:137)
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest$original$N6C0m52i$accessor$gsVj7jwA(InvocableHandlerMethod.java)
        at org.springframework.web.method.support.InvocableHandlerMethod$auxiliary$K6EC4n7Y.call(Unknown Source)
        at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:93)
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java)
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:777)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:706)
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:943)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:877)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966)
        at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:868)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504)
        at org.apache.catalina.core.StandardHostValve.invoke$original$d3OBIOOK(StandardHostValve.java:170)
        at org.apache.catalina.core.StandardHostValve.invoke$original$d3OBIOOK$accessor$XcPehfpS(StandardHostValve.java)
    
 
DEBUG 2019-12-18 19:31:31:609 pool-1109527-thread-1 TraceSegmentServiceClient : One trace segment has been abandoned, cause by buffer is full. 
ERROR 2019-12-18 19:31:31:609 http-nio-8088-exec-396 InstMethodsInter : class[class com.xxxx.xxxx.controller.xxxController] before method[xxxx] intercept failure 
java.lang.NoSuchMethodError: org.springframework.web.bind.annotation.RequestMapping.path()[Ljava/lang/String;
        at org.apache.skywalking.apm.plugin.spring.mvc.commons.interceptor.RequestMappingMethodInterceptor.getRequestURL(RequestMappingMethodInterceptor.java:38)
        at org.apache.skywalking.apm.plugin.spring.mvc.commons.interceptor.AbstractMethodInterceptor.beforeMethod(AbstractMethodInterceptor.java:82)
        at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:82)
        at com.xxxx.xxxx.controller.xxxController.xxx(xxxController.java)
        at sun.reflect.GeneratedMethodAccessor82.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest$original$N6C0m52i(InvocableHandlerMethod.java:137)
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest$original$N6C0m52i$accessor$gsVj7jwA(InvocableHandlerMethod.java)
        at org.springframework.web.method.support.InvocableHandlerMethod$auxiliary$K6EC4n7Y.call(Unknown Source)
        at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter.intercept(InstMethodsInter.java:93)
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java)
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:777)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:706)
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:943)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:877)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966)
        at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:868)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)

重點是

查看RequestMappingMethodInterceptor源碼

報錯位置在這裏 一個value() 一個path()

解決問題方案 1:將spring-web的版本升到4.2.X 因爲path()的方法是4.2之後纔有的 單獨升spring-web版本不行,spring的都要升

2:保證controller的方法上

@RequestMapping(method = RequestMethod.POST, value = "") value是有定義的,沒有定義value會報錯

線上問題已解決,使用的方案2   value = ""

如果有誤歡迎指出

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章