GetConnectionTimeoutException: wait millis 3000, active 11, maxActive 200,

異常2 GetConnectionTimeoutException: wait millis 3000, active 11, maxActive 200, creating 1

org.springframework.transaction.CannotCreateTransactionException: Could not open JPA EntityManager for transaction; nested exception is javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection
at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:431)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:373)
at org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:447)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:277)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy $DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673) at cn.taqu.account.service.AccountsPersonalInfoService$$EnhancerBySpringCGLIB$$faeb94bd.mGetSecretConfig(<generated>) at cn.taqu.account.service.UuidInfoQueryService.getInfoFromCache(UuidInfoQueryService.java:328) at cn.taqu.account.service.AccountsInfoService.lambda$ getInfoByUuid $2(AccountsInfoService.java:613) at java.util.ArrayList.forEach(ArrayList.java:1257) at cn.taqu.account.service.AccountsInfoService.getInfoByUuid(AccountsInfoService.java:613) at cn.taqu.account.service.AccountsInfoService$$FastClassBySpringCGLIB$$8e886d4d.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$ CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.transaction.interceptor.TransactionInterceptor $1.proceedWithInvocation(TransactionInterceptor.java:99) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$ DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
at cn.taqu.account.service.AccountsInfoService $EnhancerBySpringCGLIB$ 87d295dd.getInfoByUuid(<generated>)
at cn.taqu.account.controller.api.jrpc.InfoController.getInfoByUuid(InfoController.java:952)
at sun.reflect.GeneratedMethodAccessor163.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at cn.taqu.core.web.filter.CustomRequestFilter.doFilterInternal(CustomRequestFilter.java:35)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:110)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:105)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:106)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at io.micrometer.spring.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:106)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at cn.taqu.core.web.filter.HealthyFilter.doFilterInternal(HealthyFilter.java:34)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol $ConnectionHandler.process(AbstractProtocol.java:861) at org.apache.tomcat.util.net.NioEndpoint$ SocketProcessor.doRun(NioEndpoint.java:1455)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor $Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$ WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1692)
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1602)
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.throwPersistenceException(AbstractEntityManagerImpl.java:1700)
at org.hibernate.jpa.internal.TransactionImpl.begin(TransactionImpl.java:48)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.beginTransaction(HibernateJpaDialect.java:189)
at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:380)
... 99 more
org.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:47)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:109)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:95)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:90)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.getPhysicalConnection(LogicalConnectionManagedImpl.java:112)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.getConnectionForTransactionManagement(LogicalConnectionManagedImpl.java:230)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.begin(LogicalConnectionManagedImpl.java:237)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl $TransactionDriverControlImpl.begin(JdbcResourceLocalTransactionCoordinatorImpl.java:214) at org.hibernate.engine.transaction.internal.TransactionImpl.begin(TransactionImpl.java:52) at org.hibernate.internal.SessionImpl.beginTransaction(SessionImpl.java:1512) at org.hibernate.jpa.internal.TransactionImpl.begin(TransactionImpl.java:45) ... 101 more com.alibaba.druid.pool.GetConnectionTimeoutException: wait millis 3000, active 11, maxActive 200, creating 1 at com.alibaba.druid.pool.DruidDataSource.getConnectionInternal(DruidDataSource.java:1422) at com.alibaba.druid.pool.DruidDataSource.getConnectionDirect(DruidDataSource.java:1241) at com.alibaba.druid.filter.FilterChainImpl.dataSource_connect(FilterChainImpl.java:4619) at com.alibaba.druid.filter.logging.LogFilter.dataSource_getConnection(LogFilter.java:874) at com.alibaba.druid.filter.FilterChainImpl.dataSource_connect(FilterChainImpl.java:4615) at com.alibaba.druid.filter.stat.StatFilter.dataSource_getConnection(StatFilter.java:680) at com.alibaba.druid.filter.FilterChainImpl.dataSource_connect(FilterChainImpl.java:4615) at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:1219) at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:1211) at com.alibaba.druid.pool.DruidDataSource.getConnection(DruidDataSource.java:105) at org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource.getConnection(AbstractRoutingDataSource.java:164) at org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:122) at org.hibernate.internal.AbstractSessionImpl$ NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:386)

問題描述

GetConnectionTimeoutException: wait millis 3000, active 11, maxActive 200, creating 1

異常信息可以定位到 DruidDataSource 1422 行，下面是源碼 https://github.com/alibaba/druid/blob/1.1.5/src/main/java/com/alibaba/druid/pool/DruidDataSource.java
當前線程池的活躍線程數 11 ，最大活躍線程數 200 ， creating 表示正在創建的物理連接數是 1. 併發不大的情況超過3s 依然取不到連接，然後會拋出上面的異常。
druid 取連接流程其實不會創建連接，連接的創建默認在連接池初始化的時候，後期由另外一個線程createConnectionThread 維護。
DruidDataSource，DruidAbstractDataSource 的源碼很多。大家有時間可以自己下去看。大概5千行代碼。

初始化連接池創建流程

沒有開啓 keepalive 以及異步初始化，默認是不會走這個流程的。
通過輪詢的方式創建物理線程直到initialSize
如果創建連接出現了SQLException ,報 init datasource error 。（這個日誌沒有看到的）

  if (!asyncInit) {
                try {
                    // init connections
                    for (int i = 0, size = getInitialSize(); i < size; ++i) {
                        PhysicalConnectionInfo pyConnectInfo = createPhysicalConnection();
                        DruidConnectionHolder holder = new DruidConnectionHolder(this, pyConnectInfo);
                        connections[poolingCount] = holder;
                        incrementPoolingCount();
                    }

                    if (poolingCount > 0) {
                        poolingPeak = poolingCount;
                        poolingPeakTime = System.currentTimeMillis();
                    }
                } catch (SQLException ex) {
                    LOG.error("init datasource error, url: " + this.getUrl(), ex);
                    connectError = ex;
                }
            }

createConnectionThread 維護連接池流程

CreateConnectionThread 在可用連接不爲empty 情況，wait 創建連接線程，
當可用連接數> 等待獲取連接線程數且活躍連接數+可用連接數> idle連接數時 wait創建連接線程
當活躍連接數+可用連接數 > = max連接數，wait 創建連接線程
除了 1，2，3 以外，可以創建連接，在啓用異步初始化線程的時候（默認不啓用）且創建連接數小於初始化線程數時，也可以創建物理線程。

當連接創建失敗以後

創建連接默認失敗2次，失敗等待時間大於0 （默認500 ms）
記錄日誌 LOG.error("create connection SQLException, url: " + jdbcUrl + ", errorCode " + e.getErrorCode() ，這個日誌是關鍵。
當創建連接失敗以後會有默認500ms 的sleep 時間。500ms 之內獲取連接，都會拋出我們看到的GetConnectionTimeoutException異常。（默認還有些流程不走）
超時後會繼續輪詢創建連接。

通過日誌create connection SQLException, url 以及j2-5b6ff7bb95-9ckwk 查找
總共有7個異常。
異常1 1個 java.net.UnknownHostException: rds.aliyuncs.com: Name or service not known （dns解析失敗加入 unknown_array 失敗cache 報出異常）
異常2 6個 java.net.UnknownHostException: rds.aliyuncs.com （直接走unknown_array 失敗cache ，報出異常）

下面是CreateConnectionThread創建連接的源碼

public class CreateConnectionThread extends Thread {

        public CreateConnectionThread(String name){
            super(name);
            this.setDaemon(true);
        }

        public void run() {
            initedLatch.countDown();
            // 上次輪訓廢棄連接數
            long lastDiscardCount = 0;
            // 連接創建錯誤數
            int errorCount = 0;
            for (;;) {
                // addLast
                try {
                    lock.lockInterruptibly();
                } catch (InterruptedException e2) {
                    break;
                }
                
                long discardCount = DruidDataSource.this.discardCount;
                //  上次改變廢棄連接數
                boolean discardChanged = discardCount - lastDiscardCount > 0;
                lastDiscardCount = discardCount;

                try {
                    boolean emptyWait = true;

                    if (createError != null
                            && poolingCount == 0
                            && !discardChanged) {
                        emptyWait = false;
                    }
                    
                    if (emptyWait
                            && asyncInit && createCount.get() < initialSize) {
                        emptyWait = false;
                    }
                     // 如果空等待爲true
                    if (emptyWait) {
                        // 必須存在線程等待，才創建連接
                        if (poolingCount >= notEmptyWaitThreadCount //
                                && !(keepAlive && activeCount + poolingCount < minIdle)) {
                            empty.await();
                        }

                        // 防止創建超過maxActive數量的連接,poolingCount 可用連接數
                        if (activeCount + poolingCount >= maxActive) {
                            empty.await();
                            continue;
                        }
                    }

                } catch (InterruptedException e) {
                    lastCreateError = e;
                    lastErrorTimeMillis = System.currentTimeMillis();

                    if (!closing) {
                        LOG.error("create connection Thread Interrupted, url: " + jdbcUrl, e);
                    }
                    break;
                } finally {
                    lock.unlock();
                }
                // 創建物理連接
                PhysicalConnectionInfo connection = null;

                try {
                    connection = createPhysicalConnection();
                    setFailContinuous(false);
                } catch (SQLException e) {
                    LOG.error("create connection SQLException, url: " + jdbcUrl + ", errorCode " + e.getErrorCode()
                              + ", state " + e.getSQLState(), e);

                    errorCount++;
                    // 創建連接默認失敗2次 ，失敗等待時間大於0 
                    if (errorCount > connectionErrorRetryAttempts && timeBetweenConnectErrorMillis > 0) {
                        // fail over retry attempts ，和取數據failfast 有關聯
                        setFailContinuous(true);
                        // failfast 默認false
                        if (failFast) {
                            lock.lock();
                            try {
                                notEmpty.signalAll();
                            } finally {
                                lock.unlock();
                            }
                        }
                        // breakAfterAcquireFailure 默認false
                        if (breakAfterAcquireFailure) {
                            break;
                        }
                        // 失敗等待時間默認500ms
                        try {
                            Thread.sleep(timeBetweenConnectErrorMillis);
                        } catch (InterruptedException interruptEx) {
                            break;
                        }
                    }
                } catch (RuntimeException e) {
                    LOG.error("create connection RuntimeException", e);
                    setFailContinuous(true);
                    continue;
                } catch (Error e) {
                    LOG.error("create connection Error", e);
                    setFailContinuous(true);
                    break;
                }

                if (connection == null) {
                    continue;
                }

                boolean result = put(connection);
                if (!result) {
                    JdbcUtils.close(connection.getPhysicalConnection());
                    LOG.info("put physical connection to pool failed.");
                }

                errorCount = 0; // reset errorCount
            }
        }
    }

解決方案

根源還是 dns 解析出現了問題導致，獲取連接線程超時拋出異常。

解決dns 解析問題
修改druid 創建連接源碼，如果發現dns解析錯誤異常 ,從正常連接中獲取正確ip。創建連接（修改druid源碼這事情交給druid比較好，druid版本1.1.5）

GetConnectionTimeoutException: wait millis 3000, active 11, maxActive 200,

異常2 GetConnectionTimeoutException: wait millis 3000, active 11, maxActive 200, creating 1

問題描述

初始化連接池創建流程

createConnectionThread 維護連接池流程

當連接創建失敗以後

解決方案

公司新來一個幹練小夥，把 MyBatis 替換成 MyBatis-Plus，上線後哭暈在廁所。。。

10分鐘本地運行llama3及初體驗

Testin雲測上線華爲Pura 70系列真機測試服務！

手寫協議報文 c語言手法

甲骨文(Oracle)宣佈將以74億美元收購Sun公司

mendix低代碼平臺具備的能力

mendix 學習

學堂業務理解

推送平臺技術優化迭代

私信整體架構

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結

GetConnectionTimeoutException: wait millis 3000, active 11, maxActive 200,

異常2 GetConnectionTimeoutException: wait millis 3000, active 11, maxActive 200, creating 1

問題 描述

初始化連接池創建流程

createConnectionThread 維護連接池流程

當連接創建失敗以後

解決方案

問題描述