【日記】2017-10-23 oracle 和線程問題 工單不生成 工單臨時表爲空

1.工單不生成

收到陝西同事的信息，說某個時間點後，工單就不生成了。

先要來了當天的日誌：

...

2017/10/20-11:05:41 [Thread-17] INFO Queueloader.class- 添加觸點到隊列：未找到可用的觸點信息

2017/10/20-11:05:51 [Thread-17] INFO Queueloader.class- 添加觸點到隊列：未找到可用的觸點信息

2017/10/20-11:06:00 [pool-10-thread-1] INFO GenOrder.class- 工單調度任務開始

Exception in thread "Thread-17" org.apache.ibatis.exceptions.PersistenceException: 

### Error querying database. Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: Io 異常: The Network Adapter could not establish the connection

### The error may exist in URL [jar:file:/home/triber/taskthread/triber-bui-taskthread-0.1.jar!/BOOT-INF/classes!/mapper/oracle/taskthread/SkillStatusMapper.xml]

### The error may involve org.triber.bui.taskthread.dao.SkillStatusMapper.selectByOrderStatus

### The error occurred while executing a query

### Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: Io 異常: The Network Adapter could not establish the connection

at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)

at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:150)

at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:141)

at org.triber.bui.common.base.MyBatis$SqlSessionTemplate$2.doInSession(MyBatis.java:434)

at org.triber.bui.common.base.MyBatis$SqlSessionTemplate.execute(MyBatis.java:402)

at org.triber.bui.common.base.MyBatis$SqlSessionTemplate.selectList(MyBatis.java:432)

at org.triber.bui.common.base.DaoMyBatis.queryList(DaoMyBatis.java:77)

at org.triber.bui.taskthread.service.impl.SkillStatusServiceImpl.selectByOrderStatusAndGenStatus(SkillStatusServiceImpl.java:89)

at org.triber.bui.taskthread.service.impl.SkillStatusServiceImpl$$FastClassBySpringCGLIB$$8de1cd40.invoke(<generated>)

at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:651)

at org.triber.bui.taskthread.service.impl.SkillStatusServiceImpl$$EnhancerBySpringCGLIB$$5cda1752.selectByOrderStatusAndGenStatus(<generated>)

at org.triber.bui.taskthread.service.impl.SkillStatusServiceImpl.addNewTask(SkillStatusServiceImpl.java:193)

at org.triber.bui.taskthread.service.impl.SkillStatusServiceImpl.addSkillToQueue(SkillStatusServiceImpl.java:168)

at org.triber.bui.taskthread.service.impl.SkillStatusServiceImpl$$FastClassBySpringCGLIB$$8de1cd40.invoke(<generated>)

at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:651)

at org.triber.bui.taskthread.service.impl.SkillStatusServiceImpl$$EnhancerBySpringCGLIB$$5cda1752.addSkillToQueue(<generated>)

at org.triber.bui.taskthread.schedule.muiltythread.Queueloader.run(Queueloader.java:54)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: Io 異常: The Network Adapter could not establish the connection

at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80)

at org.mybatis.spring.transaction.SpringManagedTransaction.openConnection(SpringManagedTransaction.java:82)

at org.mybatis.spring.transaction.SpringManagedTransaction.getConnection(SpringManagedTransaction.java:68)

at org.apache.ibatis.executor.BaseExecutor.getConnection(BaseExecutor.java:336)

at org.apache.ibatis.executor.SimpleExecutor.prepareStatement(SimpleExecutor.java:84)

at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:62)

at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:324)

at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:156)

at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:109)

at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:83)

at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:148)

... 18 more

Caused by: java.sql.SQLException: Io 異常: The Network Adapter could not establish the connection

at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)

at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:146)

at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:255)

at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:387)

at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:414)

at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:165)

at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:35)

at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:801)

at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:310)

at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:203)

at org.apache.tomcat.jdbc.pool.ConnectionPool.createConnection(ConnectionPool.java:718)

at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:650)

at org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:187)

at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:132)

at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111)

at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77)

... 28 more

2017/10/20-11:15:36 [pool-10-thread-1] INFO GenOrder.class- 獲取到已啓用活動數：2

全省OCS拯救計劃20171018===================1

2017/10/20-11:15:36 [pool-10-thread-1] INFO GenOrder.class- 賬期驗證

Fri Oct 20 11:15:36 CST 2017

2017/10/20-11:15:36 [pool-10-thread-1] INFO GenOrder.class- 活動所需賬期未更新

新入網用戶首充特權維繫20171019===================1

...

可見 jdbc 鏈接發生異常。線程 Thread-17 崩潰，導致工單生成停止。

在要來當地部署的代碼截圖

 

線程的執行體內沒有做異常處理。。。。

補充之後，暫時的解決方案：

@Override

    public void run() {

        // TODO Auto-generated method stub

        while(true){

            try {

                updateQueue();

                updateStatus();

                queue.rebuildQueue();

                Tools.waitSomeSecond(WaitSecond);

                if(ifRun && AllRun){

                    //Tools.waitSomeSecond(cycleTime);

                    //System.err.println(CommenBeanHolder.skillStatusService.getClass());

                    //System.out.println("load一個runnner");

                    String rt = CommenBeanHolder.skillStatusService.addSkillToQueue();

                    logger.info("添加觸點到隊列：" + rt);

                }

            }catch (PersistenceException e) {

                logger.error("工單生成 隊列管理線程 運行異常:數據庫鏈接異常");

                e.printStackTrace();

                Tools.waitSomeSecond(WaitSecond);

            }catch (Exception e) {

                logger.error("工單生成 隊列管理線程 運行異常");

                e.printStackTrace();

                Tools.waitSomeSecond(WaitSecond);

            }

        }

    }

2.臨時表爲空

發現某些工單臨時表哦爲空

初步認爲是用戶羣sql問題

查看日誌後發現sql狀態正常，在xcloud執行後的用戶數量正常

後懷疑是csv文件讀取失敗或文件爲空，未驗證

最後查看日誌發現數據庫異常，可能是表空間滿導致