關注業務-把優化做到極致

SELECT .. .. ..
  FROM AWP.TTT_TTT_TRANSACTION_2018 INCT
  LEFT JOIN AWP.TTT_TTT_TELM TELM
    ON INCT.TELLER_NO = TELM.TELLER_NO
  LEFT JOIN SJB_CD_01.TMP_TT_BAN_EDP ED1P
    ON INCT.TRAN_CODE = ED1P.交易代碼
  LEFT JOIN AWP.TTT_TTT_CB_CIF CUSM
    ON INCT.CUSTOMER_NO = CUSM.CUSTOMER_NO_MICM
  LEFT JOIN AWP.TTT_TTT_CB_ACCT INVM
    ON INCT.ACCT_NO = INVM.ACCT_NO
  LEFT JOIN AWP.TTT_TTT_BRHM BRHM
    ON INVM.BRANCH_NO = BRHM.BRANCH_NO
  LEFT JOIN AWP.TTT_TTT_DEPP DEPP
    ON INVM.ACCT_TYPE = DEPP.ACCT_TYPE || DEPP.INT_CAT
  JOIN SJB_CD_01.TMP_TT_LZ_POS_XYK_ALL XYK
    ON XYK.持卡人證件號碼 = CUSM.ID_NO_MICM
 WHERE 1 = 1
   AND TO_DATE(INCT.TRAN_DATE, 'yyyy-mm-dd') BETWEEN
       TO_DATE(XYK.交易日期, 'yyyy-mm-dd') AND
       TO_DATE(XYK.交易日期, 'yyyy-mm-dd') + 5
   AND INCT.INCT_01_AMOUNT BETWEEN XYK.刷卡金額 * 0.9 AND XYK.刷卡金額

SQL執行6分鐘以後報錯:ORA-12805:並行查詢服務器意外停止

1.查看執行計劃,確認SQL是否進行的並行操作

PLAN HASH VALUE: 1375319919                                                                                                                                                                      
                                                                                                                                                                                                 
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------                
| ID  | OPERATION                                         | NAME                           | ROWS  | BYTES | COST (%CPU)| TIME     | PSTART| PSTOP |    TQ  |IN-OUT| PQ DISTRIB |                
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------                
|   0 | SELECT STATEMENT                                  |                                |     1 |  1480 | 14871   (1)| 00:02:59 |       |       |        |      |            |                
|   1 |  PX COORDINATOR                                   |                                |       |       |            |          |       |       |        |      |            |                
|   2 |   PX SEND QC (RANDOM)                             | :TQ10009                       |     1 |  1480 | 14871   (1)| 00:02:59 |       |       |  Q1,09 | P->S | QC (RAND)  |                
|*  3 |    HASH JOIN OUTER                                |                                |     1 |  1480 | 14871   (1)| 00:02:59 |       |       |  Q1,09 | PCWP |            |                
|   4 |     PX RECEIVE                                    |                                |     1 |  1231 | 14858   (1)| 00:02:59 |       |       |  Q1,09 | PCWP |            |                
|   5 |      PX SEND HASH                                 | :TQ10008                       |     1 |  1231 | 14858   (1)| 00:02:59 |       |       |  Q1,08 | P->P | HASH       |                
|*  6 |       HASH JOIN OUTER                             |                                |     1 |  1231 | 14858   (1)| 00:02:59 |       |       |  Q1,08 | PCWP |            |                
|   7 |        PX RECEIVE                                 |                                |     1 |  1103 | 14845   (1)| 00:02:59 |       |       |  Q1,08 | PCWP |            |                
|   8 |         PX SEND HASH                              | :TQ10007                       |     1 |  1103 | 14845   (1)| 00:02:59 |       |       |  Q1,07 | P->P | HASH       |                
|   9 |          NESTED LOOPS OUTER                       |                                |     1 |  1103 | 14845   (1)| 00:02:59 |       |       |  Q1,07 | PCWP |            |                
|* 10 |           HASH JOIN OUTER                         |                                |     1 |   976 | 14841   (1)| 00:02:59 |       |       |  Q1,07 | PCWP |            |                
|  11 |            PX RECEIVE                             |                                |     1 |   854 | 14532   (1)| 00:02:55 |       |       |  Q1,07 | PCWP |            |                
|  12 |             PX SEND HASH                          | :TQ10006                       |     1 |   854 | 14532   (1)| 00:02:55 |       |       |  Q1,06 | P->P | HASH       |                
|* 13 |              HASH JOIN OUTER                      |                                |     1 |   854 | 14532   (1)| 00:02:55 |       |       |  Q1,06 | PCWP |            |                
|  14 |               PX RECEIVE                          |                                |     1 |   803 | 14517   (1)| 00:02:55 |       |       |  Q1,06 | PCWP |            |                
|  15 |                PX SEND HASH                       | :TQ10005                       |     1 |   803 | 14517   (1)| 00:02:55 |       |       |  Q1,05 | P->P | HASH       |                
|  16 |                 NESTED LOOPS                      |                                |     1 |   803 | 14517   (1)| 00:02:55 |       |       |  Q1,05 | PCWP |            |                
|  17 |                  NESTED LOOPS                     |                                |  1131 |   803 | 14517   (1)| 00:02:55 |       |       |  Q1,05 | PCWP |            |                
|  18 |                   BUFFER SORT                     |                                |       |       |            |          |       |       |  Q1,05 | PCWC |            |                
|  19 |                    PX RECEIVE                     |                                |       |       |            |          |       |       |  Q1,05 | PCWP |            |                
|  20 |                     PX SEND BROADCAST             | :TQ10000                       |       |       |            |          |       |       |        | S->P | BROADCAST  |                
|  21 |                      NESTED LOOPS                 |                                |    13 |  6162 |    59   (0)| 00:00:01 |       |       |        |      |            |                
|  22 |                       TABLE ACCESS FULL           | TMP_TT_LZ_POS_XYK_ALL          |    14 |  2898 |     3   (0)| 00:00:01 |       |       |        |      |            |                
|  23 |                       TABLE ACCESS BY INDEX ROWID | TTT_TTT_CB_CIF                 |     1 |   267 |     4   (0)| 00:00:01 |       |       |        |      |            |                
|* 24 |                        INDEX RANGE SCAN           | IDX1_TTT_TTT_CB_CIF            |     1 |       |     2   (0)| 00:00:01 |       |       |        |      |            |                
|  25 |                   PX PARTITION LIST ALL           |                                |    87 |       |  1092   (0)| 00:00:14 |     1 |   364 |  Q1,05 | PCWC |            |                
|* 26 |                    INDEX RANGE SCAN               | IND2_BANTRANSACTION_CCDCX_2018 |    87 |       |  1092   (0)| 00:00:14 |     1 |   364 |  Q1,05 | PCWP |            |                
|* 27 |                  TABLE ACCESS BY LOCAL INDEX ROWID| TTT_TTT_TRANSACTION_2018       |     1 |   329 |  1132   (0)| 00:00:14 |     1 |     1 |  Q1,05 | PCWP |            |                
|  28 |               BUFFER SORT                         |                                |       |       |            |          |       |       |  Q1,06 | PCWC |            |                
|  29 |                PX RECEIVE                         |                                |  5350 |   266K|    14   (0)| 00:00:01 |       |       |  Q1,06 | PCWP |            |                
|  30 |                 PX SEND HASH                      | :TQ10001                       |  5350 |   266K|    14   (0)| 00:00:01 |       |       |        | S->P | HASH       |                
|  31 |                  TABLE ACCESS FULL                | TMP_TT_BAN_EDP                 |  5350 |   266K|    14   (0)| 00:00:01 |       |       |        |      |            |                
|  32 |            BUFFER SORT                            |                                |       |       |            |          |       |       |  Q1,07 | PCWC |            |                
|  33 |             PX RECEIVE                            |                                | 57656 |  6869K|   309   (1)| 00:00:04 |       |       |  Q1,07 | PCWP |            |                
|  34 |              PX SEND HASH                         | :TQ10002                       | 57656 |  6869K|   309   (1)| 00:00:04 |       |       |        | S->P | HASH       |                
|  35 |               TABLE ACCESS FULL                   | TTT_TTT_TELM                   | 57656 |  6869K|   309   (1)| 00:00:04 |       |       |        |      |            |                
|  36 |           TABLE ACCESS BY INDEX ROWID             | TTT_TTT_CB_ACCT                |     1 |   127 |     4   (0)| 00:00:01 |       |       |  Q1,07 | PCWP |            |                
|* 37 |            INDEX RANGE SCAN                       | IND1_TTT_TTT_CB_ACCT           |     1 |       |     3   (0)| 00:00:01 |       |       |  Q1,07 | PCWP |            |                
|  38 |        BUFFER SORT                                |                                |       |       |            |          |       |       |  Q1,08 | PCWC |            |                
|  39 |         PX RECEIVE                                |                                |  1937 |   242K|    13   (0)| 00:00:01 |       |       |  Q1,08 | PCWP |            |                
|  40 |          PX SEND HASH                             | :TQ10003                       |  1937 |   242K|    13   (0)| 00:00:01 |       |       |        | S->P | HASH       |                
|  41 |           TABLE ACCESS FULL                       | TTT_TTT_BRHM                   |  1937 |   242K|    13   (0)| 00:00:01 |       |       |        |      |            |                
|  42 |     BUFFER SORT                                   |                                |       |       |            |          |       |       |  Q1,09 | PCWC |            |                
|  43 |      PX RECEIVE                                   |                                |  1134 |   275K|    13   (0)| 00:00:01 |       |       |  Q1,09 | PCWP |            |                
|  44 |       PX SEND HASH                                | :TQ10004                       |  1134 |   275K|    13   (0)| 00:00:01 |       |       |        | S->P | HASH       |                
|  45 |        TABLE ACCESS FULL                          | TTT_TTT_DEPP                   |  1134 |   275K|    13   (0)| 00:00:01 |       |       |        |      |            |                
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------                
                                                                                                                                                                                                 
PREDICATE INFORMATION (IDENTIFIED BY OPERATION ID):                                                                                                                                              
---------------------------------------------------                                                                                                                                              
                                                                                                                                                                                                 
   3 - ACCESS("INVM"."ACCT_TYPE"="DEPP"."ACCT_TYPE"(+)||"DEPP"."INT_CAT"(+))                                                                                                                     
   6 - ACCESS("INVM"."BRANCH_NO"="BRHM"."BRANCH_NO"(+))                                                                                                                                          
  10 - ACCESS("INCT"."TELLER_NO"="TELM"."TELLER_NO"(+))                                                                                                                                          
  13 - ACCESS("INCT"."TRAN_CODE"="ED1P"."交易代碼"(+))                                                                                                                                           
  24 - ACCESS("XYK"."持卡人證件號碼"="CUSM"."ID_NO_MICM")                                                                                                                                        
  26 - ACCESS("INCT"."CUSTOMER_NO"="CUSM"."CUSTOMER_NO_MICM")                                                                                                                                    
  27 - FILTER(TO_DATE("INCT"."TRAN_DATE",'yyyy-mm-dd')>=TO_DATE("XYK"."交易日期",'yyyy-mm-dd') AND                                                                                               
              TO_DATE("INCT"."TRAN_DATE",'yyyy-mm-dd')<=TO_DATE("XYK"."交易日期",'yyyy-mm-dd')+5 AND "INCT"."INCT_01_AMOUNT">="XYK"."刷卡金額"*0.9 AND "INCT"."INCT_01_AMOUNT"<="XYK"."刷卡金額")
  37 - ACCESS("INCT"."ACCT_NO"="INVM"."ACCT_NO"(+))   

2.查看產生並行操作的原因,SQL語句本身沒有並行相關HINT,查看SQL表和索引信息以及並行信息如下:

OWNER          OBJECT_TYPE   OBJECT_NAME                   PARTITIONED    SIZE_MB   NUM_ROWS LAST_ANALYZED STATUS
-------------- ------------- ----------------------------- ----------- ---------- ---------- ------------- --------------
AWP            TABLE         TTT_TTT_BRHM                  NO               0.375       1937 2019/3/4 22:1 統計信息過期
AWP            TABLE         TTT_TTT_CB_ACCT               NO                7011   48184163 2019/3/1 22:2 統計信息過期
AWP            TABLE         TTT_TTT_CB_CIF                NO                9338   30525855 2019/3/2 1:29 統計信息過期
AWP            TABLE         TTT_TTT_DEPP                  NO               0.375       1134 2019/3/4 22:1 統計信息過期
AWP            TABLE         TTT_TTT_TELM                  NO                   9      57656 2019/3/5 22:0 統計信息未過期
AWP            TABLE         TTT_TTT_TRANSACTION_2018      YES         499774.187 1303013540 2018/3/20 10: 統計信息未過期
SJB_CD_01      TABLE         TMP_TT_BAN_EDP                NO               0.375       5350 2017/7/5 22:0 統計信息未過期
SJB_CD_01      TABLE         TMP_TT_LZ_POS_XYK_ALL         NO              0.0625         14 2019/2/22 22: 統計信息未過期

TABLE_NAME                     DEGREE
------------------------------ --------
SJB_CD_01 TMP_TT_BAN_EDP            1
SJB_CD_01 TMP_TT_LZ_POS_XYK_ALL     1
AWP TTT_TTT_TRANSACTION_2018        1
AWP TTT_TTT_CB_CIF                  1
AWP TTT_TTT_BRHM                    1
AWP TTT_TTT_TELM                    1
AWP TTT_TTT_DEPP                    1
AWP TTT_TTT_CB_ACCT                 1
INDEX_NAME                     TABLE_NAME                     DEGREE
------------------------------ ------------------------------ ----------------------------------------
IND1_TTT_TTT_CB_ACCT           TTT_TTT_CB_ACCT                1
IND2_TTT_TTT_CB_ACCT           TTT_TTT_CB_ACCT                1
IND3_TTT_TTT_CB_ACCT           TTT_TTT_CB_ACCT                8
PK_TTT_TTT_CB_ACCT             TTT_TTT_CB_ACCT                1
PK_TTT_TTT_CB_CIF              TTT_TTT_CB_CIF                 1
IDX1_TTT_TTT_CB_CIF            TTT_TTT_CB_CIF                 1
IDX2_TTT_TTT_CB_CIF            TTT_TTT_CB_CIF                 1
IDX3_TTT_TTT_CB_CIF            TTT_TTT_CB_CIF                 1
IDX1_TTT_TTT_DEPP              TTT_TTT_DEPP                   1
PK_TTT_TTT_TELM                TTT_TTT_TELM                   1
IND1_BANTRANSACTION_AITI_2018  TTT_TTT_TRANSACTION_2018       4
IND2_BANTRANSACTION_CCDCX_2018 TTT_TTT_TRANSACTION_2018       4
IND3_BANTRANSACTION_E_2018     TTT_TTT_TRANSACTION_2018       4
IND4_BANTRANSACTION_I_2018     TTT_TTT_TRANSACTION_2018       4

其實腳本可以優化一下:添加一列DEGREE

3.由上可知,因爲索引的並行,唯一的大表TTT_TTT_TRANSACTION_2018使用並行的方式訪問導致以上的ORA報錯。

4.索引並行度產生的原因

 a)創建索引的時候,爲了縮短create時間,使用了並行的方式創建

 b)分區表對分區操作的時候需要維護(rebuild)索引,使用了並行的方式

解決:只需要關閉索引的並行度即可:alter index xxxxxx noparallel

5.因是線上環境,無權限進行直接alter操作,故使用HINT從SQL語句上消除並行

6./*+ noparallel USE_HASH(INCT TELM ED1P CUSM INVM BRHM DEPP XYK) */  執行計劃如下

PLAN HASH VALUE: 374343828                                                                                                                                                                                                                                     
                                                                                                                     
---------------------------------------------------------------------------------------------------------------------
| ID  | OPERATION                | NAME                     | ROWS  | BYTES | COST (%CPU)| TIME     | PSTART| PSTOP |
---------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT         |                          |     1 |  1480 |    17M  (1)| 58:53:47 |       |       |
|*  1 |  HASH JOIN OUTER         |                          |     1 |  1480 |    17M  (1)| 58:53:47 |       |       |
|*  2 |   HASH JOIN OUTER        |                          |     1 |  1231 |    17M  (1)| 58:53:47 |       |       |
|*  3 |    HASH JOIN OUTER       |                          |     1 |  1103 |    17M  (1)| 58:53:46 |       |       |
|*  4 |     HASH JOIN OUTER      |                          |     1 |   976 |    17M  (1)| 58:05:07 |       |       |
|*  5 |      HASH JOIN OUTER     |                          |     1 |   854 |    17M  (1)| 58:05:03 |       |       |
|*  6 |       HASH JOIN          |                          |     1 |   803 |    17M  (1)| 58:05:03 |       |       |
|*  7 |        HASH JOIN         |                          |    13 |  6162 |   325K  (1)| 01:05:07 |       |       |
|   8 |         TABLE ACCESS FULL| TMP_TT_LZ_POS_XYK_ALL    |    14 |  2898 |     3   (0)| 00:00:01 |       |       |
|   9 |         TABLE ACCESS FULL| TTT_TTT_CB_CIF           |    30M|  7772M|   325K  (1)| 01:05:05 |       |       |
|  10 |        PARTITION LIST ALL|                          |  1303M|   399G|    17M  (1)| 56:58:32 |     1 |   364 |
|  11 |         TABLE ACCESS FULL| TTT_TTT_TRANSACTION_2018 |  1303M|   399G|    17M  (1)| 56:58:32 |     1 |   364 |
|  12 |       TABLE ACCESS FULL  | TMP_TT_BAN_EDP           |  5350 |   266K|    14   (0)| 00:00:01 |       |       |
|  13 |      TABLE ACCESS FULL   | TTT_TTT_TELM             | 57656 |  6869K|   309   (1)| 00:00:04 |       |       |
|  14 |     TABLE ACCESS FULL    | TTT_TTT_CB_ACCT          |    48M|  5835M|   243K  (1)| 00:48:37 |       |       |
|  15 |    TABLE ACCESS FULL     | TTT_TTT_BRHM             |  1937 |   242K|    13   (0)| 00:00:01 |       |       |
|  16 |   TABLE ACCESS FULL      | TTT_TTT_DEPP             |  1134 |   275K|    13   (0)| 00:00:01 |       |       |
---------------------------------------------------------------------------------------------------------------------
                                                                                                                     
PREDICATE INFORMATION (IDENTIFIED BY OPERATION ID):                                                                  
---------------------------------------------------                                                                  
                                                                                                                     
   1 - ACCESS("INVM"."ACCT_TYPE"="DEPP"."ACCT_TYPE"(+)||"DEPP"."INT_CAT"(+))                                         
   2 - ACCESS("INVM"."BRANCH_NO"="BRHM"."BRANCH_NO"(+))                                                              
   3 - ACCESS("INCT"."ACCT_NO"="INVM"."ACCT_NO"(+))                                                                  
   4 - ACCESS("INCT"."TELLER_NO"="TELM"."TELLER_NO"(+))                                                              
   5 - ACCESS("INCT"."TRAN_CODE"="ED1P"."交易代碼"(+))                                                               
   6 - ACCESS("INCT"."CUSTOMER_NO"="CUSM"."CUSTOMER_NO_MICM")                                                        
       FILTER(TO_DATE("INCT"."TRAN_DATE",'yyyy-mm-dd')>=TO_DATE("XYK"."交易日期",'yyyy-mm-dd') AND                   
              TO_DATE("INCT"."TRAN_DATE",'yyyy-mm-dd')<=TO_DATE("XYK"."交易日期",'yyyy-mm-dd')+5 AND                 
              "INCT"."INCT_01_AMOUNT">="XYK"."刷卡金額"*0.9 AND "INCT"."INCT_01_AMOUNT"<="XYK"."刷卡金額")           
   7 - ACCESS("XYK"."持卡人證件號碼"="CUSM"."ID_NO_MICM")  

7.由第二步可知,SQL語句中只有一張大表,猜測其他都是參數表,是典型的星型架構的SQL語句。這一類的SQL的主要消耗在這一張大表的訪問方式。這裏是全表掃描TABLE ACCESS FULL| TTT_TTT_TRANSACTION_2018 |  1303M。避免出現其他幺蛾子,所以使用HASH固定SQL的執行計劃。這樣一來SQL確實沒有報錯,但是運行了2個小時纔出結果,2個小時,這顯然不能忍......

8.查看SQL語句可以看出大表有過濾條件TO_DATE(INCT.TRAN_DATE, 'yyyy-mm-dd') BETWEEN
       TO_DATE(XYK.交易日期, 'yyyy-mm-dd') AND
       TO_DATE(XYK.交易日期, 'yyyy-mm-dd') + 5 ,而TRAN_DATE正好是分區鍵,只是SQL是通過傳值的方式從表中獲取數據,順便查了以下表XYZ的數據量,還好只有幾條。這邊把值帶入,量化變量,得出如下SQL語句

select /*+ NOPARALLEL use_hash(inct telm ed1p cusm invm brhm depp xyk) */
 ......
  from awp.t48_ban_transaction_2018 inct
  left join awp.t48_ban_telm telm
    on inct.teller_no = telm.teller_no
  left join sjb_cd_01.tmp_lj_ban_edp ed1p
    on inct.tran_code = ed1p.交易代碼
  left join awp.t48_ban_cb_cif cusm
    on inct.customer_no = cusm.customer_no_micm
  left join awp.t48_ban_cb_acct invm
    on inct.acct_no = invm.acct_no
  left join awp.t48_ban_brhm brhm
    on invm.branch_no = brhm.branch_no
  left join awp.t48_ban_depp depp
    on invm.acct_type = depp.acct_type || depp.int_cat
  join sjb_cd_01.tmp_lj_lz_pos_xyk_all xyk
    on xyk.持卡人證件號碼 = cusm.id_no_micm
 where 1 = 1
   and (inct.tran_date > '2018-12-13' AND inct.tran_date < '2018-12-18')
    OR (inct.tran_date > '2018-06-28' AND inct.tran_date < '2018-07-04')
    OR (inct.tran_date > '2018-11-02' AND inct.tran_date < '2018-11-07')
    OR (inct.tran_date > '2018-08-26' AND inct.tran_date < '2018-09-03')
    OR (inct.tran_date > '2018-07-25' AND inct.tran_date < '2018-07-30')
   and inct.inct_01_amount between xyk.刷卡金額 * 0.9 and xyk.刷卡金額;

SQL使用了分區裁剪,12分鐘出結果~這個結果勉強能忍,但是仍不盡如人意。要求儘量優化到分鐘級

於是我發出提問:這個SQL最終返回結果是多少?

因爲這個數據量要做到分鐘級,顯然HASH不可能,只有NEST LOOP可行,如果返回結果少,說明有可能走NEST LOOP

得到的回覆是:很少最多不超過100條

這樣看來可以走NEST LOOP,那麼怎麼查看是哪張表起的作用呢?顯然我們需要找與大表直接關聯的表,然後驗證這個表是否能夠把與大表關聯的結果集變小,如果可以,嵌套循環驅動到底就行。於是加如下HINT /*+ leading(xyk) noparallel */ 原SQL 0.2秒出結果!!!!

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章