RMAN-03009，ORA-19566 及坏块修复

EBS R12.1.3
RDBMS 11.1.0.7

参考文档如何格式化不属于任何段的损坏块 (Doc ID 1526163.1)

-- 在RMAN进行备份的时候，提示RMAN-03009和ORA-19566错误

RMAN-03009: failure of backup command on ORA_DISK_1 channel at 04/09/2020 11:14:59
ORA-19566: exceeded limit of 0 corrupt blocks for file /d01/oracle/PROD/db/apps_st/data/system09.dbf
continuing other job steps, job failed will not be re-run

-- 查看该错误号

[oraprod@ebsr121 ~]$ oerr ora 19566
19566, 00000, "exceeded limit of %s corrupt blocks for file %s"
// *Cause:  The user specified limit of allowable corrupt blocks was exceeded
//          while reading the specified datafile for a datafile copy or 
//          backup.
// *Action: None. The copy or backup operation fails.
//          The session trace file contains detailed information
//          about which blocks were corrupt.
[oraprod@ebsr121 ~]$

-- alert log中的日志，发现有坏块

Thu Apr 09 10:13:02 2020
Hex dump of (file 352, block 165656) in trace file /d01/oracle/PROD/db/tech_st/11.1.0/admin/TEST_ebsr121/diag/rdbms/test/TEST/trace/TEST_ora_4769.trc
Corrupt block relative dba: 0x58028718 (file 352, block 165656)
Completely zero block found during backing up datafile
Reread of blocknum=165656, file=/d01/oracle/PROD/db/apps_st/data/system09.dbf. found same corrupt data
Reread of blocknum=165656, file=/d01/oracle/PROD/db/apps_st/data/system09.dbf. found same corrupt data
Reread of blocknum=165656, file=/d01/oracle/PROD/db/apps_st/data/system09.dbf. found same corrupt data
Reread of blocknum=165656, file=/d01/oracle/PROD/db/apps_st/data/system09.dbf. found same corrupt data
Reread of blocknum=165656, file=/d01/oracle/PROD/db/apps_st/data/system09.dbf. found same corrupt data
Thu Apr 09 10:13:15 2020
Hex dump of (file 352, block 165657) in trace file /d01/oracle/PROD/db/tech_st/11.1.0/admin/TEST_ebsr121/diag/rdbms/test/TEST/trace/TEST_ora_4769.trc
Corrupt block relative dba: 0x58028719 (file 352, block 165657)
Completely zero block found during backing up datafile
Reread of blocknum=165657, file=/d01/oracle/PROD/db/apps_st/data/system09.dbf. found same corrupt data
Reread of blocknum=165657, file=/d01/oracle/PROD/db/apps_st/data/system09.dbf. found same corrupt data
Reread of blocknum=165657, file=/d01/oracle/PROD/db/apps_st/data/system09.dbf. found same corrupt data
Reread of blocknum=165657, file=/d01/oracle/PROD/db/apps_st/data/system09.dbf. found same corrupt data
Reread of blocknum=165657, file=/d01/oracle/PROD/db/apps_st/data/system09.dbf. found same corrupt data
Thu Apr 09 10:13:32 2020

-- sql语句查询坏块。看官方文档，当运行了rman后，会将坏块情况记录到该视图中。所以看准确的坏块还是通过RMAN或者DBV来查看。

SYS@TEST>select file#,block#,corruption_type  from v$database_block_corruption;

     FILE#     BLOCK# CORRUPTIO
---------- ---------- ---------
       352     165657 ALL ZERO
       352     165656 CORRUPT

SYS@TEST>

-- 使用dbv来检查system09.dbf文件。没有备份，没有办法使用rman进行恢复，并使用RMAN进行查看损坏的块

[oraprod@ebsr121 ~]$ rman target /

Recovery Manager: Release 11.1.0.7.0 - Production on Thu Apr 9 17:23:35 2020

Copyright (c) 1982, 2007, Oracle.  All rights reserved.

connected to target database: TEST (DBID=2335061898)

RMAN> backup validate check logical datafile 352;

Starting backup at 09-APR-20
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=358 device type=DISK
allocated channel: ORA_DISK_2
channel ORA_DISK_2: SID=361 device type=DISK
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00352 name=/d01/oracle/PROD/db/apps_st/data/system09.dbf
channel ORA_DISK_1: backup set complete, elapsed time: 03:55:03
List of Datafiles
=================
File Status Marked Corrupt Empty Blocks Blocks Examined High SCN
---- ------ -------------- ------------ --------------- ----------
352  FAILED 0              1967         192128          5965138801470
  File Name: /d01/oracle/PROD/db/apps_st/data/system09.dbf
  Block Type Blocks Failing Blocks Processed
  ---------- -------------- ----------------
  Data       0              131350          
  Index      0              30379           
  Other      26468          28432           

validate found one or more corrupt blocks
See trace file /d01/oracle/PROD/db/tech_st/11.1.0/admin/TEST_ebsr121/diag/rdbms/test/TEST/trace/TEST_ora_4766.trc for details
Finished backup at 09-APR-20

RMAN>

-- 使用dbv查看，最大的坏块是192128

[oraprod@ebsr121 ~]$ dbv file=/d01/oracle/PROD/db/apps_st/data/system09.dbf 
........
Page 192125 is marked corrupt
Corrupt block relative dba: 0x5802ee7d (file 352, block 192125)
Completely zero block found during dbv: 

Page 192126 is marked corrupt
Corrupt block relative dba: 0x5802ee7e (file 352, block 192126)
Completely zero block found during dbv: 

Page 192127 is marked corrupt
Corrupt block relative dba: 0x5802ee7f (file 352, block 192127)
Completely zero block found during dbv: 

Page 192128 is marked corrupt
Corrupt block relative dba: 0x5802ee80 (file 352, block 192128)
Completely zero block found during dbv: 



DBVERIFY - Verification complete

Total Pages Examined         : 192128
Total Pages Processed (Data) : 66285
Total Pages Failing   (Data) : 0
Total Pages Processed (Index): 30379
Total Pages Failing   (Index): 0
Total Pages Processed (Other): 1964
Total Pages Processed (Seg)  : 0
Total Pages Failing   (Seg)  : 0
Total Pages Empty            : 67027
Total Pages Marked Corrupt   : 26473
Total Pages Influx           : 0
Total Pages Encrypted        : 0
Highest block SCN            : 3723896841 (1388.3723896841)
[oraprod@ebsr121 ~]$

-- 再次查看v$database_block_corruption视图

SYS@TEST>select * from v$database_block_corruption;

     FILE#     BLOCK#     BLOCKS CORRUPTION_CHANGE# CORRUPTIO
---------- ---------- ---------- ------------------ ---------
       352     165661      10051                  0 ALL ZERO
       352     175713      16416                  0 ALL ZERO
       352     175712          1                  0 ALL ZERO

SYS@TEST>

-- 检查块是否是某个对象的一部分，-- 适用于损坏的块号小的情况

SQL> select segment_name, segment_type, owner
       from dba_extents
      where file_id = <Absolute file number>
        and <corrupted block number> between block_id
            and block_id + blocks -1;

-- 如果块不属于任何对象，查询dba_free_space确认块是否属于数据文件的可用空间

SQL> Select * from dba_free_space where file_id= <Absolute file number>
     and <corrupted block number> between block_id and block_id + blocks -1;

-- 查找受影响的块并验证其是否属于任何段的一种比较好的方法是使用RMAN，该方法好用又便捷

backup check logical validate database;  -- 前面已经针对352号文件运行过

-- 运行以下SQL，确认块是位于可用空间中还是已占用空间中，查询结果是free block

set lines 200 pages 10000
col segment_name format a30

SELECT e.owner, e.segment_type, e.segment_name, e.partition_name, c.file#
, greatest(e.block_id, c.block#) corr_start_block#
, least(e.block_id+e.blocks-1, c.block#+c.blocks-1) corr_end_block#
, least(e.block_id+e.blocks-1, c.block#+c.blocks-1)
- greatest(e.block_id, c.block#) + 1 blocks_corrupted
, null description
FROM dba_extents e, v$database_block_corruption c
WHERE e.file_id = c.file#
AND e.block_id <= c.block# + c.blocks - 1
AND e.block_id + e.blocks - 1 >= c.block#
UNION
SELECT s.owner, s.segment_type, s.segment_name, s.partition_name, c.file#
, header_block corr_start_block#
, header_block corr_end_block#
, 1 blocks_corrupted
, 'Segment Header' description
FROM dba_segments s, v$database_block_corruption c
WHERE s.header_file = c.file#
AND s.header_block between c.block# and c.block# + c.blocks - 1
UNION
SELECT null owner, null segment_type, null segment_name, null partition_name, c.file#
, greatest(f.block_id, c.block#) corr_start_block#
, least(f.block_id+f.blocks-1, c.block#+c.blocks-1) corr_end_block#
, least(f.block_id+f.blocks-1, c.block#+c.blocks-1)
- greatest(f.block_id, c.block#) + 1 blocks_corrupted
, 'Free Block' description
FROM dba_free_space f, v$database_block_corruption c
WHERE f.file_id = c.file#
AND f.block_id <= c.block# + c.blocks - 1
AND f.block_id + f.blocks - 1 >= c.block#
ORDER BY file#, corr_start_block#;


OWNER                          SEGMENT_TYPE       SEGMENT_NAME                   PARTITION_NAME                      FILE# CORR_START_BLOCK# CORR_END_BLOCK# BLOCKS_CORRUPTED DESCRIPTION
------------------------------ ------------------ ------------------------------ ------------------------------ ---------- ----------------- --------------- ---------------- --------------
                                                                                                                       352            165656          165656                1 Free Block
                                                                                                                       352            165657          165657                1 Free Block

SQL>

-- 以非sys或system用户的身份创建一个表，该表位于出现坏块的表空间system中，使用nologging选项，避免生成redo

conn scott/tiger 
create table s (n number,c varchar2(4000)) nologging tablespace system;

-- 验证表是否创建在了正确的表空间

select segment_name,tablespace_name from user_segments where segment_name='S' ;

-- 在表上创建触发器 , 根据dbv的结果，块号最高块 192128 ，数据文件352

conn / as sysdba 
CREATE OR REPLACE TRIGGER corrupt_trigger 
  AFTER INSERT ON scott.s 
  REFERENCING OLD AS p_old NEW AS new_p 
  FOR EACH ROW 
DECLARE 
  corrupt EXCEPTION; 
BEGIN 
  IF (dbms_rowid.rowid_block_number(:new_p.rowid)=&blocknumber)
 and (dbms_rowid.rowid_relative_fno(:new_p.rowid)=&filenumber) THEN 
     RAISE corrupt; 
  END IF; 
EXCEPTION 
  WHEN corrupt THEN 
     RAISE_APPLICATION_ERROR(-20000, 'Corrupt block has been formatted'); 
END; 
/

-- 为受影响的数据文件中的表分配空间，这里计算下来是732264K 。官方文档计算下来是64K

Select BYTES from dba_free_space where file_id=352 and 192128 between block_id and block_id + blocks -1;

SYS@TEST>Select BYTES from dba_free_space where file_id=352 and 192128 between block_id and block_id + blocks -1;

     BYTES
----------
 749838336

SYS@TEST>

-- 在本例子中，是732264K，按照以下方法分配extent

alter table scott.s allocate extent (DATAFILE '/d01/oracle/PROD/db/apps_st/data/system09.dbf' SIZE 732264K);

SYS@TEST>alter table scott.s allocate extent (DATAFILE '/d01/oracle/PROD/db/apps_st/data/system09.dbf' SIZE 732264K);

Table altered.

SYS@TEST>

-- 如果在这个数据文件中有多个空闲extent，则可能需要使用这个循环（本例子中没有使用）

##BEGIN
##for i in 1..1000000 loop
##EXECUTE IMMEDIATE 'alter table scott.s allocate extent (DATAFILE '||'''/d01/oracle/PROD/db/apps_st/data/system09.dbf''' ||'SIZE 64K) ';
##end loop;
##end ;
##/

-- 继续分配空间，直到坏块称为scott.s的一部分，使用以下查询进行检查

select segment_name, segment_type, owner
       from dba_extents
      where file_id = 352
        and 192128 between block_id
            and block_id + blocks -1 ;


SYS@TEST>select segment_name, segment_type, owner
       from dba_extents
      where file_id = 352
        and 192128 between block_id
            and block_id + blocks -1 ;  2    3    4    5

SEGMENT_NAME
--------------------------------------------------------------------------------
SEGMENT_TYPE       OWNER
------------------ ------------------------------
S
TABLE              SCOTT


SYS@TEST>

-- 向表中插入数据格式化块。每向表中插入一行就会触发触发器，当向坏块中插入第一行数据的时候，会产生ORA-2000异常。

BEGIN
FOR i IN 1..1000000000 LOOP
INSERT /*+ APPEND */ INTO scott.s select i, lpad('REFORMAT',3092, 'R') from dual;
commit ;
END LOOP;
END;

SYS@TEST>BEGIN
FOR i IN 1..1000000000 LOOP
INSERT /*+ APPEND */ INTO scott.s select i, lpad('REFORMAT',3092, 'R') from dual;
commit ;
END LOOP;
END;  2    3    4    5    6
  7  /
BEGIN
*
ERROR at line 1:
ORA-20000: Corrupt block has been formatted
ORA-06512: at "SYS.CORRUPT_TRIGGER", line 10
ORA-04088: error during execution of trigger 'SYS.CORRUPT_TRIGGER'
ORA-06512: at line 3

-- 再次通过dbv和rman来检查坏块情况，dbv检查结果是0坏块。rman 检查结果是ok的，检查视图v$database_block_corruption，结果是0 。

[oraprod@ebsr121 ~]$ dbv file=/d01/oracle/PROD/db/apps_st/data/system09.dbf

DBVERIFY: Release 11.1.0.7.0 - Production on Fri Apr 10 10:03:44 2020

Copyright (c) 1982, 2007, Oracle.  All rights reserved.

DBVERIFY - Verification starting : FILE = /d01/oracle/PROD/db/apps_st/data/system09.dbf


DBVERIFY - Verification complete

Total Pages Examined         : 195984
Total Pages Processed (Data) : 157818
Total Pages Failing   (Data) : 0
Total Pages Processed (Index): 30379
Total Pages Failing   (Index): 0
Total Pages Processed (Other): 1964
Total Pages Processed (Seg)  : 0
Total Pages Failing   (Seg)  : 0
Total Pages Empty            : 5823
Total Pages Marked Corrupt   : 0
Total Pages Influx           : 0
Total Pages Encrypted        : 0
Highest block SCN            : 3724398416 (1388.3724398416)
[oraprod@ebsr121 ~]$

[oraprod@ebsr121 ~]$ rman target /

Recovery Manager: Release 11.1.0.7.0 - Production on Fri Apr 10 09:59:20 2020

Copyright (c) 1982, 2007, Oracle.  All rights reserved.

connected to target database: TEST (DBID=2335061898)

RMAN> backup validate check logical datafile 352;

Starting backup at 10-APR-20
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=354 device type=DISK
allocated channel: ORA_DISK_2
channel ORA_DISK_2: SID=353 device type=DISK
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00352 name=/d01/oracle/PROD/db/apps_st/data/system09.dbf
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:55
List of Datafiles
=================
File Status Marked Corrupt Empty Blocks Blocks Examined High SCN
---- ------ -------------- ------------ --------------- ----------
352  OK     0              5823         195984          5965139005264
  File Name: /d01/oracle/PROD/db/apps_st/data/system09.dbf
  Block Type Blocks Failing Blocks Processed
  ---------- -------------- ----------------
  Data       0              157818          
  Index      0              30379           
  Other      0              1964            

Finished backup at 10-APR-20

RMAN>

SYS@TEST>select * from v$database_block_corruption;

no rows selected

SYS@TEST>

-- 清理，drop 掉表，切换日志和checkpoint后删除触发器。

drop table scott.s;
alter system switch logfile;
alter system switch logfile;
alter system checkpoint;
DROP trigger corrupt_trigger ;

END

RMAN-03009，ORA-19566 及坏块修复

OGG的DDL測試及CONVERTUCS2CLOBS參數

12c以上的版本，對錶進行在線分區（不使用在線重定義）

RMAN-03009，ORA-19566 及壞塊修復

一個schedule的小例子

Oracle EBS 更改主機名/web端口/監聽端口 -- 通過autoconfig

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結