standby database recovery fails with ORA-17627 ORA-17629

刘泽宇

现象：
Standby database recovery fails with these errors:

2020-03-17T18:24:41.713017-04:00  Errors in file /path/<sid>_pr0o_56438.trc:
2020-03-17T18:24:41.713017-04:00  ORA-17627:
2020-03-17T18:24:41.713017-04:00  ORA-17629: Cannot connect to the remote database server


原因：
The standby database is used in Active Data Guard (ADG) mode.

Oracle attempts to use Automatic Block Recover Feature (AMBR) to fix a corrupted block

2020-03-17T17:02:03.893663-04:00  Errors in file /path/trace/<SID>_w005_5726.trc (incident=8458):
2020-03-17T17:02:03.893663-04:00  ORA-01578: ORACLE data block corrupted (file # 4, block # 52160)
2020-03-17T17:02:03.893663-04:00  ORA-01110: data file 4: '<Path and name of datafile>'
2020-03-17T17:04:59.996674-04:00  Errors in file /path/trace/<SID>_tt00_60916.trc:
2020-03-17T17:04:59.996674-04:00  ORA-19815: WARNING: db_recovery_file_dest_size of 2147483648000 bytes is 100.00% used, and has 0 remaining bytes available.



Trace file shows
-------------------

kgfpRubout hdl=0x7fc568ead7c8

OCI error val is 291576348 and errmsg is ''
ORA-17627:
ORA-17629: Cannot connect to the remote database server
SUCCESS: retrieved DB password file location: <path>
kgfpInitEnv2: hdl=0x7fc568ead7c8 env=0x7fc568ead7e0
2020-03-17 18:29:48.059 : [ default][ default]clsvactversion:4: Retrieving Active Version from local storage.

解决方法：
There are several options that could be used to resolve the issue


Option 1 :  Using Service Option in 12c and above


**

** Stop managed recovery on the standby database using :

**

SQL > recover managed standby database cancel

  -- or, in case DG Broker is configured

DGMGRL > edit database "<standby database name>" set state = 'APPLY-OFF';

**

** Restore the data file

**

rman target /

RMAN> shutdown immediate;

RMAN> startup mount ;

RMAN> restore datafile 4 from service '<service name of primary>' ;



**

** Start managed recovery on the standby database using :

**

SQL > recover managed standby database disconnect;

-- or, in case DG Broker is configured

DGMGRL > edit database "<standby database name>" set state = 'APPLY-ON';





Option 2 : Using Backupset from primary



Step 1: On Primary :

RMAN> backup as compressed backupset datafile 4 format '<path/%U>' tag 'corrupt' ;
Step 2: scp this backuppiece to standby

Step 3:On Standby
----------------------

**

** Stop managed recovery on the standby database using :

**

SQL > recover managed standby database cancel

-- or, in case DG Broker is configured

DGMGRL > edit database "<standby database name>" set state = 'APPLY-OFF';

**

** Restore the copied data file

**

RMAN> catalog backuppiece '<name and path>' ;

RMAN> List backup of datafile 4 ;

RMAN> restore datafile 4 ;

RMAN> validate datafile 4 ;

RMAN> select * from v$database_block_corruption ;

**

** Start managed recovery on the standby database using :

**

SQL > recover managed standby database disconnect;

-- or, in case DG Broker is configured

DGMGRL > edit database "<standby database name>" set state = 'APPLY-ON';





Option 3 : Disable AMBR on standby



You can try disabling AMBR and see if managed recovery continues however the block would still be Corrupted

Alter system set "_auto_bmr"=disabled scope=spfile ;

Then restart the Standby database instance

SQL > shutdown immediate;

SQL > startup mount;

SQL > recover managed standby database cancel

-- or, in case DG Broker is configured

DGMGRL > edit database "<standby database name>" set state = 'APPLY-OFF';