ORACLE 11.2.0.3 asm BUG

lihui

Last login: Tue Nov 13 04:03:36 2012 from 192.168.208.1

[root@oel63 ~]# su - oracle

[oracle@oel63 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Tue Nov 13 12:25:37 2012

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to an idle instance.

idle>startup

ORA-01078: failure in processing system parameters

ORA-01565: error in identifying file '+TESTDB2_DBAT/ractest1/spfileractest1.ora'

ORA-17503: ksfdopn:2 Failed to open file +TESTDB2_DBAT/ractest1/spfileractest1.ora

ORA-27140: attach to post/wait facility failed

ORA-27300: OS system dependent operation:invalid_egid failed with status: 1

ORA-27301: OS failure message: Operation not permitted

ORA-27302: failure occurred at: skgpwinit6

ORA-27303: additional information: startup egid = 54321 (oinstall), current egid = 504 (asmadmin)

idle>exit

Disconnected

[oracle@oel63 ~]$ exit

logout

[root@oel63 ~]# cd /etc/init.d

[root@oel63 init.d]# ls

abrt-ccpp     crond         ipmi             messagebus         oddjobd                                         PowerPath    saslauthd       udev-post

abrtd         cups          ipmidetectd      mysqld             ohasd                                           psacct       sblim-sfcb      uuidd

abrt-oops     dnsmasq       ipmievd          netcf-transaction  openct                                          qpidd        single          vncserver

acpid         edac          ipsec            netconsole         openhpid                                        quota_nld    smartd          vsftpd

atd           emcp_mond.sh  iptables         netfs              openhpi-subagent                                rdisc        snmpd           watchdog

auditd        firstboot     irqbalance       network            oracleasm                                       restorecond  snmptrapd       wdaemon

autofs        functions     kdump            NetworkManager     oracle-rdbms-server-11gR2-preinstall-firstboot  rhnsd        spice-vdagentd  wpa_supplicant

avahi-daemon  gpm           killall          nfs                pcscd                                           rngd         sshd            xinetd

bmc-watchdog  haldaemon     libvirt-guests   nfslock            portreserve                                     rpcbind      sssd            ypbind

certmonger    halt          lvm2-lvmetad     ntpd               postfix                                         rpcgssd      stap-server

cgconfig      hsqldb        lvm2-monitor     ntpdate            PowerFcoe                                       rpcidmapd    svnserve

cgred         httpd         matahari-broker  numad              PowerIscsi                                      rpcsvcgssd   sysstat

conman        init.ohasd    mcelogd          o2cb               PowerMig                                        rsyslog      tcsd

cpuspeed      ip6tables     mdmonitor        ocfs2              PowerMigRecoverAll                              sandbox      tog-pegasus

[root@oel63 init.d]# ./oracleasm listdisks

VOL1

VOL2

VOL3

VOL4

VOL5

VOL6

VOL7

[root@oel63 init.d]# ./oracleasm scandisk

Usage: ./oracleasm {start|stop|restart|enable|disable|configure|createdisk|deletedisk|querydisk|listdisks|scandisks|status}

[root@oel63 init.d]# ./oracleasm scandisks

Scanning the system for Oracle ASMLib disks: [  OK  ]

[root@oel63 init.d]# su - oracle

[oracle@oel63 ~]$ su - oracle

ePassword: 

su: incorrect password

[oracle@oel63 ~]$ 

[oracle@oel63 ~]$ 

[oracle@oel63 ~]$ 

[oracle@oel63 ~]$ ^C

[oracle@oel63 ~]$ 

[oracle@oel63 ~]$ 

[oracle@oel63 ~]$ export ORACLE_SID=ractest1

[oracle@oel63 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Tue Nov 13 12:30:49 2012

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to an idle instance.

idle>startup

ORA-01078: failure in processing system parameters

ORA-01565: error in identifying file '+TESTDB2_DBAT/ractest1/spfileractest1.ora'

ORA-17503: ksfdopn:2 Failed to open file +TESTDB2_DBAT/ractest1/spfileractest1.ora

ORA-27140: attach to post/wait facility failed

ORA-27300: OS system dependent operation:invalid_egid failed with status: 1

ORA-27301: OS failure message: Operation not permitted

ORA-27302: failure occurred at: skgpwinit6

ORA-27303: additional information: startup egid = 54321 (oinstall), current egid = 504 (asmadmin)

idle>exit

Disconnected

[oracle@oel63 ~]$ cd 

[oracle@oel63 ~]$ cd $ORACLE_HOE

[oracle@oel63 ~]$ cd $ORACLE_HOME

[oracle@oel63 dbhome_1]$ pd

-bash: pd: command not found

[oracle@oel63 dbhome_1]$ pwd

/u01/app/oracle/product/11.2.0/dbhome_1

[oracle@oel63 dbhome_1]$ ls

apex/         clone/   cv/           dv/       install/        jdev/  log/      oc4j/     oraInst.loc*  plsql/     scheduler/     sysman/           wwg/

assistants/   config/  dbs/          emcli/    instantclient/  jdk/   md/       odbc/     ord/          precomp/   slax/          timingframework/  xdk/

bin/          crs/     dc_ocm/       EMStage/  inventory/      jlib/  mesg/     olap/     oui/          racg/      sqldeveloper/  ucp/

ccr/          csmig/   deinstall/    has/      j2ee/           jpub/  mgw/      OPatch/   owb/          rdbms/     sqlj/          uix/

cdata/        css/     demo/         hs/       javavm/         ldap/  network/  opmn/     owm/          relnotes/  sqlplus/       usm/

cfgtoollogs/  ctx/     diagnostics/  ide/      jdbc/           lib/   nls/      oracore/  perl/         root.sh*   srvm/          utl/

[oracle@oel63 dbhome_1]$ cd dbs

[oracle@oel63 dbs]$ ls

hc_ractest1.dat*  init.ora*  initractest1.ora*  install2012-09-13_06-59-26.log*  lkRACTEST1*  LOG_cat_owb.TXT*  orapwractest1

[oracle@oel63 dbs]$ more initractest1.ora 

SPFILE='+TESTDB2_DBAT/ractest1/spfileractest1.ora'

[oracle@oel63 dbs]$ more '+TESTDB2_DBAT/ractest1/spfileractest1.ora'

usage: more [-dflpcsu] [+linenum | +/pattern] name1 name2 ...

[oracle@oel63 dbs]$ id

uid=501(oracle) gid=54321(oinstall) groups=54321(oinstall),503(oper),504(asmadmin),505(asmdba),506(asmoper),54322(dba)

[oracle@oel63 dbs]$ export ORACLE_SID=+ASM1

[oracle@oel63 dbs]$ asmcmd

lihui

解决方法：

SRVCTL Fails to Start Instance with ORA-17503 ORA-27303 But sqlplus Startup is Fine [ID 1322959.1]
		修改时间 23-JAN-2012 类型 PROBLEM 状态 PUBLISHED

In this Document
Symptoms
Cause
Solution
References

---

Applies to:

Oracle Server - Enterprise Edition - Version: 11.2.0.1 and later [Release: 11.2 and later ]
Information in this document applies to any platform.

Symptoms

RAC Instance can be started using sqlplus but fails when using srvctl:

Case I:

$ srvctl start database -d cad
PRCR-1079 : Failed to start resource ora.cad.db
CRS-5017: The resource action "ora.cad.db start" encountered the following error:
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+RAC/cad/spfilecad.ora'
ORA-17503: ksfdopn:2 Failed to open file +RAC/cad/spfilecad.ora
ORA-27140: attach to post/wait facility failed
ORA-27300: OS system dependent operation:invalid_egid failed with status: 1
ORA-27301: OS failure message: Operation not permitted
ORA-27302: failure occurred at: skgpwinit6
ORA-27303: additional information: startup egid = 1002 (oinstall), current egid = 1001 (dba)

CRS-2674: Start of 'ora.cad.db' on 'racnode2' failed
CRS-2632: There are no more servers to try to place resource 'ora.cad.db' on that would satisfy its placement policy

Case II

$ srvctl start database -d nar
PRCR-1079 : Failed to start resource ora.nar.db
CRS-5017: The resource action "ora.nar.db start" encountered the following error:
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+NAR_DG/nar/spfilenar.ora'
ORA-17503: ksfdopn:2 Failed to open file +NAR_DG/nar/spfilenar.ora
ORA-27140: attach to post/wait facility failed
ORA-27300: OS system dependent operation:invalid_egid failed with status: 1
ORA-27301: OS failure message: Operation not permitted
ORA-27302: failure occurred at: skgpwinit6
ORA-27303: additional information: startup egid = 59800 (oinstall), current egid = 59802 (asmadmin)
. For details refer to "(:CLSN00107:)" in "/oracle/GRID/11203/log/dog1104/agent/crsd/oraagent_oracle/oraagent_oracle.log".

CRS-2674: Start of 'ora.nar.db' on 'racnode1' failed

Cause

For Case I, the issue is caused wrong group (dba) is chosen for "ASM Database Administrator","ASM Instance Administration Operator" and "ASM Instance Administrator"
group during installing grid infrastructure, while the ASM disks are owned by oinstall group.

This same issue is documented in
Bug 9786198 - SRVCTL START DATABASE ORA-0178 FAILURE IN PROCESSING SYSTEM PARAMETERS

For Case II, the issue is caused by setuid and setgid bit missing for oracle binary under GRID_HOME, eg:

$ ls -l $GRID_HOME/bin/oracle
-rwxr-x--x oracle oinstall 166648778 Dec 7 15:26 oracle

The missing setuid and setgid bit causing oracle user access failure to ASM diskgroup even though the ASMADMIN group is set correctly.

Solution

For case I,

Either reinstall Grid Infrastructure, select correct group for ASM Instance Administrator - ASMADMIN (oinstall in this case which is the ASM disk owning group)
or
stop srvctl modify oracle binary group from oinstall to dba by renaming $GRID_HOME/bin/setasmgid to $GRID_HOME/bin/setasmgid.orig. Restore the group ownership of oracle binary to oinstall and startup instance using srvctl.

For case II,

Add setuid and setgid bit for oracle binary under $GRID_HOME/bin:

$ chmod 6755 $GRID_HOME/bin/oracle


The oracle binary for both GRID_HOME/bin and RDBMS ORACLE_HOME/bin should have 6755 permission, eg: -rwsr-s--x

在本环境中具体执行如下：

[oracle@oel63 ~]$ chmod 6755 $GRID_HOME/bin/oracle

[oracle@oel63 dbs]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Tue Nov 13 12:57:00 2012

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to an idle instance.

idle>startup

ORACLE instance started.

Total System Global Area 1048059904 bytes

Fixed Size                  2235000 bytes

Variable Size             650118536 bytes

Database Buffers          390070272 bytes

Redo Buffers                5636096 bytes

Database mounted.

Database opened.

idle>Last login: Tue Nov 13 04:03:36 2012 from 192.168.208.1

lihui

SRVCTL Fails to Start Instance with ORA-17503 ORA-27303 But sqlplus Startup is Fine [ID 1322959.1]

To Bottom

---

Modified:29-Aug-2012Type:PROBLEMStatus:PUBLISHEDPriority:3

Comments (0)

In this Document

Symptoms

Cause

Solution

References

---

Applies to:

Oracle Server - Enterprise Edition - Version 11.2.0.1 and later
Information in this document applies to any platform.

Symptoms

RAC Instance can be started using sqlplus but fails when using srvctl:

Case I:

$ srvctl start database -d cad
PRCR-1079 : Failed to start resource ora.cad.db
CRS-5017: The resource action "ora.cad.db start" encountered the following error:
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+RAC/cad/spfilecad.ora'
ORA-17503: ksfdopn:2 Failed to open file +RAC/cad/spfilecad.ora
ORA-27140: attach to post/wait facility failed
ORA-27300: OS system dependent operation:invalid_egid failed with status: 1
ORA-27301: OS failure message: Operation not permitted
ORA-27302: failure occurred at: skgpwinit6
ORA-27303: additional information: startup egid = 1002 (oinstall), current egid = 1001 (dba)

CRS-2674: Start of 'ora.cad.db' on 'racnode2' failed
CRS-2632: There are no more servers to try to place resource 'ora.cad.db' on that would satisfy its placement policy

Case II

$ srvctl start database -d nar
PRCR-1079 : Failed to start resource ora.nar.db
CRS-5017: The resource action "ora.nar.db start" encountered the following error:
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+NAR_DG/nar/spfilenar.ora'
ORA-17503: ksfdopn:2 Failed to open file +NAR_DG/nar/spfilenar.ora
ORA-27140: attach to post/wait facility failed
ORA-27300: OS system dependent operation:invalid_egid failed with status: 1
ORA-27301: OS failure message: Operation not permitted
ORA-27302: failure occurred at: skgpwinit6
ORA-27303: additional information: startup egid = 59800 (oinstall), current egid = 59802 (asmadmin)
. For details refer to "(:CLSN00107:)" in "/oracle/GRID/11203/log/dog1104/agent/crsd/oraagent_oracle/oraagent_oracle.log".

CRS-2674: Start of 'ora.nar.db' on 'racnode1' failed

Cause

For Case I, the issue is caused wrong group (dba) is chosen for "ASM Database Administrator","ASM Instance Administration Operator" and "ASM Instance Administrator"
group during installing grid infrastructure, while the ASM disks are owned by oinstall group.

This same issue is documented in
Bug 9786198 - SRVCTL START DATABASE ORA-0178 FAILURE IN PROCESSING SYSTEM PARAMETERS

For Case II, the issue is caused by setuid and setgid bit missing for oracle binary under GRID_HOME, eg:

$ ls -l $GRID_HOME/bin/oracle
-rwxr-x--x oracle oinstall 166648778 Dec 7 15:26 oracle

The missing setuid and setgid bit causing oracle user access failure to ASM diskgroup even though the ASMADMIN group is set correctly.

Solution

For case I,

Either reinstall Grid Infrastructure, select correct group for ASM Instance Administrator - ASMADMIN (oinstall in this case which is the ASM disk owning group)
or
stop srvctl modify oracle binary group from oinstall to dba by renaming $GRID_HOME/bin/setasmgid to $GRID_HOME/bin/setasmgid.orig. Restore the group ownership of oracle binary to oinstall and startup instance using srvctl.

For case II,

Add setuid and setgid bit for oracle binary under $GRID_HOME/bin:

$ chmod 6751 $GRID_HOME/bin/oracle

The oracle binary for both GRID_HOME/bin and RDBMS ORACLE_HOME/bin should have 6751 permission, eg: -rwsr-s--x

还是最美

登录/注册后可看大图