Video Screencast Help
Give us your opinion and win with Symantec! Please help us by taking this survey to tell us about your experience with Symantec Connect, so that we can continue to grow and improve.  Take the survey.

Oracle RAC vote disk cannot be discover on 11.2.0.3 with Solaris 10u10 and SFRAC 6.0.1

Created: 24 Jul 2013 | 3 comments

Hi community,

My client met a problem that Oracle RAC vote disk cannot be discover which caused OS keeps rebooting frequently. Oracle has already reply there is a known issue on SFRAC 5.1. But my client is using Oracle RAC 11.2.0.3 with Solaris 10u10 and SFRAC 6.0.1.

Anyone has any clue about this ?

The Grid Alert log shows:

Oracle Grid Alert Log



2013-07-24 13:01:24.186

[ohasd(4929)]CRS-8011:reboot advisory message from host: xhdb-server3, component: cssmonit, with time stamp: L-2013-07-24-11:19:48.012

[ohasd(4929)]CRS-8013:reboot advisory message text: clsnvmon_main: error registering in skgxn rc 3

2013-07-24 13:01:24.210

[ohasd(4929)]CRS-8011:reboot advisory message from host: xhdb-server3, component: cssagent, with time stamp: L-2013-07-24-12:21:07.693

[ohasd(4929)]CRS-8013:reboot advisory message text: clsnvmon_main: error registering in skgxn rc 3

2013-07-24 13:01:24.211

[ohasd(4929)]CRS-8017:location: /var/opt/oracle/lastgasp has 2 reboot advisory log files, 2 were announced and 0 errors occurred

2013-07-24 13:01:43.850

[/u01/app/11.2.0/grid/bin/orarootagent.bin(5084)]CRS-5016:Process "/u01/app/11.2.0/grid/bin/acfsload" spawned by agent "/u01/app/11.2.0/grid/bin/orarootagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/app/11.2.0/grid/log/xhdb-server3/agent/ohasd/orarootagent_root/orarootagent_root.log"

2013-07-24 13:01:48.976

[ohasd(4929)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running). 

2013-07-24 13:01:49.139

[gpnpd(6273)]CRS-2328:GPNPD started on node xhdb-server3. 

2013-07-24 13:01:52.838

[cssd(6299)]CRS-1713:CSSD daemon is started in clustered mode

2013-07-24 13:01:57.792

[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log

2013-07-24 13:02:03.559

[ohasd(4929)]CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE

2013-07-24 13:02:12.944

[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log

2013-07-24 13:02:28.099

[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log

2013-07-24 13:02:43.248

[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log

2013-07-24 13:02:58.396

[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log

2013-07-24 13:03:13.544

[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log

2013-07-24 13:03:35.359

[cssd(6299)]CRS-1707:Lease acquisition for node xhdb-server3 number 1 completed

2013-07-24 13:03:47.951

[cssd(6299)]CRS-1605:CSSD voting file is online: /ocr2/vdsk; details in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log.

ocssd.log

2013-07-24 13:02:43.247: [ GPNP][6]clsgpnp_profileCallUrlInt: [at clsgpnp.c:2234] Result: (0) CLSGPNP_OK. Successful get-profile CALL to remote "ipc://GPNPD_xhdb-server3" disco ""

2013-07-24 13:02:43.248: [ CSSD][6]clssnmReadDiscoveryProfile: voting file discovery string(/ocr2/vdsk)

2013-07-24 13:02:43.248: [ CSSD][6]clssnmvDDiscThread: using discovery string /ocr2/vdsk for initial discovery 

2013-07-24 13:02:43.248: [ SKGFD][6]Discovery with str:/ocr2/vdsk:

2013-07-24 13:02:43.248: [ SKGFD][6]UFS discovery with :/ocr2/vdsk:

2013-07-24 13:02:43.248: [ SKGFD][6]OSS discovery with :/ocr2/vdsk:

2013-07-24 13:02:43.248: [ CSSD][6]clssnmvDiskVerify: Successful discovery of 0 disks

2013-07-24 13:02:43.248: [ CSSD][6]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery

2013-07-24 13:02:43.248: [ CSSD][6]clssnmvFindInitialConfigs: No voting files found

2013-07-24 13:02:43.249: [ CSSD][6](:CSSNM00070:)clssnmCompleteInitVFDiscovery: Voting file not found. Retrying discovery in 15 seconds

2013-07-24 13:02:43.635: [ CSSD][5]clssscSelect: cookie accept request 100881af0

2013-07-24 13:02:43.636: [ CSSD][5]clssgmAllocProc: (10115de50) allocated

2013-07-24 13:02:43.637: [ CSSD][5]clssgmClientConnectMsg: properties of cmProc 10115de50 - 1,2,3,4,5

2013-07-24 13:02:43.637: [ CSSD][5]clssgmClientConnectMsg: Connect from con(61d) proc(10115de50) pid(8485) version 11:2:1:4, properties: 1,2,3,4,5

2013-07-24 13:02:43.637: [ CSSD][5]clssgmClientConnectMsg: msg flags 0x0000

2013-07-24 13:02:45.166: [ CSSD][5]clssscSelect: cookie accept request 100d26190

2013-07-24 13:02:45.167: [ CSSD][5]clssscevtypSHRCON: getting client with cmproc 100d26190

2013-07-24 13:02:45.167: [ CSSD][5]clssgmRegisterClient: proc(4/100d26190), client(1/100a3b2d0)

2013-07-24 13:02:45.168: [ CSSD][5]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(100d26190) client(100a3b2d0)

2013-07-24 13:02:45.169: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 643

2013-07-24 13:02:46.175: [ CSSD][5]clssscSelect: cookie accept request 100d26190

2013-07-24 13:02:46.175: [ CSSD][5]clssscevtypSHRCON: getting client with cmproc 100d26190

2013-07-24 13:02:46.175: [ CSSD][5]clssgmRegisterClient: proc(4/100d26190), client(2/100a3b2d0)

2013-07-24 13:02:46.176: [ CSSD][5]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(100d26190) client(100a3b2d0)

2013-07-24 13:02:46.176: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 659

2013-07-24 13:02:47.183: [ CSSD][5]clssscSelect: cookie accept request 100d26190

2013-07-24 13:02:47.183: [ CSSD][5]clssscevtypSHRCON: getting client with cmproc 100d26190

2013-07-24 13:02:47.183: [ CSSD][5]clssgmRegisterClient: proc(4/100d26190), client(3/100a3b2d0)

2013-07-24 13:02:47.184: [ CSSD][5]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(100d26190) client(100a3b2d0)

2013-07-24 13:02:47.184: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 66f

2013-07-24 13:02:47.551: [ CSSD][5]clssgmExecuteClientRequest(): type(37) size(80) only connect and exit messages are allowed before lease acquisition proc(100d26790) client(0)

2013-07-24 13:02:47.555: [ CSSD][5]clssgmDeadProc: proc 100d26790

2013-07-24 13:02:47.555: [ CSSD][5]clssgmDestroyProc: cleaning up proc(100d26790) con(59f) skgpid ospid 6284 with 0 clients, refcount 0

2013-07-24 13:02:47.555: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 59f

2013-07-24 13:02:47.576: [ CSSD][5]clssscSelect: cookie accept request 100881af0

2013-07-24 13:02:47.576: [ CSSD][5]clssgmAllocProc: (100d26610) allocated

2013-07-24 13:02:47.577: [ CSSD][5]clssgmClientConnectMsg: properties of cmProc 100d26610 - 1,2,3,4,5

2013-07-24 13:02:47.577: [ CSSD][5]clssgmClientConnectMsg: Connect from con(6c0) proc(100d26610) pid(6284) version 11:2:1:4, properties: 1,2,3,4,5

2013-07-24 13:02:47.577: [ CSSD][5]clssgmClientConnectMsg: msg flags 0x0000

2013-07-24 13:02:48.190: [ CSSD][5]clssscSelect: cookie accept request 100d26190

2013-07-24 13:02:48.191: [ CSSD][5]clssscevtypSHRCON: getting client with cmproc 100d26190

2013-07-24 13:02:48.191: [ CSSD][5]clssgmRegisterClient: proc(4/100d26190), client(4/100a3b2d0)

2013-07-24 13:02:48.192: [ CSSD][5]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(100d26190) client(100a3b2d0)

2013-07-24 13:02:48.192: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 6e6

2013-07-24 13:02:48.627: [ CSSD][5]clssscSelect: cookie accept request 10115de50

2013-07-24 13:02:48.627: [ CSSD][5]clssscevtypSHRCON: getting client with cmproc 10115de50

2013-07-24 13:02:48.627: [ CSSD][5]clssgmRegisterClient: proc(5/10115de50), client(1/100a3b2d0)

2013-07-24 13:02:48.628: [ CSSD][5]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(10115de50) client(100a3b2d0)

2013-07-24 13:02:48.629: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 6fc

Any thoughs could be helpful is appreciate !

Operating Systems:

Comments 3 CommentsJump to latest comment

stinsong's picture

Ooh, foget to tell that my client has already confirmed about the permission of vote volume and disks is right.

rsharma1's picture

Hi Stinsong,

                                 from the msg "error registering in skgxn rc 3 2013-07-24 13:01:24.211"

seems like some issue with node membership communication between VCS and Oracle grid. Could you check if the vcsmm port - port "o" is showing up as joined in gabconfig -a o/p for all nodes? If not, probably starting vcsmm manully might help..

gaurav_dong's picture

"gabconfig -a" from all the nodes will be helpful.

is the vote disk a veritas volume manager volume ?

Also the main.cf file if you can.

Gaurav D