Video Screencast Help

Oracle RAC vote disk cannot be discover on 11.2.0.3 with Solaris 10u10 and SFRAC 6.0.1

Created: 24 Jul 2013 | 3 comments

Hi community,

My client met a problem that Oracle RAC vote disk cannot be discover which caused OS keeps rebooting frequently. Oracle has already reply there is a known issue on SFRAC 5.1. But my client is using Oracle RAC 11.2.0.3 with Solaris 10u10 and SFRAC 6.0.1.

Anyone has any clue about this ?

The Grid Alert log shows:

Oracle Grid Alert Log

2013-07-24 13:01:24.186
[ohasd(4929)]CRS-8011:reboot advisory message from host: xhdb-server3, component: cssmonit, with time stamp: L-2013-07-24-11:19:48.012
[ohasd(4929)]CRS-8013:reboot advisory message text: clsnvmon_main: error registering in skgxn rc 3
2013-07-24 13:01:24.210
[ohasd(4929)]CRS-8011:reboot advisory message from host: xhdb-server3, component: cssagent, with time stamp: L-2013-07-24-12:21:07.693
[ohasd(4929)]CRS-8013:reboot advisory message text: clsnvmon_main: error registering in skgxn rc 3
2013-07-24 13:01:24.211
[ohasd(4929)]CRS-8017:location: /var/opt/oracle/lastgasp has 2 reboot advisory log files, 2 were announced and 0 errors occurred
2013-07-24 13:01:43.850
[/u01/app/11.2.0/grid/bin/orarootagent.bin(5084)]CRS-5016:Process "/u01/app/11.2.0/grid/bin/acfsload" spawned by agent "/u01/app/11.2.0/grid/bin/orarootagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/u01/app/11.2.0/grid/log/xhdb-server3/agent/ohasd/orarootagent_root/orarootagent_root.log"
2013-07-24 13:01:48.976
[ohasd(4929)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running). 
2013-07-24 13:01:49.139
[gpnpd(6273)]CRS-2328:GPNPD started on node xhdb-server3. 
2013-07-24 13:01:52.838
[cssd(6299)]CRS-1713:CSSD daemon is started in clustered mode
2013-07-24 13:01:57.792
[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log
2013-07-24 13:02:03.559
[ohasd(4929)]CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE
2013-07-24 13:02:12.944
[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log
2013-07-24 13:02:28.099
[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log
2013-07-24 13:02:43.248
[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log
2013-07-24 13:02:58.396
[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log
2013-07-24 13:03:13.544
[cssd(6299)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log
2013-07-24 13:03:35.359
[cssd(6299)]CRS-1707:Lease acquisition for node xhdb-server3 number 1 completed
2013-07-24 13:03:47.951
[cssd(6299)]CRS-1605:CSSD voting file is online: /ocr2/vdsk; details in /u01/app/11.2.0/grid/log/xhdb-server3/cssd/ocssd.log.
ocssd.log
2013-07-24 13:02:43.247: [ GPNP][6]clsgpnp_profileCallUrlInt: [at clsgpnp.c:2234] Result: (0) CLSGPNP_OK. Successful get-profile CALL to remote "ipc://GPNPD_xhdb-server3" disco ""
2013-07-24 13:02:43.248: [ CSSD][6]clssnmReadDiscoveryProfile: voting file discovery string(/ocr2/vdsk)
2013-07-24 13:02:43.248: [ CSSD][6]clssnmvDDiscThread: using discovery string /ocr2/vdsk for initial discovery 
2013-07-24 13:02:43.248: [ SKGFD][6]Discovery with str:/ocr2/vdsk:
2013-07-24 13:02:43.248: [ SKGFD][6]UFS discovery with :/ocr2/vdsk:
2013-07-24 13:02:43.248: [ SKGFD][6]OSS discovery with :/ocr2/vdsk:
2013-07-24 13:02:43.248: [ CSSD][6]clssnmvDiskVerify: Successful discovery of 0 disks
2013-07-24 13:02:43.248: [ CSSD][6]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery
2013-07-24 13:02:43.248: [ CSSD][6]clssnmvFindInitialConfigs: No voting files found
2013-07-24 13:02:43.249: [ CSSD][6](:CSSNM00070:)clssnmCompleteInitVFDiscovery: Voting file not found. Retrying discovery in 15 seconds
2013-07-24 13:02:43.635: [ CSSD][5]clssscSelect: cookie accept request 100881af0
2013-07-24 13:02:43.636: [ CSSD][5]clssgmAllocProc: (10115de50) allocated
2013-07-24 13:02:43.637: [ CSSD][5]clssgmClientConnectMsg: properties of cmProc 10115de50 - 1,2,3,4,5
2013-07-24 13:02:43.637: [ CSSD][5]clssgmClientConnectMsg: Connect from con(61d) proc(10115de50) pid(8485) version 11:2:1:4, properties: 1,2,3,4,5
2013-07-24 13:02:43.637: [ CSSD][5]clssgmClientConnectMsg: msg flags 0x0000
2013-07-24 13:02:45.166: [ CSSD][5]clssscSelect: cookie accept request 100d26190
2013-07-24 13:02:45.167: [ CSSD][5]clssscevtypSHRCON: getting client with cmproc 100d26190
2013-07-24 13:02:45.167: [ CSSD][5]clssgmRegisterClient: proc(4/100d26190), client(1/100a3b2d0)
2013-07-24 13:02:45.168: [ CSSD][5]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(100d26190) client(100a3b2d0)
2013-07-24 13:02:45.169: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 643
2013-07-24 13:02:46.175: [ CSSD][5]clssscSelect: cookie accept request 100d26190
2013-07-24 13:02:46.175: [ CSSD][5]clssscevtypSHRCON: getting client with cmproc 100d26190
2013-07-24 13:02:46.175: [ CSSD][5]clssgmRegisterClient: proc(4/100d26190), client(2/100a3b2d0)
2013-07-24 13:02:46.176: [ CSSD][5]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(100d26190) client(100a3b2d0)
2013-07-24 13:02:46.176: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 659
2013-07-24 13:02:47.183: [ CSSD][5]clssscSelect: cookie accept request 100d26190
2013-07-24 13:02:47.183: [ CSSD][5]clssscevtypSHRCON: getting client with cmproc 100d26190
2013-07-24 13:02:47.183: [ CSSD][5]clssgmRegisterClient: proc(4/100d26190), client(3/100a3b2d0)
2013-07-24 13:02:47.184: [ CSSD][5]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(100d26190) client(100a3b2d0)
2013-07-24 13:02:47.184: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 66f
2013-07-24 13:02:47.551: [ CSSD][5]clssgmExecuteClientRequest(): type(37) size(80) only connect and exit messages are allowed before lease acquisition proc(100d26790) client(0)
2013-07-24 13:02:47.555: [ CSSD][5]clssgmDeadProc: proc 100d26790
2013-07-24 13:02:47.555: [ CSSD][5]clssgmDestroyProc: cleaning up proc(100d26790) con(59f) skgpid ospid 6284 with 0 clients, refcount 0
2013-07-24 13:02:47.555: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 59f
2013-07-24 13:02:47.576: [ CSSD][5]clssscSelect: cookie accept request 100881af0
2013-07-24 13:02:47.576: [ CSSD][5]clssgmAllocProc: (100d26610) allocated
2013-07-24 13:02:47.577: [ CSSD][5]clssgmClientConnectMsg: properties of cmProc 100d26610 - 1,2,3,4,5
2013-07-24 13:02:47.577: [ CSSD][5]clssgmClientConnectMsg: Connect from con(6c0) proc(100d26610) pid(6284) version 11:2:1:4, properties: 1,2,3,4,5
2013-07-24 13:02:47.577: [ CSSD][5]clssgmClientConnectMsg: msg flags 0x0000
2013-07-24 13:02:48.190: [ CSSD][5]clssscSelect: cookie accept request 100d26190
2013-07-24 13:02:48.191: [ CSSD][5]clssscevtypSHRCON: getting client with cmproc 100d26190
2013-07-24 13:02:48.191: [ CSSD][5]clssgmRegisterClient: proc(4/100d26190), client(4/100a3b2d0)
2013-07-24 13:02:48.192: [ CSSD][5]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(100d26190) client(100a3b2d0)
2013-07-24 13:02:48.192: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 6e6
2013-07-24 13:02:48.627: [ CSSD][5]clssscSelect: cookie accept request 10115de50
2013-07-24 13:02:48.627: [ CSSD][5]clssscevtypSHRCON: getting client with cmproc 10115de50
2013-07-24 13:02:48.627: [ CSSD][5]clssgmRegisterClient: proc(5/10115de50), client(1/100a3b2d0)
2013-07-24 13:02:48.628: [ CSSD][5]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are allowed before lease acquisition proc(10115de50) client(100a3b2d0)
2013-07-24 13:02:48.629: [ CSSD][5]clssgmDiscEndpcl: gipcDestroy 6fc

 

Any thoughs could be helpful is appreciate !

Operating Systems:

Comments 3 CommentsJump to latest comment

stinsong's picture

Ooh, foget to tell that my client has already confirmed about the permission of vote volume and disks is right.

rsharma1's picture

Hi Stinsong,

                                 from the msg "error registering in skgxn rc 3 2013-07-24 13:01:24.211"

seems like some issue with node membership communication between VCS and Oracle grid. Could you check if the vcsmm port - port "o" is showing up as joined in gabconfig -a o/p for all nodes? If not, probably starting vcsmm manully might help..

 

                      

gaurav_dong's picture

"gabconfig -a" from all the nodes will be helpful.

is the vote disk a veritas volume manager volume ?

Also the main.cf file if you can.

 

Gaurav D