Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

SCSI reservation on data disk

Created: 22 Jul 2010 • Updated: 23 Aug 2010 | 7 comments
This issue has been solved. See solution.

Hello All,

I have installed VCS HA clusterware on solaris 10 sparc arch. I have configured vxfen in SCSI3 mode.
What I observe is that all the disks which are part of co-ordinated DG have SCSI3 reservation but my data
disk doesn't have any reservation. My apprehension is that if VCS nodes doesnt do reservation how VCS is
going to protect the disk from outside world, in fact I tried to write on my data disk from a linux box which is not
the part of the cluster?

I am wondering if I have done the configuration correct?

Please find some of command output below which might help to analyze if there is some
configuration mistake:
===============
bash-3.00# vxfenadm -d

I/O Fencing Cluster Information:
================================

 Fencing Protocol Version: 201
 Fencing Mode: SCSI3
 Fencing SCSI3 Disk Policy: raw
 Cluster Members:

          0 (Solaris146)
        * 1 (Solaris148)

 RFSM State Information:
        node   0 in state  8 (running)
        node   1 in state  8 (running)

bash-3.00# hagrp -state
#Group       Attribute             System     Value
groupw       State                 Solaris146 |OFFLINE|
groupw       State                 Solaris148 |ONLINE|
bash-3.00# hares  -state
#Resource    Attribute           System     Value
fodg         State               Solaris146 OFFLINE
fodg         State               Solaris148 ONLINE
fofs         State               Solaris146 OFFLINE
fofs         State               Solaris148 ONLINE
bash-3.00#

============


 

Comments 7 CommentsJump to latest comment

Gaurav Sangamnerkar's picture

Hello Ajit,

In Veritas IOFencing, coordinator disks get only registration keys however data disks will have both registration keys & reservation keys...

To check Registration keys..

# /sbin/vxfenadm -g  /dev/rdsk/<cxtxdx>

To check reservation keys..

# /sbin/vxfenadm -r  /dev/rdsk/<cxtxdx>

Since you have configured VCS as well, did the diskgroups were imported by VCS ?  because VCS will import the diskgroups with reservations... then only you will see reservation keys...

Gaurav

PS: If you are happy with the answer provided, please mark the post as solution. You can do so by clicking link "Mark as Solution" below the answer provided.
 

Gaurav Sangamnerkar's picture

Also, just running of vxfen module doesn't indicate that fencing is active....

Does your main.cf had an entry "UseFence = SCSI3" ? If yes, then only cluster would be using IOfencing...

As you mentioned you tried writing via a linux box .... how did it go ? did it succeed ? Only that node can write to the disk who has registration keys & wrtie-exclusive reservation key ... your linux box should get an error while attempting write...

Gaurav

PS: If you are happy with the answer provided, please mark the post as solution. You can do so by clicking link "Mark as Solution" below the answer provided.
 

ajit jain's picture

Hi Gaurav,

Thanks for your quick reply.

As you said I could see the registration as well as reservation once VCS imported the disk group.
But at a time I only see one registration (from the node where DG is currently online) on the disk while I was expecting two registration keys as it is two node cluster.

One more question as we started seeing SCSI keys after VCS imported VxVM diskgroup does that means if I were to create a VCS HA file-system on a raw disk then the disk would not be protected by VCS?

thanks again,
ajit jain

ajit jain's picture

In reply to Gaurav's previous message:-

Yes, in main.cf I have added entry "UseFence = SCSI3", I believe VCS will not start if there is discrepency in main.cf and vxfen.
From linux box it succeeded at the first time because VCS has not imported the diskgroup but later it failed.
But I dont see any reservation or registration for the raw disk (without VXVM diskgroup) which are part of a hares of Mount type.

thanks,
ajit

Dev Roy's picture

Ajit,

The essential components of VCS I/O fencing are as follows:

1.> LLT, GAB, and VXFEN drivers
2.> Coordinator disks with SCSI-3 Persistent Reservations (PR)
3.> Veritas Volume Manager (VxVM) and Dynamic Multipathing (DMP)
4.> VCS I/O fencing data disks using CVM/DMP and VCS Disk Group agent

As you could see in point number 3 that volume manager is required, that means that it assumes that disks / luns / arrays are being managed by volume manager. For any other disk (without VXVM diskgroup), you would not see fencing keys in it.

Hope this helps.

Regards,
Dev

Consulting Storage Foundation, VCS, VVR and CVM/CFS on Unix. If this post has helped you, please "Vote" or "Mark as Solution" as appropriate.

Gaurav Sangamnerkar's picture

Hi Ajit,

To answer your questions......

Once the DG is imported by VCS, you will see the registraiton & reservation of only 1 node (in case of a failover diskgroup) ... However if its a Shared diskgroup (CVM diskgroup), then you would see registration & reservation of both nodes....

Yes you are right, VCS diskgroup agent imports the diskgroup with specific options ( -o groupreserve) then only reservations are placed.... If you have a normal filesystem where diskgroup was not imported with above options, then it is NOT protected by VCS IOFencing..... however you can configure that FS in VCS ... so basically you just loose the split brain protection (which is real purpose of fencing)....

Gaurav

PS: If you are happy with the answer provided, please mark the post as solution. You can do so by clicking link "Mark as Solution" below the answer provided.
 

SOLUTION
ajit jain's picture

Hi Dev/Gaurav,
All your messages really helped.

thanks a lot.
ajit jain