Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

diskgroup resource failed to online in a VCS environment with "failed to import with groupreserve option as scsi3pr is disabled"

Created: 27 Jun 2013 • Updated: 27 Jun 2013 | 7 comments
ddh's picture
This issue has been solved. See solution.

Hi,

I have a two node cluster. I am trying to create a simple service group with VxFS on VxVM. I have followed the steps outlined in http://www.symantec.com/business/support/index?page=content&id=TECH20539

When I try to bring the service group online, the diskgroup would fail to import with "groupreserve option as scsi3pr is disabled". I've checked the diskgroup for scsi3-pr compliance with the vxfentsthdw utility. It is passing as shown below.

Can you please tell me how can I enable scsi3pr on the diskgroup so it can online successfully.

Thanks,

David

################### Diskgroup  Service online Error ####################################

Jun 27 15:56:36 sca-m532-504-gd9 Had[3429]: [ID 702911 daemon.notice] VCS ERROR V-16-10001-1095 (sca-m532-504-gd10) DiskGroup:data_dg:online:Diskgroup [datadg] failed to import with groupreserve option as scsi3pr is disabled.
Jun 27 15:58:37 sca-m532-504-gd9 Had[3429]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13066 (sca-m532-504-gd10) Agent is calling clean for resource(data_dg) because the resource is not up even after online completed.
Jun 27 15:58:39 sca-m532-504-gd9 Had[3429]: [ID 702911 daemon.notice] VCS ERROR V-16-10001-1095 (sca-m532-504-gd10) DiskGroup:data_dg:online:Diskgroup [datadg] failed to import with groupreserve option as scsi3pr is disabled.
Jun 27 16:00:40 sca-m532-504-gd9 Had[3429]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13066 (sca-m532-504-gd10) Agent is calling clean for resource(data_dg) because the resource is not up even after online completed.
Jun 27 16:00:42 sca-m532-504-gd9 Had[3429]: [ID 702911 daemon.notice] VCS ERROR V-16-1-54031 Resource data_dg (Owner: Unspecified, Group: datagrp) is FAULTED on sys sca-m532-504-gd10
Jun 27 16:00:42 sca-m532-504-gd9 Had[3429]: [ID 702911 daemon.notice] VCS ERROR V-16-1-10205 Group datagrp is faulted on system sca-m532-504-gd10
 

################# SCSI3-PR Test of Diskgroup ######################################

sca-m532-504-gd9:/root
[rootD:]825> /opt/VRTS/bin/vxfentsthdw -n -c datadg          

Veritas vxfentsthdw version 6.0.3 Solaris

The utility vxfentsthdw works on the two nodes of the cluster.
The utility verifies that the shared storage one intends to use is
configured to support I/O fencing.  It issues a series of vxfenadm
commands to setup SCSI-3 registrations on the disk, verifies the
registrations on the disk, and removes the registrations from the disk.

The logfile generated for vxfentsthdw is /var/VRTSvcs/log/vxfen/vxfentsthdw.log.10882

Enter the first node of the cluster:
sca-m532-504-gd9
Enter the second node of the cluster:
sca-m532-504-gd10

********************************************

Testing sca-m532-504-gd9 /dev/vx/rdmp/zfs_71200_3 sca-m532-504-gd10 /dev/vx/rdmp/zfs_71200_3

Evaluate the disk before testing  ........................ No Pre-existing keys
RegisterIgnoreKeys on disk /dev/vx/rdmp/zfs_71200_3 from node sca-m532-504-gd9  Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_3 on node sca-m532-504-gd9  Passed
RegisterIgnoreKeys on disk /dev/vx/rdmp/zfs_71200_3 from node sca-m532-504-gd10  Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_3 on node sca-m532-504-gd9  Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_3 on node sca-m532-504-gd10  Passed
Preempt and abort key KeyA using key KeyB on node sca-m532-504-gd10 .... Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_3 on node sca-m532-504-gd9  Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_3 on node sca-m532-504-gd10  Passed
Remove key KeyB on node sca-m532-504-gd10 .............................. Passed
Check to verify there are no keys from node sca-m532-504-gd9 ........... Passed

ALL tests on the disk /dev/vx/rdmp/zfs_71200_3 have PASSED.
The disk is now ready to be configured for I/O Fencing on node sca-m532-504-gd9
as a COORDINATOR DISK.

ALL tests on the disk /dev/vx/rdmp/zfs_71200_3 have PASSED.
The disk is now ready to be configured for I/O Fencing on node sca-m532-504-gd10
as a COORDINATOR DISK.

********************************************

Testing sca-m532-504-gd9 /dev/vx/rdmp/zfs_71200_4 sca-m532-504-gd10 /dev/vx/rdmp/zfs_71200_4

Evaluate the disk before testing  ........................ No Pre-existing keys
RegisterIgnoreKeys on disk /dev/vx/rdmp/zfs_71200_4 from node sca-m532-504-gd9  Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_4 on node sca-m532-504-gd9  Passed
RegisterIgnoreKeys on disk /dev/vx/rdmp/zfs_71200_4 from node sca-m532-504-gd10  Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_4 on node sca-m532-504-gd9  Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_4 on node sca-m532-504-gd10  Passed
Preempt and abort key KeyA using key KeyB on node sca-m532-504-gd10 .... Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_4 on node sca-m532-504-gd9  Passed
Verify registrations for disk /dev/vx/rdmp/zfs_71200_4 on node sca-m532-504-gd10  Passed
Remove key KeyB on node sca-m532-504-gd10 .............................. Passed
Check to verify there are no keys from node sca-m532-504-gd9 ........... Passed

ALL tests on the disk /dev/vx/rdmp/zfs_71200_4 have PASSED.
The disk is now ready to be configured for I/O Fencing on node sca-m532-504-gd9
as a COORDINATOR DISK.

ALL tests on the disk /dev/vx/rdmp/zfs_71200_4 have PASSED.
The disk is now ready to be configured for I/O Fencing on node sca-m532-504-gd10
as a COORDINATOR DISK.

Removing test keys and temporary files, if any...

sca-m532-504-gd9:/root
[rootD:]826>

Operating Systems:
Discussion Filed Under:

Comments 7 CommentsJump to latest comment

sajith_cr's picture

From the message, it loks like SCSI3-PR is disabled on the system

You can verify it from

# cat /etc/vx/volboot | grep scsi3_pr

If you find it ias off, enable SCSI3-PR with following command.

# vxdctl scsi3pr on

Now clear the fault and online the resource again.

Just for clarity,

After the Error message "Diskgroup [datadg] failed to import with groupreserve option as scsi3pr is disabled." another Info message should appear as follows.

Enable the scsi3pr option and retry online. To enable scsi3pr run the command [vxdctl scsi3pr on].

Do you have this message in your engine log?

Regards,

Sajith

If this post has helped you, please vote or mark as solution.

SOLUTION
ddh's picture

Thanks! My resources can come online after I enabled the scsi3-pr on the system with "vxdctl scsi3pr on"

ddh's picture

Hi,

The solution shown above does not seem to persist across reboots. After I reboot the system, "scsi3_pr off" reappeared in /etc/vx/volboot. Is that a normal behavior?  How can I make it persist across reboots?

Thanks in advance for you help.

sajith_cr's picture

Hi,

The scsi3pr value will be persistent in volboot file. Need to check if there is any script that is setting it to disabled after reboot.  Is there any such command in /etc/vx/log/cmdlog that during boot time “vxdctl scsi3pr off”?

Also compare the /etc/vx/volboot file before and after setting it the scsi3pr to on, before reboot and after reboot to rule out any case where the updations is failing.

Regards,

Sajith

If this post has helped you, please vote or mark as solution.

ddh's picture

Hi Sajith,

The command log does shows that there are "vxdctl scsi3pr off" commands. Where can I look to find the startup scripts for VCS that disables this. Is there any commands to run that may remove this setting from the startup scripts?

Thanks!!! Really appreciate your help!!!

sajith_cr's picture

Hi,

VCS start up scripts do not disable scsi3pr settings.

I recommend to post this question on VxVM forum.

Regards,

Sajith

If this post has helped you, please vote or mark as solution.