Disk exclusion does not work consistently for DMP devices under Linux

Article:TECH174808

Created: 2011-11-18

Updated: 2011-11-20

Article URL http://www.symantec.com/docs/TECH174808

NOTE: If you are experiencing this particular known issue, we recommend that you to receive email notification each time this article is updated. Subscribers will be the first to learn about any releases, status changes, workarounds or decisions made.

Article Type
Technical Solution

Product(s)

Show all

Environment

Show all

Languages

Show all

Problem

Excluding a device under 5.1SP1RP1 and later does not always work correctly for VERITAS DMP (Dymanic Multi-Pathing ) devices. The command `vxdmpadm exclude vxvm dmpnodename=<device>` will suppress all paths, but after a reboot paths may become visible when viewed via command `vxdisk list`.

Error

For example:

We can exclude the dmp node sdp from Volume Manager.

# vxdmpadm exclude vxvm dmpnodename=sdp

The /etc/vx/vxvm.exclude file will get populated as follows:

# cat /etc/vx/vxvm.exclude

exclude_all 0

paths

sdag pci-0000:0d:00.1-fc-0x5006016030216f40:0x000c000000000000 sdp

sdp pci-0000:0d:00.0-fc-0x5006016130216f40:0x000c000000000000 sdp

At this point everything should be working correctly. `vxdisk list` no longer shows dmpnode sdp.

We can then change device sdp to sdpz by editing the vxvm.exclude file. The idea here is to check whether VxVM (Veritas Volume Manager) is tracking the sd (scsi device) name or the hardware path for its exclusion.

# cat /etc/vx/vxvm.exclude

exclude_all 0

paths

sdag pci-0000:0d:00.1-fc-0x5006016030216f40:0x000c000000000000 sdp

sdpz pci-0000:0d:00.0-fc-0x5006016130216f40:0x000c000000000000 sdp

After rebooting, everything is still working. The ‘sdp’ device is still hidden from view in ‘vxdisk list’

Next lets change the sd name for the alternate path.

# cat /etc/vx/vxvm.exclude

exclude_all 0

paths

sdaz pci-0000:0d:00.1-fc-0x5006016030216f40:0x000c000000000000 sdp

sdpz pci-0000:0d:00.0-fc-0x5006016130216f40:0x000c000000000000 sdp

After rebooting we can see that ‘sdp’ is showing up in ‘vxdisk list’. This proves that device name changes are not being tracked correctly.

# vxdisk -o alldgs list |grep sdp

sdp auto:cdsdisk - (testdg) online

If we do a ‘vxdisk list sdp’ we can see that only the secondary path is detected, so the primary path is still being suppressed correctly.

# vxdisk list sdp

Device: sdp

devicetag: sdp

type: auto

hostid:

disk: name= id=1302728494.28.sprs1950a0-28

group: name=testdg id=1316677972.42.sprs1950a0-27

info: format=cdsdisk,privoffset=256,pubslice=3,privslice=3

flags: online ready private autoconfig

pubpaths: block=/dev/vx/dmp/sdp3 char=/dev/vx/rdmp/sdp3

guid: -

udid: DGC%5FRAID%205%5FAPM00064800091%5F60060160C6821800761EF93504EBDC11

site: -

version: 3.1

iosize: min=512 (bytes) max=1024 (blocks)

public: slice=3 offset=65792 len=7094016 disk_offset=0

private: slice=3 offset=256 len=65536 disk_offset=0

update: time=1320135639 seqno=0.86

ssb: actual_seqno=0.0

headers: 0 240

configs: count=1 len=51360

logs: count=1 len=4096

Defined regions:

config priv 000048-000239[000192]: copy=01 offset=000000 enabled

config priv 000256-051423[051168]: copy=01 offset=000192 enabled

log priv 051424-055519[004096]: copy=01 offset=000000 enabled

lockrgn priv 055520-055663[000144]: part=00 offset=000000

Multipathing information:

numpaths: 1

sdag state=enabled type=secondary

Environment

Storage Foundation 5.1SP1RP1 and Later for supported Linux environments.

Cause

Storage Foundation 5.1SP1RP1 introduced enhanced device suppression functionality, which tracks the hardware physical device path as well as the sd (scsi device) name. If the OS (Operating System) device sd name changes on reboot VxVM (Veritas Volume Manager) will track the hardware physical device path; thus still suppress the device excluded, irrespective of the sd name which may change on reboots. VxVM tracks the hardware physical path by creating directory "/dev/vx/.dmp/", which contains symbolic links between sd names and hardware paths.

example,

# ls -al /dev/vx/.dmp
total 0
drwxr-xr-x 2 root root 800 Nov 19 19:32 .
drwxr-xr-x 6 root root 380 Nov 19 19:29 ..
lrwxrwxrwx 1 root root 82 Nov 19 19:32 sda -> /dev/disk/by-path/pci-0000:02:08.0-sas-0x500188b12ff5f300:1:0-0x500000e0127f9d62:0
lrwxrwxrwx 1 root root 75 Nov 19 19:32 sdaa -> /dev/disk/by-path/pci-0000:0d:00.1-fc-0x5006016030216f40:0x0006000000000000
lrwxrwxrwx 1 root root 75 Nov 19 19:32 sdab -> /dev/disk/by-path/pci-0000:0d:00.1-fc-0x5006016030216f40:0x0007000000000000
lrwxrwxrwx 1 root root 75 Nov 19 19:32 sdac -> /dev/disk/by-path/pci-0000:0d:00.1-fc-0x5006016030216f40:0x0008000000000000

The problem is `/lib/udev/vxpath_links` script is not working correctly. The script is responsible for creating the ".dmp" directory and links and is ran during boot up of the server. On reboot the `/lib/udev/vxpath_links` script is not creating the "/dev/vx/.dmp" directory resulting in device exclusion issues.

[ Side affects of the issue ]

a. After reboot directory "/dev/vx/.dmp" is not created.

b. After a reboot since directory "/dev/vx/.dmp" does not exist if the user attempts to exclude devices it will default to pre 5.1SP1RP1 device exclusion where hardware physical path is no longer used and instead only sd device paths are included in /etc/vx/vxvm.exclude.

example,

paths
sdak /dev/sdak emc_clariion0_127
sdt /dev/sdt emc_clariion0_127
#
controllers
#
product
#
pathgroups
#

b. If a device other than the boot disk is excluded the boot disk will also inadevertently be excluded. If the user attempts to encapsulate the boot disk as the bootdisk is not visible in `vxdisk list`, then encapsulation will fail.

example,

# df -k|grep "sda1"
/dev/sda1 29753556 18630228 9587516 67% /

# vxdmpadm listexclude
Devices excluded from VxVM:
--------------------------
Paths : sdak sdt

Controllers : None

VID:PID : None

Devices excluded from multipathing by vxdmp:
-------------------------------------------
Paths : None

Controllers : None

VID:PID : None

Pathgroups : None
----------

# Yet command `vxdisk list` will not show device sda.

Solution

This is scheduled to be fixed in 5.1SP1RP3.

Workarounds:

a. After the system has been rebooted, you can manually run `/sbin/udevtrigger`. This should generate the missing /dev/vx/.dmp/ directory which should fix the issue.

b. Includ the `/sbin/udevtrigger` in /etc/rc.d/rc.local file so that on reboot the `/sbin/udevtrigger` automatically gets ran and directory "/dev/vx/.dmp" gets created along with associated sd links to hardwar path.

example,

# cat /etc/rc.d/rc.local
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local
/sbin/udevtrigger

Supplemental Materials

Source	ETrack
Value	2600931
Description	New dmp exclusion feature introduced in 5.1SP1RP1 is not working for alternate paths if device name changes.

TECH148170 Veritas Volume Manager 5.1 SP1 RP1 (Linux) introduces enhanced device suppression functionality

Article URL http://www.symantec.com/docs/TECH174808

Disk exclusion does not work consistently for DMP devices under Linux

Problem

Error

Environment

Cause

Solution

Supplemental Materials

Related Articles

Please Sign In

Knowledge Base Search

Knowledge Base Search

MySymantec

MySymantec

Contacting Support

Contacting Support