Video Screencast Help

New disk "ERROR" in vxdisk list

Created: 04 Oct 2010 • Updated: 06 Nov 2010 | 20 comments
This issue has been solved. See solution.

I have 2 new SAN disks attached to a host.  One looks normal and Veritas can see it and initialize it.  The other, shows an "ERROR" in vxdisk list.

 vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
c0t0d0s2     auto:none       -            -            online invalid
c2t0d0s2     auto:none       -            -            online invalid
c2t6d0s2     auto:none       -            -            online invalid
fabric_50    auto:cdsdisk    fabric_11    ocsrawdg     online shared

. . .

fabric_78    auto:cdsdisk    fabric_28    ocsrawdg     online shared
fabric_79    auto            -            -            error
fabric_80    auto:cdsdisk    -            -            online

I left out a bunch of other disks between fabric_50 and fabric_78 as they are not relevant.  Note fabric_79 and fabric_80 are the 2 new disks.  They both appear normal in Solaris format, and the NetApp host tools show them as both good.

#format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848>
          /ssm@0,0/pci@18,600000/scsi@2/sd@0,0
       1. c2t0d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848>
          /ssm@0,0/pci@1c,600000/scsi@2/sd@0,0
       2. c2t6d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848>
          /ssm@0,0/pci@1c,600000/scsi@2/sd@6,0
       3. c8t60A98000486E5A71675A5A447168634Bd0 <NETAPP-LUN-7320 cyl 6526 alt 2 hd 16 sec 2048>
          /scsi_vhci/ssd@g60a98000486e5a71675a5a447168634b
       4. c8t60A98000486E5A7153345A4471373748d0 <NETAPP-LUN-7320 cyl 48820 alt 2 hd 255 sec 189>
          /scsi_vhci/ssd@g60a98000486e5a7153345a4471373748
 
#sanlun lun show
controller:                      lun-pathname                                          device filename                       adapter  protocol          lun size         lun state
     filer2:  /vol/acqbiz_vis_prod_nona_nox_cluster_ebsfsdg/lun1   /dev/rdsk/c8t60A98000486E5A71675A5A447168634Bd0s2          qlc1     FCP          102g (109521666048)   GOOD    
     filer1:  /vol/acqbiz_vis_prod_nona_nox_cluster_ocsrawdg/lun1  /dev/rdsk/c8t60A98000486E5A7153345A4471373748d0s2          qlc1     FCP          1.1t (1204738326528)  GOOD  
 
I've left out a lot of excess output, but the interesting stuff should be here.  Finally, issuing a vxdisk init give an error.
 
#vxdisk init fabric_79
VxVM vxdisk ERROR V-5-1-5433 Device fabric_79: init failed:
        Device path not valid
 
I even tried dd'ing /dev/zero onto the first 4 blocks of the device and relabeling the disk with format.  Still no joy.  Does anyone have any idea what the problem might be?  I'm going to have hard time convincing the SAN folks it's a problem since it looks fine with format and the NetApp tool, but there must be something I've missed.  One thing I probably should mention is the LUN is 1.1T in size. 

Comments 20 CommentsJump to latest comment

Gaurav Sangamnerkar's picture

Interesting....

Can you paste the current prtvtoc output from the disk ? Also paste these:

# vxdisk -e list  |egrep 'fabric_79|fabric_80'

# prtvtoc /dev/vx/rdmp/fabric_79

# prtvtoc /dev/vx/rdmp/fabric_80

# vxdisk list fabric_79

# vxdisk list fabric_80

# vxdmpadm listenclosure all

# vxdmpadm listctlr all

 

Gaurav

PS: If you are happy with the answer provided, please mark the post as solution. You can do so by clicking link "Mark as Solution" below the answer provided.
 

Bill Ranck's picture

Here are the requested outputs:

#vxdisk -e list  |egrep 'fabric_79|fabric_80'
fabric_79    auto      -             -            error        c8t60A98000486E5A7153345A4471373748d0s2
fabric_80    auto      -             -            online       c8t60A98000486E5A71675A5A447168634Bd0s2

#prtvtoc /dev/vx/rdmp/fabric_79
* /dev/vx/rdmp/fabric_79 partition map
*
* Dimensions:
*     512 bytes/sector
*     189 sectors/track
*     255 tracks/cylinder
*   48195 sectors/cylinder
*   48822 cylinders
*   48820 accessible cylinders
*
* Flags:
*   1: unmountable
*  10: read-only
*
*                          First     Sector    Last
* Partition  Tag  Flags    Sector     Count    Sector  Mount Directory
       2      5    01          0 2352879900 2352879899
       6      4    00          0 2352879900 2352879899

#prtvtoc /dev/vx/rdmp/fabric_80
* /dev/vx/rdmp/fabric_80 partition map
*
* Dimensions:
*     512 bytes/sector
*    2048 sectors/track
*      16 tracks/cylinder
*   32768 sectors/cylinder
*    6528 cylinders
*    6526 accessible cylinders
*
* Flags:
*   1: unmountable
*  10: read-only
*
*                          First     Sector    Last
* Partition  Tag  Flags    Sector     Count    Sector  Mount Directory
       2      5    01          0 213843968 213843967
       7     15    01          0 213843968 213843967

#vxdmpadm listenclosure all
ENCLR_NAME        ENCLR_TYPE     ENCLR_SNO            STATUS       ARRAY_TYPE
============================================================================
Disk              Disk           DISKS                DISCONNECTED -
OTHER_DISKS       OTHER_DISKS    OTHER_DISKS          CONNECTED    OTHER_DISKS

#vxdmpadm listctlr all
CTLR-NAME       ENCLR-TYPE      STATE      ENCLR-NAME
=====================================================
c8              OTHER_DISKS     ENABLED      OTHER_DISKS
c2              OTHER_DISKS     ENABLED      OTHER_DISKS
c0              OTHER_DISKS     ENABLED      OTHER_DISKS

Gaurav Sangamnerkar's picture

paste below as well

 

# vxdisk list fabric_79

# vxdisk list fabric_80

 

Need to know if paths to disk are enabled or disabled...

 

Gaurav

PS: If you are happy with the answer provided, please mark the post as solution. You can do so by clicking link "Mark as Solution" below the answer provided.
 

Bill Ranck's picture

Oops, sorry I missed those.  Here they are:

#vxdisk list fabric_79
Device:    fabric_79
devicetag: fabric_79
type:      auto
flags:     online error private autoconfig
errno:     Device path not valid
Multipathing information:
numpaths:   1
c8t60A98000486E5A7153345A4471373748d0s2 state=enabled

#vxdisk list fabric_80
Device:    fabric_80
devicetag: fabric_80
type:      auto
hostid:   
disk:      name= id=1285941906.39.

group:     name= id=
info:      format=cdsdisk,privoffset=256,pubslice=2,privslice=2
flags:     online ready private autoconfig autoimport
pubpaths:  block=/dev/vx/dmp/fabric_80s2 char=/dev/vx/rdmp/fabric_80s2
version:   3.1
iosize:    min=512 (bytes) max=2048 (blocks)
public:    slice=2 offset=2304 len=213841664 disk_offset=0
private:   slice=2 offset=256 len=2048 disk_offset=0
update:    time=1285941906 seqno=0.1
ssb:       actual_seqno=0.0
headers:   0 240
configs:   count=1 len=1280
logs:      count=1 len=192
Defined regions:
 config   priv 000048-000239[000192]: copy=01 offset=000000 disabled
 config   priv 000256-001343[001088]: copy=01 offset=000192 disabled
 log      priv 001344-001535[000192]: copy=01 offset=000000 disabled
 lockrgn  priv 001536-001679[000144]: part=00 offset=000000
Multipathing information:
numpaths:   1
c8t60A98000486E5A71675A5A447168634Bd0s2 state=enabled

Gaurav Sangamnerkar's picture

hmm nothing looks wrong there.... can u also let me know the SF version you are using ?

# modinfo |grep -i vx

 

Also give a try on these steps:

 -- since fabric_79 is a fresh disk, I hope it doesn't have any data.... so delete slice 6 using format utility

-- Label the disk using format utility after deleting slice 6

-- rescan OS tree   using  "devfsadm -Cv"   (notice any errors)

-- rescan veritas tree using "vxdctl enable"

-- Now see if disk still shows in error state or no..

 

Gaurav

PS: If you are happy with the answer provided, please mark the post as solution. You can do so by clicking link "Mark as Solution" below the answer provided.
 

Bill Ranck's picture

#modinfo |grep -i vx
 32  1347890  29a00 259   1  vxdmp (VxVM 4.1_p3.1: DMP Driver)
 33 7be00000 216778 260   1  vxio (VxVM 4.1_p3.1 I/O driver)
 35  136ee28   13f8 261   1  vxspec (VxVM 4.1_p3.1 control/status dr)

 

I followed the procedure above, used format to remove slice6, relabeled, did the devfsadm (no errors), vxdctl enable, and still the same result.  It has an error flag set.

Here's a bunch of output from the procedure:

partition> pr
Current partition table (original):
Total disk cylinders available: 48820 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders         Size            Blocks
  0       root    wm       0                0         (0/0/0)              0
  1       swap    wu       0                0         (0/0/0)              0
  2     backup    wu       0 - 48819        1.10TB    (48820/0/0) 2352879900
  3 unassigned    wm       0                0         (0/0/0)              0
  4 unassigned    wm       0                0         (0/0/0)              0
  5 unassigned    wm       0                0         (0/0/0)              0
  6        usr    wm       0 - 48819        1.10TB    (48820/0/0) 2352879900
  7 unassigned    wm       0                0         (0/0/0)              0
partition> 6
Part      Tag    Flag     Cylinders         Size            Blocks
  6        usr    wm       0 - 48819        1.10TB    (48820/0/0) 2352879900
Enter partition id tag[usr]:
Enter partition permission flags[wm]:
Enter new starting cyl[0]:
Enter partition size[2352879900b, 48820c, 48819e, 1148867.12mb, 1121.94gb]: 0
partition> pr
Current partition table (unnamed):
Total disk cylinders available: 48820 + 2 (reserved cylinders)
Part      Tag    Flag     Cylinders         Size            Blocks
  0       root    wm       0                0         (0/0/0)              0
  1       swap    wu       0                0         (0/0/0)              0
  2     backup    wu       0 - 48819        1.10TB    (48820/0/0) 2352879900
  3 unassigned    wm       0                0         (0/0/0)              0
  4 unassigned    wm       0                0         (0/0/0)              0
  5 unassigned    wm       0                0         (0/0/0)              0
  6 unassigned    wm       0                0         (0/0/0)              0
  7 unassigned    wm       0                0         (0/0/0)              0
partition> label
Ready to label disk, continue? y
partition> q
FORMAT MENU:
        disk       - select a disk
        type       - select (define) a disk type
        partition  - select (define) a partition table
        current    - describe the current disk
        format     - format and analyze the disk
        repair     - repair a defective sector
        label      - write label to the disk
        analyze    - surface analysis
        defect     - defect list management
        backup     - search for backup labels
        verify     - read and display labels
        save       - save new disk/partition definitions
        inquiry    - show vendor, product and revision
        volname    - set 8-character volume name
        !<cmd>     - execute <cmd>, then return
        quit
format> q
Nox#devfsadm -Cv
devfsadm[9517]: verbose: chown/chmod /devices/scsi_vhci:devctl 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2:devctl 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2,1:devctl 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@0,0:a 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@0,0:b 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@0,0:c 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@0,0:d 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@0,0:e 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@0,0:f 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@0,0:g 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@0,0:h 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@0,0:a,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@0,0:b,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@0,0:c,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@0,0:d,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@0,0:e,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@0,0:f,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@0,0:g,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@0,0:h,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@6,0:a 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@6,0:b 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@6,0:c 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@6,0:d 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@6,0:e 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@6,0:f 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@6,0:g 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@6,0:h 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@6,0:a,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@6,0:b,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@6,0:c,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@6,0:d,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@6,0:e,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@6,0:f,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@6,0:g,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@18,600000/scsi@2/sd@6,0:h,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2:devctl 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2,1:devctl 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@0,0:a 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@0,0:b 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@0,0:c 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@0,0:d 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@0,0:e 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@0,0:f 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@0,0:g 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@0,0:h 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@0,0:a,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@0,0:b,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@0,0:c,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@0,0:d,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@0,0:e,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@0,0:f,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@0,0:g,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@0,0:h,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@6,0:a 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@6,0:b 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@6,0:c 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@6,0:d 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@6,0:e 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@6,0:f 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@6,0:g 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@6,0:h 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@6,0:a,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@6,0:b,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@6,0:c,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@6,0:d,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@6,0:e,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@6,0:f,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@6,0:g,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1c,600000/scsi@2/sd@6,0:h,raw 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1d,600000/pci@2/scsi@2:devctl 0l/3l/0
devfsadm[9517]: verbose: chown/chmod /devices/ssm@0,0/pci@1d,600000/pci@2/scsi@2,1:devctl 0l/3l/0
devfsadm[9517]: verbose: removing file: /dev/es/ses0
devfsadm[9517]: verbose: removing file: /dev/es/ses1
#vxdctl enable
#vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
c0t0d0s2     auto:none       -            -            online invalid
c2t0d0s2     auto:none       -            -            online invalid
c2t6d0s2     auto:none       -            -            online invalid
fabric_50    auto:cdsdisk    fabric_11    ocsrawdg     online shared
fabric_51    auto:cdsdisk    fabric_10    ocsrawdg     online shared
fabric_52    auto:cdsdisk    fabric_9     ocsrawdg     online shared
fabric_53    auto:cdsdisk    fabric_0     ocsfsdg      online
fabric_54    auto:cdsdisk    -            -            online
fabric_55    auto:cdsdisk    fabric_2     ocsfsdg      online
fabric_56    auto:cdsdisk    fabric_3     ocsrawdg     online shared
fabric_57    auto:cdsdisk    fabric_4     ocsrawdg     online shared
fabric_58    auto:cdsdisk    fabric_5     ocsrawdg     online shared
fabric_59    auto:cdsdisk    fabric_6     ocsrawdg     online shared
fabric_60    auto:cdsdisk    ocsfsdg01    ocsfsdg      online
fabric_61    auto:cdsdisk    fabric_8     ocsrawdg     online shared
fabric_62    auto:cdsdisk    fabric_12    ocsrawdg     online shared
fabric_63    auto:cdsdisk    fabric_22    ocsrawdg     online shared
fabric_64    auto:cdsdisk    fabric_26    ocsrawdg     online shared
fabric_65    auto:cdsdisk    fabric_15    ocsrawdg     online shared
fabric_66    auto:cdsdisk    fabric_14    ocsrawdg     online shared
fabric_67    auto:cdsdisk    fabric_16    ocsrawdg     online shared
fabric_68    auto:cdsdisk    fabric_24    ocsrawdg     online shared
fabric_69    auto:cdsdisk    fabric_13    ocsrawdg     online shared
fabric_70    auto:cdsdisk    fabric_19    ocsrawdg     online shared
fabric_71    auto:cdsdisk    fabric_17    ocsrawdg     online shared
fabric_72    auto:cdsdisk    fabric_23    ocsrawdg     online shared
fabric_73    auto:cdsdisk    fabric_18    ocsrawdg     online shared
fabric_74    auto:cdsdisk    fabric_20    ocsrawdg     online shared
fabric_75    auto:cdsdisk    fabric_25    ocsrawdg     online shared
fabric_76    auto:cdsdisk    fabric_27    ocsrawdg     online shared
fabric_77    auto:cdsdisk    fabric_21    ocsrawdg     online shared
fabric_78    auto:cdsdisk    fabric_28    ocsrawdg     online shared
fabric_79    auto            -            -            error
fabric_80    auto:cdsdisk    -            -            online
#vxdisk list fabric_79
Device:    fabric_79
devicetag: fabric_79
type:      auto
flags:     online error private autoconfig
errno:     Device path not valid
Multipathing information:
numpaths:   1
c8t60A98000486E5A7153345A4471373748d0s2 state=enabled
Bill Ranck's picture

Just to be sure, I did mention that this is a large LUN.  1.1T

Is that a problem?

Gaurav Sangamnerkar's picture

whether both fabric_79 & fabric_80 are of same size ? If only fabric_79 is greater than 1TB, try changing the type of disk using Format utility..

 

If its a 1.1T Lun, you might face issue in initializing it veritas, but atleast it should be visible to vx ....

If this is a standalone server, then try below procedure... (if this is a cluster server, don't use below procedure)

 

can you try this....

# mv /etc/vx/disk.info /etc/vx/disk.info.old  (move disk.info to old name)

# rm /dev/vx/rdmp/fabric_79

# rm /dev/vx/dmp/fabric_79

# rm /dev/rdsk/c8t60A98000486E5A7153345A4471373748d0*

# rm /dev/dsk/c8t60A98000486E5A7153345A4471373748d0*

# devfsadm -Cv

# vxconfigd -k   (this is important step, it will restart vxconfigd daemon & will regenerate disk.info file)

If you are using veritas cluster CFS/CVM, restarting vxconfigd may cause failover to happen, so if you use VCS, don't do above steps... if it is a standalone server, you can try above procedure..

Gaurav

PS: If you are happy with the answer provided, please mark the post as solution. You can do so by clicking link "Mark as Solution" below the answer provided.
 

Bill Ranck's picture

This is a Sun Cluster setup.  So, I went with G Lee's suggestions.

g_lee's picture

Bill,

The disk is >1TB so it needs an EFI label, it currently has an SMI label which is why you're having issues. Remove disk from vxvm, relabel as EFI, rescan

1. Ensure the event source daemon is not running so it won't recreate disk while you're trying to cleanup
# ps -ef |grep vxesd
If it is running, stop it
# vxddladm stop eventsource
# ps -ef |grep vxesd

2. Remove disks from vxdisk list and the underlying dmp devices
# vxdisk rm fabric_79
# ls -la /dev/vx/*dmp/fabric_79* ### ensure this just lists the fabric_79 devices/slices
# rm /dev/vx/*dmp/fabric_79*
# ls -la /dev/vx/*dmp/fabric_79*  ### should not find anything

3. Relabel as EFI
# format -e c8t60A98000486E5A7153345A4471373748d0
format> label
[will give you option for SMI or EFI label. select EFI label, follow prompts]
format> q
# prtvtoc /dev/rdsk/c8t60A98000486E5A7153345A4471373748d0s0
### should now see slice 8 for EFI label

4. Now this disk has correct label, rebuild dmp dev tree, rescan into vxvm
# vxdctl initdmp
# vxdctl enable
# vxdisk list

Will add one additional caveat - you're on 4.1MP1 (ie: fairly old) - I *think* this version does still have support for disks >1TB but can't find a specific mention when I skim the release notes - if you run into issues it might be best to log a support case.

If this post has helped you, please vote or mark as solution

Bill Ranck's picture

Almost there!  The procedure G Lee suggested seems to have worked, but there is a problem at the very end.  Here is output from all the steps.

Nox#ps -ef |grep vxesd
    root   345     1   0   Jul 31 ?           0:27 /sbin/vxesd
    root 18947 17791   0 08:28:18 pts/1       0:00 grep vxesd
Nox#vxddladm stop eventsource
Nox#ps -ef |grep vxesd
    root 19127 17791   0 08:28:36 pts/1       0:00 grep vxesd

Nox#vxdisk rm fabric_79
Nox#ls -la /dev/vx/*dmp/fabric_79*
brw-------   1 root     root     259, 258 Oct  4 15:32 /dev/vx/dmp/fabric_79
brw-------   1 root     root     259, 256 Oct  4 15:32 /dev/vx/dmp/fabric_79s0
brw-------   1 root     root     259, 257 Oct  4 15:32 /dev/vx/dmp/fabric_79s1
brw-------   1 root     root     259, 258 Oct  4 15:32 /dev/vx/dmp/fabric_79s2
brw-------   1 root     root     259, 259 Oct  4 15:32 /dev/vx/dmp/fabric_79s3
brw-------   1 root     root     259, 260 Oct  4 15:32 /dev/vx/dmp/fabric_79s4
brw-------   1 root     root     259, 261 Oct  4 15:32 /dev/vx/dmp/fabric_79s5
brw-------   1 root     root     259, 262 Oct  4 15:32 /dev/vx/dmp/fabric_79s6
brw-------   1 root     root     259, 263 Oct  4 15:32 /dev/vx/dmp/fabric_79s7
crw-------   1 root     root     259, 258 Oct  4 15:32 /dev/vx/rdmp/fabric_79
crw-------   1 root     root     259, 256 Oct  4 15:32 /dev/vx/rdmp/fabric_79s0
crw-------   1 root     root     259, 257 Oct  4 15:32 /dev/vx/rdmp/fabric_79s1
crw-------   1 root     root     259, 258 Oct  4 15:32 /dev/vx/rdmp/fabric_79s2
crw-------   1 root     root     259, 259 Oct  4 15:32 /dev/vx/rdmp/fabric_79s3
crw-------   1 root     root     259, 260 Oct  4 15:32 /dev/vx/rdmp/fabric_79s4
crw-------   1 root     root     259, 261 Oct  4 15:32 /dev/vx/rdmp/fabric_79s5
crw-------   1 root     root     259, 262 Oct  4 15:32 /dev/vx/rdmp/fabric_79s6
crw-------   1 root     root     259, 263 Oct  4 15:32 /dev/vx/rdmp/fabric_79s7
Nox#rm /dev/vx/*dmp/fabric_79*
Nox#ls -la /dev/vx/*dmp/fabric_79*
/dev/vx/*dmp/fabric_79*: No such file or directory
Nox#sanlun lun show
controller:                      lun-pathname                                          device filename                       adapter  protocol          lun size         lun state
     filer2:  /vol/acqbiz_vis_prod_nona_nox_cluster_ebsfsdg/lun1   /dev/rdsk/c8t60A98000486E5A71675A5A447168634Bd0s2          qlc1     FCP          102g (109521666048)   GOOD    
     filer1:  /vol/acqbiz_vis_prod_nona_nox_cluster_ocsrawdg/lun1  /dev/rdsk/c8t60A98000486E5A7153345A4471373748d0s2          qlc1     FCP          1.1t (1204738326528)  GOOD    
Nox#format -e c8t60A98000486E5A7153345A4471373748d0
selecting c8t60A98000486E5A7153345A4471373748d0
[disk formatted]

FORMAT MENU:
        disk       - select a disk
        type       - select (define) a disk type
        partition  - select (define) a partition table
        current    - describe the current disk
        format     - format and analyze the disk
        repair     - repair a defective sector
        label      - write label to the disk
        analyze    - surface analysis
        defect     - defect list management
        backup     - search for backup labels
        verify     - read and display labels
        save       - save new disk/partition definitions
        inquiry    - show vendor, product and revision
        scsi       - independent SCSI mode selects
        cache      - enable, disable or query SCSI disk cache
        volname    - set 8-character volume name
        !<cmd>     - execute <cmd>, then return
        quit
format> label
[0] SMI Label
[1] EFI Label
Specify Label type[0]: 1
Warning: This disk has an SMI label. Changing to EFI label will erase all
current partitions.
Continue? y
format> q
Nox#prtvtoc /dev/rdsk/c8t60A98000486E5A7153345A4471373748d0s2
* /dev/rdsk/c8t60A98000486E5A7153345A4471373748d0s2 partition map
*
* Dimensions:
*     512 bytes/sector
* 2353004544 sectors
* 2353004477 accessible sectors
*
* Flags:
*   1: unmountable
*  10: read-only
*
*                          First     Sector    Last
* Partition  Tag  Flags    Sector     Count    Sector  Mount Directory
       0      4    00         34 2352988093 2352988126
       8     11    00  2352988127     16384 2353004510
Nox#vxdctl initdmp                                  
Nox#vxdctl enable
Nox#vxdisk list
DEVICE       TYPE            DISK         GROUP        STATUS
c0t0d0s2     auto:none       -            -            online invalid
c2t0d0s2     auto:none       -            -            online invalid
c2t6d0s2     auto:none       -            -            online invalid
fabric_50    auto:cdsdisk    fabric_11    ocsrawdg     online shared
fabric_51    auto:cdsdisk    fabric_10    ocsrawdg     online shared
fabric_52    auto:cdsdisk    fabric_9     ocsrawdg     online shared
fabric_53    auto:cdsdisk    fabric_0     ocsfsdg      online
fabric_54    auto:cdsdisk    -            -            online
fabric_55    auto:cdsdisk    fabric_2     ocsfsdg      online
fabric_56    auto:cdsdisk    fabric_3     ocsrawdg     online shared
fabric_57    auto:cdsdisk    fabric_4     ocsrawdg     online shared
fabric_58    auto:cdsdisk    fabric_5     ocsrawdg     online shared
fabric_59    auto:cdsdisk    fabric_6     ocsrawdg     online shared
fabric_60    auto:cdsdisk    ocsfsdg01    ocsfsdg      online
fabric_61    auto:cdsdisk    fabric_8     ocsrawdg     online shared
fabric_62    auto:cdsdisk    fabric_12    ocsrawdg     online shared
fabric_63    auto:cdsdisk    fabric_22    ocsrawdg     online shared
fabric_64    auto:cdsdisk    fabric_26    ocsrawdg     online shared
fabric_65    auto:cdsdisk    fabric_15    ocsrawdg     online shared
fabric_66    auto:cdsdisk    fabric_14    ocsrawdg     online shared
fabric_67    auto:cdsdisk    fabric_16    ocsrawdg     online shared
fabric_68    auto:cdsdisk    fabric_24    ocsrawdg     online shared
fabric_69    auto:cdsdisk    fabric_13    ocsrawdg     online shared
fabric_70    auto:cdsdisk    fabric_19    ocsrawdg     online shared
fabric_71    auto:cdsdisk    fabric_17    ocsrawdg     online shared
fabric_72    auto:cdsdisk    fabric_23    ocsrawdg     online shared
fabric_73    auto:cdsdisk    fabric_18    ocsrawdg     online shared
fabric_74    auto:cdsdisk    fabric_20    ocsrawdg     online shared
fabric_75    auto:cdsdisk    fabric_25    ocsrawdg     online shared
fabric_76    auto:cdsdisk    fabric_27    ocsrawdg     online shared
fabric_77    auto:cdsdisk    fabric_21    ocsrawdg     online shared
fabric_78    auto:cdsdisk    fabric_28    ocsrawdg     online shared
fabric_79    auto:none       -            -            online invalid
fabric_80    auto:cdsdisk    -            -            online
Nox#vxdisk init fabric_79
VxVM vxdisk ERROR V-5-1-5433 Device fabric_79: init failed:
        Disk VTOC does not list private partition
 
Note the vxdisk init step fails. :-(
 
I tried adding a slice 2 backup, but that failed.  Here is what the partition table lookes like:
 
partition> pr
Current partition table (original):
Total disk sectors available: 2352988126 + 16384 (reserved sectors)
 
Part      Tag    Flag     First Sector          Size          Last Sector
  0        usr    wm                34         1.10TB           2352988126   
  1 unassigned    wm                 0            0                0   
  2 unassigned    wm                 0            0                0   
  3 unassigned    wm                 0            0                0   
  4 unassigned    wm                 0            0                0   
  5 unassigned    wm                 0            0                0   
  6 unassigned    wm                 0            0                0   
  8   reserved    wm        2352988127         8.00MB           2353004510
g_lee's picture

Bill,

Are you trying to initialise the disk?

Can you please try:
# vxdisksetup -i fabric_79

see this technote for some background (ie: vxdisksetup may be setting additional attributes): http://www.symantec.com/business/support/index?pag...

... however, this still may not work as you're using an old version - eg: the following technote references Powerpath, but also mentions that 4.1MP2 is required for full EFI support (""Note that the full support of the EFI disks requires VxVM 4.1 MP2 or VxVM 5.0 MP1 RP1. A reconfiguration reboot is necessary if either PowerPath or VxVM was upgraded.")

http://www.symantec.com/docs/TECH54064

Try the vxdisksetup, if this still does not work, please post the output and will see if I can find anything about EFI on 4.1MP1 in the meantime.

If this post has helped you, please vote or mark as solution

SOLUTION
Bill Ranck's picture

The SAN storage manager took the big LUN back and gave me a new one that he said was, "set up for EFI."  This made no difference.

It seems I don't have vxdisksetup, so that probably means my version is too old to handle >1T LUNs.  So, I have asked the SAN storage manager to split the space up into 4 pieces (may as well stripe it while I'm at it.)

Thanks for all the help.  I'm learning a lot here.

g_lee's picture

Bill,

Sorry to hear you couldn't get the >1TB lun to work. Although, given your version is quite old, it might be better to use a smaller lun for now (ie: until you can upgrade) to avoid running into any issues that would be fixed in more recent updates.

For reference, VxVM 4.1 does have vxdisksetup, but it's not under /usr/sbin so it doesn't come up by default unless you've set up your PATH variable (which many people don't do)

Location is /etc/vx/bin/vxdisksetup (or /usr/lib/vxvm/bin/vxdisksetup as /etc/vx/bin -> /usr/lib/vxvm/bin)

Hope that helps,

Grace

If this post has helped you, please vote or mark as solution

Marianne's picture

In addition to G Lee's post - I think we need to get back to basics and check compatilbility... If my memory serves me right, I think that EFI label is only supported on Solaris 10. I don't see the Solaris level mentioned anywhere...

To use latest disk technology, always ensure that the O/S and SF versions supports that.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

Bill Ranck's picture

It's Solaris 10.

SunOS xxxxxxxxxx 5.10 Generic_142900-15 sun4u sparc SUNW,Sun-Fire

g_lee's picture

Marianne,

You are right, I got a bit ahead of myself :\

EFI label is mainly for Solaris 10, however it also appears to be available from Solaris 9 4/03 from the following document:
http://docs.sun.com/app/docs/doc/817-6960/6mmah946...

Additional details/restrictions for EFI labels here:
http://docs.sun.com/app/docs/doc/817-5093/diskscon...

If this post has helped you, please vote or mark as solution

Gaurav Sangamnerkar's picture

just one suggestion, tried to initialize disk as sliced ?

# vxdisk init fabric_79  format=sliced

 

somehow I remember EFI label working with 4.1 (but not very sure)...

 

Gaurav

PS: If you are happy with the answer provided, please mark the post as solution. You can do so by clicking link "Mark as Solution" below the answer provided.
 

Bill Ranck's picture

Same error with format=sliced.  Apparently I have too old a version of 4.1

Marianne's picture

Correct - only supported as from 4.1 MP2 - as per TN posted by G.Lee:

http://www.symantec.com/business/support/index?pag...

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links