solution needed for vxfen issue
there is a two node cluster and we split two node cluster for upgrade. The isolated node is not coming up as vxfen is not starting
/02/01 11:52:05 VCS CRITICAL V-16-1-10037 VxFEN driver not configured. Retrying...
2013/02/01 11:52:20 VCS CRITICAL V-16-1-10037 VxFEN driver not configured. Retrying...
2013/02/01 11:52:35 VCS CRITICAL V-16-1-10037 VxFEN driver not configured. Retrying...
2013/02/01 11:52:50 VCS CRITICAL V-16-1-10037 VxFEN driver not configured. Retrying...
2013/02/01 11:53:05 VCS CRITICAL V-16-1-10037 VxFEN driver not configured. Retrying...
2013/02/01 11:53:20 VCS CRITICAL V-16-1-10031 VxFEN driver not configured. VCS Stopping. Manually restart VCS after configuring fencing
^C
IOFENCING configuration seems okay on node 2 as /etc/vxfentab has the entry for co-ordinator disks and /etc/vxfendg has diskgroup entry vxfendg2 and these disks and diskgroup are visible too.
DEVICE TYPE DISK GROUP STATUS
c0t5006016047201339d0s2 auto:sliced - (ossdg) online
c0t5006016047201339d1s2 auto:sliced - (sybasedg) online
c0t5006016047201339d2s2 auto:sliced - (vxfendg1) online
c0t5006016047201339d3s2 auto:sliced - (vxfendg1) online
c0t5006016047201339d4s2 auto:sliced - (vxfendg1) online
c0t5006016047201339d5s2 auto:sliced - (vxfendg2) online
c0t5006016047201339d6s2 auto:sliced - (vxfendg2) online
c0t5006016047201339d7s2 auto:sliced - - online
c2t0d0s2 auto:SVM - - SVM
c2t1d0s2 auto:SVM - - SVM
On checking vxfen.log
nvoked S97vxfen. Starting
Fri Feb 1 11:50:37 CET 2013 starting vxfen..
Fri Feb 1 11:50:37 CET 2013 calling start_fun
Fri Feb 1 11:50:38 CET 2013 found vxfenmode file
Fri Feb 1 11:50:38 CET 2013 calling generate_vxfentab
Fri Feb 1 11:50:38 CET 2013 checking for /etc/vxfendg
Fri Feb 1 11:50:38 CET 2013 found /etc/vxfendg.
Fri Feb 1 11:50:38 CET 2013 calling generate_disklist
Fri Feb 1 11:50:38 CET 2013 Starting vxfen.. Done.
Fri Feb 1 11:50:38 CET 2013 starting in vxfen-startup
Fri Feb 1 11:50:38 CET 2013 calling regular vxfenconfig
Fri Feb 1 11:50:38 CET 2013 return value from above operation is 1
Fri Feb 1 11:50:38 CET 2013 output was VXFEN vxfenconfig ERROR V-11-2-1003 At least three coordinator disks must be defined
Log Buffer: 0xfffffffff4041090
refadm2-oss1{root} # cat /etc/vxfendg
vxfendg2
and there are below mentioned two disks in vxfendg2
c0t5006016047201339d5s2 auto:sliced - (vxfendg2) online
c0t5006016047201339d6s2 auto:sliced - (vxfendg2) online
is it due to two disks in coordinator diskgroup? Is it a known issue ?
Comments 16 Comments • Jump to latest comment
could you please provide output of 'gabconfig -a' from both the nodes?
Thanks and Warm Regards,
Amit Rangari
If this post helped you resolving the issue, please mark it as solution. _____________________________________________________________________________
refadm2-oss1{root} # gabconfig -a
GAB Port Memberships
===============================================================
Port a gen d22e01 membership 0
refadm2-oss1{root} # gabconfig -a
GAB Port Memberships
===============================================================
Port a gen d22e01 membership 0
Both the nodes are split and cannot see eachother.
The coordinator diskgroup must have at least 3 disks. It is ok if a node can only see 2 of the 3 disks, must the diskgroup must contain 3 disks, and it looks as though vxfendg only contains 2 disks.
What version of VCS is this as with upgrades FROM 5.1 or higher you can do rolling upgrades, which means you do not need to split cluster - you can upgrade one node and VCS on the other node can still communicate with the other node even though it is on a different version.
Mike
UK Symantec Consultant in VCS, GCO, SF, VVR, VxAT on Solaris, AIX, HP-ux, Linux & Windows
If this post has helped you, please vote or mark as solution
version is 5.1SP1RP2P2.
how can i add the third disk in the coordinator diskgroup?
I am trying to add c0t5006016047201339d7s2 in vxfendg2 by importing vxfendg2.but it failed.
vxdisk -o alldgs list
DEVICE TYPE DISK GROUP STATUS
c0t5006016047201339d0s2 auto:sliced - (ossdg) online
c0t5006016047201339d1s2 auto:sliced - (sybasedg) online
c0t5006016047201339d2s2 auto:sliced - (vxfendg1) online
c0t5006016047201339d3s2 auto:sliced - (vxfendg1) online
c0t5006016047201339d4s2 auto:sliced - (vxfendg1) online
c0t5006016047201339d5s2 auto:sliced - (vxfendg2) online
c0t5006016047201339d6s2 auto:sliced - (vxfendg2) online
c0t5006016047201339d7s2 auto:sliced - - online
c2t0d0s2 auto:SVM - - SVM
c2t1d0s2 auto:SVM - - SVM
refadm2-oss1{root} # vxdg -t import vxfendg2
VxVM vxdg ERROR V-5-1-10978 Disk group vxfendg2: import failed:
Disk for disk group not found
refadm2-oss1{root} #
You could try to import with "-f" to see if there is just one disk that can't be seen or if it is all the disks.
Mike
UK Symantec Consultant in VCS, GCO, SF, VVR, VxAT on Solaris, AIX, HP-ux, Linux & Windows
If this post has helped you, please vote or mark as solution
refadm2-oss1{root} # vxdg -t import vxfendg2
VxVM vxdg ERROR V-5-1-10978 Disk group vxfendg2: import failed:
Disk for disk group not found
refadm2-oss1{root} # vxdg -tfC import vxfendg2
VxVM vxdg WARNING V-5-1-560 Disk c0t5006016C47201339d7: Not found, last known location: c0t5006016047201339d7s2
refadm2-oss1{root} # vxdg list
NAME STATE ID
vxfendg2 enabled 1340186515.29.refadm1-oss1
refadm2-oss1{root} # vxdisk -o alldgs list
DEVICE TYPE DISK GROUP STATUS
c0t5006016047201339d0s2 auto:sliced - (ossdg) online
c0t5006016047201339d1s2 auto:sliced - (sybasedg) online
c0t5006016047201339d2s2 auto:sliced - (vxfendg1) online
c0t5006016047201339d3s2 auto:sliced - (vxfendg1) online
c0t5006016047201339d4s2 auto:sliced - (vxfendg1) online
c0t5006016047201339d5s2 auto:sliced c0t5006016C47201339d5 vxfendg2 online
c0t5006016047201339d6s2 auto:sliced c0t5006016C47201339d6 vxfendg2 online
c0t5006016047201339d7s2 auto:sliced - - online
c2t0d0s2 auto:SVM - - SVM
c2t1d0s2 auto:SVM - - SVM
- - c0t5006016C47201339d7 vxfendg2 failed was:c0t5006016047201339d7s2
refadm2-oss1{root} #
Can you pls suggest me the action plan ?
Try 'vxreattach' command.
See http://www.symantec.com/docs/TECH175491
Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links
refadm2-oss1{root} # /etc/vx/bin/vxreattach -c c0t5006016C47201339d7
VxVM vxreattach ERROR V-5-2-238 No matching Volume Manager disk and device IDs found for c0t5006016047201339d7s2
refadm2-oss1{root} # /usr/lib/vxvm/bin/vxreattach
refadm2-oss1{root} # vxdisk -o alldgs list
DEVICE TYPE DISK GROUP STATUS
c0t5006016047201339d0s2 auto:sliced - (ossdg) online
c0t5006016047201339d1s2 auto:sliced - (sybasedg) online
c0t5006016047201339d2s2 auto:sliced - (vxfendg1) online
c0t5006016047201339d3s2 auto:sliced - (vxfendg1) online
c0t5006016047201339d4s2 auto:sliced - (vxfendg1) online
c0t5006016047201339d5s2 auto:sliced c0t5006016C47201339d5 vxfendg2 online
c0t5006016047201339d6s2 auto:sliced c0t5006016C47201339d6 vxfendg2 online
c0t5006016047201339d7s2 auto:sliced - - online
c2t0d0s2 auto:SVM - - SVM
c2t1d0s2 auto:SVM - - SVM
- - c0t5006016C47201339d7 vxfendg2 failed was:c0t5006016047201339d7s2
refadm2-oss1{root} #
Try:
vxdg -k -g <group> adddisk <disk>=<device>
vxdg -k -g vxfendg2 adddisk c0t5006016047201339d7s2=c0t5006016047201339d7s2
(We don't normally use Device name for Disk name, but it seems this is what you've done for other disks in vxfendg2).
Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links
You could:
1) disable fencing (/opt/VRTS/install/installvcs -fencing) and then enable fencing.
2) run vxfenswap to change fencing devices (https://sort.symantec.com/public/documents/sfha/6....)
Hello,
what does -k option specifies in below coomand
vxdg -k -g vxfendg2 adddisk c0t5006016047201339d7s2=c0t5006016047201339d7s2
I mean the use of -k option please.
re: vxdg -k option - from manpage:
https://sort.symantec.com/public/documents/sfha/5....
----------
KEYWORDS
adddisk
[...]
If the -k option is specified, the disk media name must represent a disk media record that was previously dissociated from its disk access record with -k rmdisk; otherwise, a new disk media record is created to represent the disk. With the -k option, plexes requiring recovery are flagged as stale.
----------
minor amendment to Marianne's command as the other dmnames don't use s2:
# vxdg -g vxfendg2 -k adddisk c0t5006016047201339d7=c0t5006016047201339d7s2
If the adddisk doesn't work, please provide the output from:
# vxdisk list c0t5006016047201339d7s2
If this post has helped you, please vote or mark as solution
I have added the disk with below command
vxdg -g vxfendg2 -o coordinator -kf adddisk c0t5006016C47201339d7=c0t5006016047201339d7s2.
a slight modification needed to add -o coordinator as its a coordinator diskgroup
Thanks all for the excellent support
Please mark Marianne's post as solution since she pointed out the command.
If this post has helped you, please vote or mark as solution
Would you like to reply?
Login or Register to post your comment.