Video Screencast Help
Search Video Help Close Back
to help
Not able to make it to Vision this year? Get a sampling in the Best of Vision on Demand group.

veritas cluster Server Installation problem

Created: 29 Nov 2011 | 12 comments
Nagpal's picture
+1 1 Vote
Login to vote

Hi,

I am facing problem with Installation of Veritas Cluster Server 5.1 in solaris. Everytime when i install this package get below error ; -

-----------------------------------------------------------------------------------------------------------------------------------------------------------------

Starting VCS: 87%                                                                                    _____________

    Estimated time remaining: 1:10                                                                              7 of 8

    Performing VCS configuration ................................................................................ Done
    Starting llt ................................................................................................ Done
    Starting gab ................................................................................................ Done
    Starting vxfen .............................................................................................. Done
    Starting had ................................................................................................ Done
    Starting hashadow ........................................................................................... Done
    Starting CmdServer .......................................................................................... Done

Veritas Cluster Server Startup did not complete successfully

llt failed to start on sun1
gab failed to start on sun1
vxfen failed to start on sun1
had failed to start on sun1
hashadow failed to start on sun1

It is strongly recommended to reboot the following systems:
        sun1

Execute '/usr/sbin/shutdown -y -i6 -g0' to properly restart your systems

installvcs log files, summary file, and response file are saved at:

        /opt/VRTS/install/logs/installvcs-201111291507mZT

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

bash-3.00# cd /opt/VRTS/install/logs/installvcs-201111291507mZT
bash-3.00# ls
installvcs-201111291507mZT.log0         start.gab.sun1                          stop.gab.sun1
installvcs-201111291507mZT.log1         start.had.sun1                          stop.had.sun1
installvcs-201111291507mZT.qa_response  start.hashadow.sun1                     stop.hashadow.sun1
installvcs-201111291507mZT.response     start.llt.sun1                          stop.llt.sun1
installvcs-201111291507mZT.summary      start.vxfen.sun1                        stop.vxatd.sun1
start.CmdServer.sun1                    stop.CmdServer.sun1                     stop.vxfen.sun1
bash-3.00# more stop.CmdServer.sun1
1 15:10:05 BEGIN-proc_stop_sys(CmdServer,sun1)
1 15:10:05 ## Stopping CmdServer on sun1
1 15:10:05 ## Begin prestop checking for CmdServer on sun1
1 15:10:05 cmd LANG=C LC_ALL=C /usr/bin/ps -ef | /usr/bin/grep 'CmdServer' | /usr/bin/grep -v grep | /usr/bin/grep -v defu
nct 2>&1
1 15:10:05 cmd exit=1
1 15:10:05 ## The prestop checking return '0'
1 15:10:05 ## CmdServer is not running on sun1
1 15:10:05 ## The proc_stop_sys() return '1'
1 15:10:05 END-proc_stop_sys(CmdServer,sun1)
bash-3.00# more stop.gab.sun1
1 15:10:09 BEGIN-proc_stop_sys(gab,sun1)
1 15:10:09 ## Stopping gab on sun1
1 15:10:09 ## Begin prestop checking for gab on sun1
1 15:10:09 cmd LANG=C LC_ALL=C /usr/sbin/modinfo | /usr/bin/grep -w gab 2>&1
1 15:10:09 cmd exit=1
1 15:10:09 gab not loaded on sys sun1
1 15:10:09 ## The prestop checking return '0'
1 15:10:09 ## gab is not running on sun1
1 15:10:09 ## The proc_stop_sys() return '1'
1 15:10:09 END-proc_stop_sys(gab,sun1)
bash-3.00# more stop.had.sun1
1 15:10:01 BEGIN-proc_stop_sys(had,sun1)
1 15:10:01 ## Stopping had on sun1
1 15:10:01 ## Begin prestop checking for had on sun1
1 15:10:01 cmd LANG=C LC_ALL=C /usr/bin/ps -ef | /usr/bin/grep 'bin/had' | /usr/bin/grep -v grep | /usr/bin/grep -v defunc
t 2>&1
1 15:10:01 cmd exit=1
1 15:10:01 had not running on sun1
1 15:10:01 cmd LANG=C LC_ALL=C /usr/bin/ls /var/svc/manifest/system/vcs-onenode.xml 2>/dev/null
1 15:10:01 cmd exit=2
1 15:10:01 cmd LANG=C LC_ALL=C /usr/bin/ls /var/svc/manifest/system/vcs.xml 2>/dev/null
1 15:10:01 cmd stdout=
/var/svc/manifest/system/vcs.xml
1 15:10:01 cmd exit=0
1 15:10:01 cmd LANG=C LC_ALL=C /usr/bin/svcs -v -p /system/vcs | /usr/bin/awk '{print $1}' 2>&1
1 15:10:01 cmd stdout=
STATE
offline
1 15:10:01 cmd exit=0
1 15:10:01 ## The prestop checking return '0'
1 15:10:01 ## had is not running on sun1
1 15:10:01 ## The proc_stop_sys() return '1'
1 15:10:01 END-proc_stop_sys(had,sun1)bash-3.00# more stop.had.sun1
1 15:10:01 BEGIN-proc_stop_sys(had,sun1)
1 15:10:01 ## Stopping had on sun1
1 15:10:01 ## Begin prestop checking for had on sun1
1 15:10:01 cmd LANG=C LC_ALL=C /usr/bin/ps -ef | /usr/bin/grep 'bin/had' | /usr/bin/grep -v grep | /usr/bin/grep -v defunc
t 2>&1
1 15:10:01 cmd exit=1
1 15:10:01 had not running on sun1
1 15:10:01 cmd LANG=C LC_ALL=C /usr/bin/ls /var/svc/manifest/system/vcs-onenode.xml 2>/dev/null
1 15:10:01 cmd exit=2
1 15:10:01 cmd LANG=C LC_ALL=C /usr/bin/ls /var/svc/manifest/system/vcs.xml 2>/dev/null
1 15:10:01 cmd stdout=
/var/svc/manifest/system/vcs.xml
1 15:10:01 cmd exit=0
1 15:10:01 cmd LANG=C LC_ALL=C /usr/bin/svcs -v -p /system/vcs | /usr/bin/awk '{print $1}' 2>&1
1 15:10:01 cmd stdout=
STATE
offline
1 15:10:01 cmd exit=0
1 15:10:01 ## The prestop checking return '0'
1 15:10:01 ## had is not running on sun1
1 15:10:01 ## The proc_stop_sys() return '1'
1 15:10:01 END-proc_stop_sys(had,sun1)
bash-3.00# more stop.hashadow.sun1
1 15:10:03 BEGIN-proc_stop_sys(hashadow,sun1)
1 15:10:03 ## Stopping hashadow on sun1
1 15:10:03 ## Begin prestop checking for hashadow on sun1
1 15:10:03 cmd LANG=C LC_ALL=C /usr/bin/ps -ef | /usr/bin/grep 'hashadow' | /usr/bin/grep -v grep | /usr/bin/grep -v defun
ct 2>&1
1 15:10:03 cmd exit=1
1 15:10:03 ## The prestop checking return '0'
1 15:10:03 ## hashadow is not running on sun1
1 15:10:03 ## The proc_stop_sys() return '1'
1 15:10:03 END-proc_stop_sys(hashadow,sun1)
bash-3.00# more stop.llt.sun1
1 15:10:11 BEGIN-proc_stop_sys(llt,sun1)
1 15:10:11 ## Stopping llt on sun1
1 15:10:11 ## Begin prestop checking for llt on sun1
1 15:10:11 cmd LANG=C LC_ALL=C /usr/sbin/modinfo | /usr/bin/grep -w llt 2>&1
1 15:10:11 cmd exit=1
1 15:10:11 llt not loaded on sys sun1
1 15:10:11 ## The prestop checking return '0'
1 15:10:11 ## llt is not running on sun1
1 15:10:11 ## The proc_stop_sys() return '1'
1 15:10:11 END-proc_stop_sys(llt,sun1)
bash-3.00#
=========================================================================================================

 Above are log files prapared between failed installation .Please help me out this.

Thanks

Prince Nagpal
 

 

Comments

jdangelo_symc's picture
29
Nov
2011
0 Votes 0
Login to vote

One Node Cluster?

Prince,

Are you attempting to create a one node VCS cluster?  If so,  LLT and GAB will not be used. It would appear that the packages did install but the cluster is not configuring correctly. 

Can I presume you use the installvcs installation script? 

You can try to rerun the installer with the -installonly option.  This will skip over the configuration portion.

If that is successful, then you may want to try the installvcs -configure option.

Lastly,  a misconfigured license would also cause this issue.  Can you verify you have a valid key installed.

#> vxlicrep

You should see a Permanent VCS key applied. 

#> vxkeyless display

VCS

 

Joe D

Marianne van den Berg's picture
29
Nov
2011
0 Votes 0
Login to vote

More info please?

You have posted step 7 of 8.

Please share steps 1 - 6?

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows.
Handy NBU links

DMadhavan's picture
02
Dec
2011
0 Votes 0
Login to vote

 If you are not dealing with

 

If you are not dealing with a single node cluster here :-

All the #more outputs here are from the stop function.. Please paste the output of start function as well.

 

 Also it helps if you can provide the following file outputs as well.

cd /opt/VRTS/install/logs/

Here locate the latest installvcs log directory (look for the directory with time stamp matching with the time you tried above installation steps)

There you will see install logs for each package, say llt,gab,vxfen,vcsag etc..Pick log files of llt,gab and paste the reason for any package installation failure logged in those files here.

If this is a solaris system,provide outputs of 

1.pkginfo -l VRTSllt

2.pkginfo -l VRTSgab

3. ifconfig -a

4. cat path_to_inst |grep <your NIC driver instance for LLT device>

5.cat /etc/llttab and /etc/llthosts

I will also try to start the llt and gab modules manually by the invoking the start scripts in /etc/rc2.d, and see whats stopping them from loading up (provided packages are installed properly. verify those pkgadd's gave a non error return code in the above log file)

Test LLT interfaces with a private IP (lets say 80.0.0.1 for node a and 80.0.0.2 for node b - Make sure no one has that IP before you do this) and try ping'ing each other. You can also test kstat to see if the interface link is up or not. another utility called lltping/llttest under /opt/VRTSllt which are useful to test the LLT traffic.

If llt fails all fails, So start troubleshooting LLT first.

 

Good Luck!!

Nagpal's picture
23
Jan
2012
0 Votes 0
Login to vote

Hi Joe D,

Hi Joe D,

i am trying with one node cluster and two node cluster, but facing same issue on both installation.
installonly script runs properly and installed vcs successfully, but getting error while configuring the vcs.

These are thr output abour licence of VCS
bash-3.2# vxlicrep

 License Key                           = XXXXXXXXXXXXXXXXXXXXXXXXXXX
   Product Name                        = VERITAS Cluster Server
   Serial Number                       = 8546
   License Type                        = PERMANENT
   OEM ID                              = 2006
   Site License                        = YES
   Editions Product                    = YES

 Features :=
   Platform                            = Unused
   Version                             = 5.1
   Tier                                = Unused
   Reserved                            = 0

   Mode                                = VCS
   VXKEYLESS                           = Enabled

bash-3.2# vxkeyless display
VCS

Nagpal's picture
23
Jan
2012
0 Votes 0
Login to vote

Hi Marianne van den Berg,

Hi

Please reffer to the attachemnt of installation of VCS while i am getting error while configuring the VCS on single Node.

 

Thanks

Prince Nagpal

AttachmentSize
installation.txt 16.58 KB
Nagpal's picture
23
Jan
2012
0 Votes 0
Login to vote

Hi DMadhavan

Hi

Please reffer to the attachemnt of installation as attached on above post of VCS while i am getting error while configuring the VCS on single Node.

Also attached the output of below commands in above attachment.

1.pkginfo -l VRTSllt

2.pkginfo -l VRTSgab

3. ifconfig -a

4. cat path_to_inst |grep <your NIC driver instance for LLT device>

5.cat /etc/llttab and /etc/llthosts

 

Thanks

 

 

 

Gaurav Sangamnerkar's picture
23
Jan
2012
0 Votes 0
Login to vote

Can you paste output of #

Can you paste output of

# modinfo |egrep 'llt|gab'

# lltconfig

# lltstat -nv v | head

 

Also what is the error you are getting when you are trying to start LLT manually using /etc/rc2.d script ?

 

G

PS: If you are happy with the answer provided, please mark the post as solution. You can do so by clicking link "Mark as Solution" below the answer provided.
 

Nagpal's picture
23
Jan
2012
0 Votes 0
Login to vote

Hi Gaurav,

These are outputs of commands:-

bash-3.2# modinfo |egrep 'llt|gab'
bash-3.2#
bash-3.2# pkginfo -l |grep -i llt
   PKGINST:  VRTSllt
bash-3.2#  pkginfo -l |grep -i gab
      NAME:  Broadcom NetXtreme II Gigabit Ethernet Adapter Driver
      DESC:  Broadcom NetXtreme II Gigabit Ethernet PCI-X/PCIE Adapter Driver
      DESC:  Broadcom NetXtreme II 10 Gigabit Ethernet PCI-X/PCIE Adapter Driver
      NAME:  SysKonnect SK-NET Gigabit Ethernet Adapter SK-98xx
      DESC:  SysKonnect SK-NET Gigabit Ethernet Adapter SK-98xx
      DESC:  Chelsio N110 10 Gigabit Ethernet Network Adapter Driver
      DESC:  Pluggable Authentication Module for Smart Card Authentication
      NAME:  Realtek Gigabit Ethernet Driver
      DESC:  Realtek Gigabit Ethernet Network Adapter Driver
      DESC:  Neterion Xframe (I/II) 10 Gigabit Ethernet Adapter Driver
   PKGINST:  VRTSgab
bash-3.2# lltconfig
LLT lltconfig ERROR V-14-2-15000 open /dev/llt failed: No such file or directory
bash-3.2# lltstat -nv v | head
LLT lltstat ERROR V-14-2-15000 open /dev/llt failed: No such file or directory

######There is no entry in /etc/rc2.d for llt ############

bash-3.2# cd /etc/rc2.d
bash-3.2# ls
K05appserv          K16apache           S10lu               S42ncakmod          S72autoinstall      S89PRESERVE
K06mipagent         K27boot.server      S20sysetup          S47pppd             S73cachefs.daemon   S94ncalogd
K15imq              README              S40llc2             S70uucp             S81dodatadm.udaplt  S98deallocate

Anoop_Kumar's picture
23
Jan
2012
1 Vote +1
Login to vote

The problem exists with

The problem exists with /dev/llt and /dev/gab not created. These are required modules to start/stop llt and gab.

I will advise to install VRTSllt and VRTSgab manually using pkgadd and share the results. /dev/llt and /dev/gab should be created, if not, then need to troubleshoot more.

~Anoop

Gaurav Sangamnerkar's picture
23
Jan
2012
0 Votes 0
Login to vote

Ideally I would expect S70llt

Ideally I would expect S70llt & S92gab to be created when the packages are added ..

so we see above that package has been added , though can u check the consistency..

# pkgchk VRTSllt

# pkgchk VRTSgab

If you see any inconsistency reported, would suggest to resinstall the package as Anoop said above & share the results..

G

PS: If you are happy with the answer provided, please mark the post as solution. You can do so by clicking link "Mark as Solution" below the answer provided.
 

Nagpal's picture
09
Feb
2012
0 Votes 0
Login to vote

Hi

Hi have re-installed both the packages again but still facing same problem i.e llt and gab not created in rc2.d. Its solaris 10 i think llt and gab managed through SMF in solaris 10.

Nagpal's picture
09
Feb
2012
0 Votes 0
Login to vote

consistency

bash-3.2# pkgchk VRTSllt
ERROR: /etc/default/llt
    permissions <0744> expected <0644> actual
    group name <sys> expected <root> actual
    modtime <02/05/09 11:29:32 AM> expected <01/25/12 12:04:14 AM> actual
    file cksum <20718> expected <20716> actual
bash-3.2# pkgchk VRTSgab
ERROR: /etc/default/gab
    permissions <0744> expected <0644> actual
    group name <sys> expected <root> actual
    modtime <02/05/09 11:30:37 AM> expected <01/25/12 12:04:14 AM> actual
    file cksum <20514> expected <20512> actual