Video Screencast Help

VCS 6.0.1 Solairs 11 Oracle 11Rg2 everything works fine now, except for....

Created: 15 May 2013 • Updated: 19 Jun 2013 | 11 comments
This issue has been solved. See solution.

getting this error message while offlining the oracle SG.

 

 

2013/05/15 10:43:31 VCS ERROR V-16-10001-14058 (solaris11-chi-2) Zone:z3_zone_res:offline:Command [/usr/sbin/zoneadm -z "z3" detach 2>&1] failed with output [umount: warning: /z3root/root not in mnttab
 umount: /z3root/root not mounted
 ERROR: unable to unmount /z3root/root.
 ERROR: Unable to unmount boot environment.
]
 
 
I am able to bring up the oracle resource on one of the nodes in a HA zone....
 
Please see main.cf attached....
Operating Systems:
Discussion Filed Under:

Comments 11 CommentsJump to latest comment

Paresh Bafna's picture

Hi,

 

Could you also share zone configuration file for zone 'z3'? It would be @ /etc/zones/z3.xml

Also, did you online the z3grp through VCS comepletely or manually outside VCS?

 

Thanks and Regards,
Paresh Bafna

dariuszz's picture

Okay, Thanks for responding. I am able to get zone/database/listener up on all nodes in the VCS cluster. I am testing hagrp -switch z5grp -to solaris11-chi-1/solaris11-chi-2

 

I reinstalled VCS, so 

 

However, the -switch, does not leave the zone in a configured state(when the SG is going offline), I have to manually set configured state  for zone on both nodes.

I am attaching the main.cf as main.txt. 

 

 

root@solaris11-chi-1:/# cat /etc/zones/index
# ident "%Z%%M% %I%     %E% SMI"
# Copyright 2005 Sun Microsystems, Inc.  All rights reserved.
# Use is subject to license terms.
#
# DO NOT EDIT: this file is automatically generated by zoneadm(1M)
# and zonecfg(1M).  Any manual changes will be lost.
#
global:installed:/
z5:installed:/z4root:71e7f013-acc1-eaa6-e0ab-8821d024bd57
root@solaris11-chi-1:/#
 
root@solaris11-chi-2:/# cat /etc/zones/index
# ident "%Z%%M% %I%     %E% SMI"
# Copyright 2005 Sun Microsystems, Inc.  All rights reserved.
# Use is subject to license terms.
#
# DO NOT EDIT: this file is automatically generated by zoneadm(1M)
# and zonecfg(1M).  Any manual changes will be lost.
#
global:installed:/
z5:installed:/z4root:79f123fc-f4fb-6c2c-98d5-b0732d3bd248
root@solaris11-chi-2:/#
 
root@solaris11-chi-2:/# hastatus -sum
 
-- SYSTEM STATE
-- System               State                Frozen
 
A  solaris11-chi-1      RUNNING              0
A  solaris11-chi-2      RUNNING              0
 
-- GROUP STATE
-- Group           System               Probed     AutoDisabled    State
 
B  z5grp           solaris11-chi-1      Y          N               OFFLINE
B  z5grp           solaris11-chi-2      Y          N               ONLINE
root@solaris11-chi-2:/#
 
root@solaris11-chi-2:/# cat /etc/zones/z5.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE zone PUBLIC "-//Sun Microsystems Inc//DTD Zones//EN" "file:///usr/share/lib/xml/dtd/zonecfg.dtd.1">
<!--
    DO NOT EDIT THIS FILE.  Use zonecfg(1M) instead.
-->
<zone name="z5" zonepath="/z4root" autoboot="true" brand="solaris" ip-type="exclusive" bootargs="-m verbose">
  <dataset name="orabin/orabin" alias="orabin"/>
  <dataset name="oradata/oradata" alias="oradata"/>
  <automatic-network lower-link="auto" linkname="net0" configure-allowed-address="true" link-protection="mac-nospoof" auto-mac-address="2:8:20:5b:1e:d6" mac-address="random"/>
</zone>
root@solaris11-chi-2:/#
 
 
 
 
AttachmentSize
main.txt 1.56 KB
dariuszz's picture

 

 umount: /z4root/root not mounted
 ERROR: unable to unmount /z4root/root.
 ERROR: Unable to unmount boot environment.
]
dariuszz's picture

My issue is this:

 

failover is not working. zone remains in "installed" state from the original node it fails FROM, it get's into "running" mode just fine on the node it fails TO, all services come up, database, listener...I have to manually edit the index file to change the zone from installed to configured state for failovber to work....

Paresh Bafna's picture

Hi,

Zone agent does have an attibute to control whether to detach zone root on offline/clean - DetachZonePath.

Though your main.cf shows that DetachZonePath has its default value which is 1, I would request you to check the value of DetachZonePath on running clsuter just to capture any unsaved configuration changes. You could check the value of this attribute by following command:

# /opt/VRTSvcs/bin/hares -value z5_zone DetachZonePath

Value should be 1. If it is not 1, please run following command to set it to 1.

# /opt/VRTSvcs/bin/hares -modify z5_zone DetachZonePath 1

If you see the issue that even after offline zone is in installed state, please enable debug logs for Zone agent and reproduce the issue. Debug log would help capture the exact error.

Debug logs can be enalbed using following command:

# /opt/VRTSvcs/bin/hatype -modify Zone LogDbg 1 2 3 4 5

 

Thanks and Regards,
Paresh Bafna

Paresh Bafna's picture

Also note that never manually edit any of the zone file like /etc/zones/index or /etc/zones/z5.xml.

If you want to manually change state of the zone from 'installed' to 'configured' run following command.

# /usr/sbin/zoneadm -z z5 detach

Manually editing these files may leave zone in inconsistent state.

Thanks and Regards,
Paresh Bafna

SOLUTION
dariuszz's picture

I wish you continued comfort and peace.Thank you for responding. All values check out, debug levels are set as you suggested.

 

I did a 

 

hagrp -switch z5grp -to solaris11-chi-2
 

 

and saw this in the engine log:

 

==============================================
Got notification for group: z5_zone
 
==============================================
 
2013/05/20 15:24:42 VCS ERROR V-16-10001-14058 (solaris11-chi-1) Zone:z5_zone:offline:Command [/usr/sbin/zoneadm -z "z5" detach 2>&1] failed with output [umount: warning: /z4root/root not in mnttab
 umount: /z4root/root not mounted
 ERROR: unable to unmount /z4root/root.
 ERROR: Unable to unmount boot environment.
]
 
 
 
Also, saw this:
 
2013/05/20 15:25:03 VCS INFO V-16-2-13716 (solaris11-chi-2) Resource(z5_pool): Output of the completed operation (online)
==============================================
cannot mount 'z4root' on '/z4root': directory is not empty
cannot import 'z4root': a pool with that name is already created/imported,
and no additional pools with that name were found
==============================================
 
 
I can delete the subdirectory called "root" under /z4root....
 
 
The good news is that it failed back to node 1, so this means that the "switch" operation left node 1  zone in "configured" state........
 
will try switching to node 2 again.....hope you read this....
 
dariuszz's picture

 

2013/05/20 15:36:13 VCS WARNING V-16-10001-20004 (solaris11-chi-2) Zpool:z5_pool:monitor:Warning: The filesystem z4root/rpool with mountpoint /rpool is not mounted. Administrative action may be required
 
dariuszz's picture

 

root@solaris11-chi-2:/# zfs list
NAME                              USED  AVAIL  REFER  MOUNTPOINT
orabin                           13.1G  20.4G    31K  /orabin
orabin/orabin                    13.1G  20.4G  13.1G  /orabin/orabin
oradata                           368K  33.0G    31K  /oradata
oradata/oradata                    32K  33.0G    32K  /oradata/oradata
rpool                            26.7G  39.8G  73.5K  /rpool
rpool/ROOT                       6.11G  39.8G    31K  legacy
rpool/ROOT/solaris               6.11G  39.8G  2.76G  /
rpool/ROOT/solaris/var           3.27G  39.8G  3.23G  /var
rpool/VARSHARE                    320K  39.8G   320K  /var/share
rpool/dump                       4.06G  39.9G  3.94G  -
rpool/export                       63K  39.8G    32K  /export
rpool/export/home                  31K  39.8G    31K  /export/home
rpool/swap                       16.5G  40.3G  16.0G  -
z4root                           8.00G  25.5G    33K  /z4root
z4root/rpool                     8.00G  25.5G    31K  /rpool
z4root/rpool/ROOT                8.00G  25.5G    31K  legacy
z4root/rpool/ROOT/solaris        8.00G  25.5G  2.95G  /
z4root/rpool/ROOT/solaris/var    5.04G  25.5G  5.04G  /var
z4root/rpool/VARSHARE              39K  25.5G    39K  /var/share
z4root/rpool/export               102K  25.5G    32K  /export
z4root/rpool/export/home           70K  25.5G    32K  /export/home
z4root/rpool/export/home/oracle    38K  25.5G    38K  /export/home/oracle
root@solaris11-chi-2:/#
 
dariuszz's picture

 

2013/05/20 15:47:49 VCS ERROR V-16-1-10205 Group z5grp is faulted on system solaris11-chi-2
2013/05/20 15:47:49 VCS NOTICE V-16-1-10446 Group z5grp is offline on system solaris11-chi-2
2013/05/20 15:47:49 VCS INFO V-16-1-10493 Evaluating solaris11-chi-1 as potential target node for group z5grp
2013/05/20 15:47:49 VCS INFO V-16-1-10493 Evaluating solaris11-chi-2 as potential target node for group z5grp
2013/05/20 15:47:49 VCS INFO V-16-1-50010 Group z5grp is online or faulted on system solaris11-chi-2
 
 
then it faults over to node 1 where it is in "installed" state and gets booted up there - under VCS control
dariuszz's picture

This is where things are now:

I reset ZoneResName to null ONLY on the zone root pool z5_pool.

Now, i can switch to node 2 to node2 but cannot switch back to node 1.

 

I get the following message:

 

 

2013/05/21 10:31:35 VCS INFO V-16-1-10298 Resource oradata_pool (Owner: Unspecified, Group: z5grp) is online on solaris11-chi-1 (VCS initiated)
2013/05/21 10:31:35 VCS NOTICE V-16-1-10301 Initiating Online of Resource z5_zone (Owner: Unspecified, Group: z5grp) on System solaris11-chi-1
2013/05/21 10:31:36 VCS DBG_1 V-16-10001-0 (solaris11-chi-1) Zone:z5_zone:online:None of the keys have a valid value. Configuration is not enabled for DR
2013/05/21 10:31:36 VCS DBG_1 V-16-10001-0 (solaris11-chi-1) Zone:z5_zone:online:Attribute WorkLoad not found in the argument list.
2013/05/21 10:31:37 VCS DBG_3 V-16-10001-0 (solaris11-chi-1) Zone:z5_zone:online:Zone [z5] is in [installed] state. Skipping zoneadm attach operation..
2013/05/21 10:31:37 VCS WARNING V-16-10001-14053 (solaris11-chi-1) Zone:z5_zone:online:The value of the BootState attribute is not set. Defaulting BootState to multi-user.
2013/05/21 10:31:44 VCS DBG_3 V-16-10001-0 (solaris11-chi-1) Zone:z5_zone:online:Command [/usr/sbin/zoneadm -z "z5" boot 2>&1] exited with output [zone 'z5': ERROR: no active dataset.
zone 'z5': ERROR: Unable to mount zone root dataset.
zoneadm: zone 'z5': call to zoneadmd failed
] and exitcode [1]