How To upgrade your linux kernel when the boot drive in encapsulated.

Article:HOWTO62220  |  Created: 2011-12-14  |  Updated: 2012-07-28  |  Article URL http://www.symantec.com/docs/HOWTO62220
Article Type
How To




Starting in version 5.1SP1, there is functionality to upgrade the OS kernel  without unencapsulating. 

* The new kernel version cannot break kABI with the previous version.

* See the Admin Guide

//// Per the vxvm admin guide

1 Apply the minor upgrade or patch to the system.

2 After applying the upgrade, run the commands:
# . /etc/vx/modinst-vxvm
# upgrade_encapped_root


3 reboot.


Example, with workaround when applying all versions 5.1SP1RP2P1HF4. The recommended method is to install the VRTSvxvm 5.1SP1RP2P2HF4 rpm directly on top of 5.1SP1.  However, both methods are given here for completeness.  See  Upgrade of Volume Manager on 5.1SP1 systems with encapsulated root corrupts OS file. 

Here is a complete transcript of the upgrade procedure from 5.1SP1, to the hotfix, with the OS upgrade without encapsulation.


1. No VRTS rpms installed:
[root@icculus ~]# rpm -qa | grep VRTS
[root@icculus ~]#
?
2. Install 5.1 SP1.
[root@icculus rhel5_x86_64]# ./installer
# uname -a
Linux icculus.spt.spr.symantec.com 2.6.18-194.el5 #1 SMP Tue Mar 16 21:52:39 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
Installed VCS 1 node cluster.
[root@icculus ~]# rpm -qa | grep kernel
kernel-2.6.18-194.el5
[root@icculus ~]#
?
3. encap root disk:
VxVM vxencap INFO V-5-2-1923 Encapsulating root disk...
The sda disk has been configured for encapsulation.
-At this point we are at the same place of the customer systems.-

4. Upgrade to the 5.1SP1RP2P2HF4 VRTSvxvm rpm directly on top of 5.1SP1.
[root@icculus 51SP1RP2HF4]# rpm -Uvh VRTSvxvm-5.1.132.204-5.1SP1RP2P2HF4_RHEL5.x86_64.rpm
Preparing... ########################################### [100%]
Found encapsulated root-disk. Doing in-place upgrade.
Please reboot system immediately after upgrade.
stopping vxrelocd
stopping vxattachd
stopping vxcached
stopping vxconfigbackupd
stopping vxnotify
stopping vxesd
stopping vxconfigd
1:VRTSvxvm ########################################### [100%]
Installing file /etc/init.d/vxvm-boot
creating VxVM device nodes under /dev
Recreating links in /lib/modules/
Installing keys for APMs
Backed up /boot/VxVM_initrd.img as /boot/VxVM_initrd.img.b4vmupgrade
Generating VxVM initrd image...
Copied new VxVM_initrd.img
Note: Failsafe entry "recover_vxvm_root" added to file grub.conf
In case the system fails to boot with new VxVM_initrd, please
interrupt bootup sequence at grub menu and select entry #3
[root@icculus 51SP1RP2HF4]#
-- reboot --


5. Now safe to perform kernel upgrade without unencapsulating.
[root@icculus rhel57]# rpm -ivh kernel-2.6.18-274.el5.x86_64.rpm
warning: kernel-2.6.18-274.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing... ########################################### [100%]
1:kernel ########################################### [100%]
[root@icculus rhel57]#
6. Update the VX ram disk and grub entry. By this time grub.conf is getting a little crowded.
[root@icculus rhel57]# cd /etc/vx
[root@icculus vx]# . ./modinst-vxvm
[root@icculus vx]# upgrade_encapped_root
umount: /tmp/vx.30723.3155.10805.14434: not mounted
Created /tmp/vx.4741.677.2328.14434/img
44779 blocks
The VxVM root encapsulation upgrade has succeeded.
Please reboot the machine to load the new kernel.
?
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/sda2
# initrd /initrd-version.img
#boot=/dev/sda
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
#default=0
?
#vxvm_root_default_START ( do not remove)
# Default menu entry number has been set to vxvm_root.
# - the vxvm_root default entry number is: 2
# - the original default entry number is: 0
# - the selected default entry number is: 0
# - the original grub configuration is in: /boot/grub/menu.lst.b4vxvm
default=5
#vxvm_root_default_END ( do not remove)
title Red Hat Enterprise Linux Server (2.6.18-274.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-274.el5 root=/dev/sda2 ro rhgb quiet
initrd /initrd-2.6.18-274.el5.img
title Red Hat Enterprise Linux Server (2.6.18-194.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-194.el5 ro root=/dev/sda2 rhgb quiet crashkernel=128M@16M
initrd /initrd-2.6.18-194.el5.img
#vxvm_backup_root_START ( do not remove)
title vxvm_root_backup_backup
root (hd0,0)
kernel /vmlinuz-2.6.18-194.el5 root=/dev/sda2 ro rhgb quiet
initrd /VxVM_initrd-2.6.18-194.el5-2.6.18-194.el5.img
#vxvm_backup_root_END ( do not remove)
?
#vxvm_backup_root_START ( do not remove)
title vxvm_root_backup
root (hd0,0)
kernel /vmlinuz-2.6.18-194.el5 root=/dev/vx/dsk/bootdg/rootvol ro rhgb quiet
initrd /VxVM_initrd-2.6.18-194.el5.img
#vxvm_backup_root_END ( do not remove)
#recover_vxvm_root_START ( do not remove )
title recover_vxvm_root
root (hd0,0)
kernel /vmlinuz-2.6.18-194.el5 root=/dev/sda2 ro rhgb quiet
initrd /VxVM_initrd.img.b4vmupgrade
#recover_vxvm_root_END ( do not remove )
#vxvm_root_START (do not remove)
title vxvm_root
root (hd0,0)
kernel /vmlinuz-2.6.18-274.el5 root=/dev/vx/dsk/bootdg/rootvol ro rhgb quiet
initrd /VxVM_initrd.img
#vxvm_root_END (do not remove)
?
?
Note that KDUMP will need to be set up again if it was implemented before the VxVM or kernel upgrades.
/sbin/nash is undisturbed:
[root@icculus ~]# ls -l /sbin/nash
-rwxr-xr-x 1 root root 2419232 Dec 12 11:04 /sbin/nash

6. reboot
[root@icculus ~]# uname -a
Linux icculus.spt.spr.symantec.com 2.6.18-274.el5 #1 SMP Fri Jul 8 17:36:59 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
[root@icculus ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/vx/dsk/rootdg/rootvol
40375376 5232840 33058504 14% /
/dev/vx/dsk/rootdg/tmpvol
40304324 181040 38042872 1% /tmp

?
7. For extra credit, install 5.1SP1RP2 with the 'installrp script' if desired.
RP2 installed just fine. It just skips the VRTSvxvm install since installrp found a newer version. So no manual fixing of /sbin/nash is needed.
?
-- reboot --
?
[root@icculus ~]# rpm -qa | grep VRTS
VRTSsfmh-3.1.830.0-0
VRTScps-5.1.132.000-SP1RP2_RHEL5
VRTSvlic-3.02.51.010-0
VRTSvxfen-5.1.132.000-SP1RP2_RHEL5
VRTSperl-5.10.0.7-RHEL5.3
VRTSatClient-5.0.32.0-0
VRTSodm-5.1.100.000-SP1_GA_RHEL5
VRTSob-3.4.312-0
VRTSfssdk-5.1.132.000-SP1RP2o_RHEL5
VRTSamf-5.1.132.000-SP1RP2_RHEL5
VRTSvcsdr-5.1.132.000-SP1RP2_RHEL5
VRTSspt-5.5.000.005-GA
VRTSatServer-5.0.32.0-0
VRTSlvmconv-5.1.132.000-SP1RP2_RHEL5
VRTSllt-5.1.132.000-SP1RP2_RHEL5
VRTSvcs-5.1.132.000-SP1RP2_RHEL5
VRTSvcsea-5.1.132.000-SP1RP2_RHEL5
VRTSvxvm-5.1.132.204-5.1SP1RP2P2HF4_RHEL5
VRTSgab-5.1.132.000-SP1RP2_RHEL5
VRTSdbed-5.1.132.000-SP1RP2_RHEL5
VRTSaslapm-5.1.100.000-SP1_RHEL5
VRTSvxfs-5.1.132.000-SP1RP2o_RHEL5
VRTSvcsag-5.1.132.000-SP1RP2_RHEL5

 

Otherwise, use the Technote workaround if you apply any of the 5.1SP1RP2, 5.1SP1RP2P1 or 5.1SP1RP2P2:

 

1.  Fresh OS install.
Install Version 5.1 SP1:
VRTSvxvm-5.1.100.000-SP1_RHEL5

2. Encapsulate Root disk:


 VxVM vxencap INFO V-5-2-1923 Encapsulating root disk...

 VxVM vxencap INFO V-5-2-2123 Verifying generated partition-table...

 VxVM vxencap INFO V-5-2-4086 Checking private region location of root disk ...

 VxVM vxencap INFO V-5-2-4085 An old private region was found on the root disk and
 has been disabled.

 VxVM vxencap INFO V-5-2-1946 Generating VxVM initrd image...
Created /tmp/vx.29575.859.24098.14457/vx.20932.12043.23258.14457

 NOTE: Until the next reboot has completed, do not modify any 'grub'
 settings, otherwise your system might become unbootable!

 VxVM vxencap INFO V-5-2-2414 Adding vxvm_root entry to 'grub'...

 VxVM vxencap INFO V-5-2-2605 The grub 'default' boot entry is:

       0. "title Red Hat Enterprise Linux Server (2.6.18-194.el5)"

 Do you want to use this menu entry for the encapsulation process?
[y,n,q,?] (default: y)

 VxVM vxencap INFO V-5-2-2689 saving /boot/grub/menu.lst to backup file /boot/grub/menu.lst.b4vxvm ...

 VxVM vxencap INFO V-5-2-4487 converting volume labels in /boot/grub/menu.lst to explicit devices ...

 VxVM vxencap INFO V-5-2-2696 updating /boot/grub/menu.lst ...

 VxVM vxencap INFO V-5-2-2377 Please save /etc/vx/rootdisk_info.icculus.spt.spr.symantec.com
 on some other server. Information in this file can be useful for recovery in
 case of unknown failures.

 The sda disk has been configured for encapsulation.

 The first stage of encapsulation has completed successfully.  You
 should now reboot your system at the earliest possible opportunity.

 VxVM  NOTICE V-5-2-2044
The encapsulation will require two or three reboots which will happen
 automatically after the next reboot.  To reboot execute the command:

shutdown -r now

 This will update the /etc/fstab file so that volume devices are
 used to mount the file systems on this disk device.  You will need
 to update any other references such as backup scripts, databases,
 or manually created swap devices.

[root@icculus ~]# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/vx/dsk/rootdg/rootvol
                     40375376   3945220  34346124  11% /
/dev/vx/dsk/rootdg/tmpvol
                     40304324    180336  38043576   1% /tmp
/dev/vx/dsk/rootdg/bootvol
                      1011448     38544    920696   5% /boot
tmpfs                  1029508         0   1029508   0% /dev/shm
tmpfs                        4         0         4   0% /dev/vx



3.  Apply RP2.

Veritas Storage Foundation Install completed successfully...

4.  Apply RP2P2 patch.  FAIL:

VRTSvxvm-5.1.132.200-SP1RP2P2_RHEL5.x86_64.rpm
[root@icculus rpms]# rpm -Uvh *
Preparing...                ########################################### [100%]
Found encapsulated root-disk. Doing in-place upgrade.
Please reboot system immediately after upgrade.
stopping vxrelocd
stopping vxattachd
stopping vxcached
stopping vxconfigbackupd
stopping vxnotify
stopping vxesd
stopping vxconfigd
  1:VRTSvxvm               ########################################### [100%]
Installing file /etc/init.d/vxvm-boot
creating VxVM device nodes under /dev
Recreating links in /lib/modules/
Installing keys for APMs
Backed up /boot/VxVM_initrd.img as /boot/VxVM_initrd.img.b4vmupgrade
Generating VxVM initrd image...
cp: cannot create regular file `/tmp/vx_initrd/sbin/': Is a directory

 VxVM vxinitrd ERROR V-5-2-2489 Error copying files. Try again
Error in generating vxvm initrd image
ERROR: Error while creating new VxVM-initrd.imgerror: %post(VRTSvxvm-5.1.132.200-SP1RP2P2_RHEL5.x86_64) scriptlet failed, exit status 1


Workaround.  Copy /sbin/nash from a stock RHEL system:

scp /sbin/nash 10.167.18.214:/sbin

Retry:

-rwxr-xr-x 1 root root 27162558 Oct 21 04:52 VRTSvxvm-5.1.132.200-SP1RP2P2_RHEL5.x86_64.rpm
[root@icculus rpms]#
[root@icculus rpms]# set -o vi
[root@icculus rpms]# rpm -Uvh --force VRTSvxvm-5.1.132.200-SP1RP2P2_RHEL5.x86_64.rpm
Preparing...                ########################################### [100%]
ERROR: Pending system reboot from previous VxVM package upgrade.
Please reboot the system to complete the previous VxVM package upgrade.
You will need to re-initiate this upgrade.
error: %pre(VRTSvxvm-5.1.132.200-SP1RP2P2_RHEL5.x86_64) scriptlet failed, exit status 1
error:   install: %pre scriptlet failed (2), skipping VRTSvxvm-5.1.132.200-SP1RP2P2_RHEL5


-- reboot --

Reinstall:

[root@icculus rpms]# rpm -Uvh VRTSvxvm-5.1.132.200-SP1RP2P2_RHEL5.x86_64.rpm
Preparing...                ########################################### [100%]
       package VRTSvxvm-5.1.132.200-SP1RP2P2_RHEL5.x86_64 is already installed
[root@icculus rpms]# rpm -Uvh --force VRTSvxvm-5.1.132.200-SP1RP2P2_RHEL5.x86_64.rpm
Preparing...                ########################################### [100%]
Found encapsulated root-disk. Doing in-place upgrade.
Please reboot system immediately after upgrade.
stopping vxrelocd
stopping vxattachd
stopping vxcached
stopping vxconfigbackupd
stopping vxnotify
stopping vxesd
stopping vxconfigd
  1:VRTSvxvm               ########################################### [100%]
Installing file /etc/init.d/vxvm-boot
creating VxVM device nodes under /dev
Recreating links in /lib/modules/
Installing keys for APMs
Backed up /boot/VxVM_initrd.img as /boot/VxVM_initrd.img.b4vmupgrade
Generating VxVM initrd image...
Copied new VxVM_initrd.img
Note: Failsafe entry "recover_vxvm_root" added to file grub.conf
In case the system fails to boot with new VxVM_initrd, please
  interrupt bootup sequence at grub menu and select entry #3

[root@icculus rpms]# rpm -qa VRTSvxvm
VRTSvxvm-5.1.132.200-SP1RP2P2_RHEL5
[root@icculus rpms]#

MANUALLY FIX /sbin/nash AGAIN as it is munged again.

[root@icculus 51SP1RP2P2]# ls -l /sbin/nash
-rwxr-xr-x 1 root root 101952 Dec  9 09:33 /sbin/nash

-----------------------

Install the hotfix:

[root@icculus 51SP1RP2HF4]# rpm -Uvh VRTSvxvm-5.1.132.204-SP1RP2P2HF4_RHEL5.x86_64.rpm
Preparing...                ########################################### [100%]
ERROR: Pending system reboot from previous VxVM package upgrade.
Please reboot the system to complete the previous VxVM package upgrade.
You will need to re-initiate this upgrade.
error: %pre(VRTSvxvm-5.1.132.204-SP1RP2P2HF4_RHEL5.x86_64) scriptlet failed, exit status 1
error:   install: %pre scriptlet failed (2), skipping VRTSvxvm-5.1.132.204-SP1RP2P2HF4_RHEL5

< SHUTDOWN -R >

[root@icculus 51SP1RP2HF4]# rpm -Uvh VRTSvxvm-5.1.132.204-SP1RP2P2HF4_RHEL5.x86_64.rpm
Preparing...                ########################################### [100%]
Found encapsulated root-disk. Doing in-place upgrade.
Please reboot system immediately after upgrade.
stopping vxrelocd
stopping vxattachd
stopping vxcached
stopping vxconfigbackupd
stopping vxnotify
stopping vxesd
stopping vxconfigd
  1:VRTSvxvm               ########################################### [100%]
Installing file /etc/init.d/vxvm-boot
creating VxVM device nodes under /dev
Recreating links in /lib/modules/
Installing keys for APMs
Backed up /boot/VxVM_initrd.img as /boot/VxVM_initrd.img.b4vmupgrade
Generating VxVM initrd image...
Copied new VxVM_initrd.img
Note: Failsafe entry "recover_vxvm_root" added to file grub.conf
In case the system fails to boot with new VxVM_initrd, please
  interrupt bootup sequence at grub menu and select entry #4

(fix /sbin/nash again)

Apply the TEST .5x vxvm hostfix from Kevin:  (Thanks Kevin!!!)

[root@icculus 51SP1RP2HF4]# cp modinst-vxvm.5x /etc/vx/modinst-vxvm
cp: overwrite `/etc/vx/modinst-vxvm'? y
[root@icculus 51SP1RP2HF4]#

<reboot for good measure...>

Next, upgrade the kernel...

[root@icculus rhel57]# rpm -ivh kernel-headers-2.6.18-274.el5.x86_64.rpm
warning: kernel-headers-2.6.18-274.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
  1:kernel-headers         ########################################### [100%]
[root@icculus rhel57]# rpm -ivh kernel-2.6.18-274.el5.x86_64.rpm
warning: kernel-2.6.18-274.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
  1:kernel                 ########################################### [100%]


Run the new modinst-vxvm script:

[root@icculus vx]# . ./modinst-vxvm
[root@icculus vx]# upgrade_encapped_root
umount: /tmp/vx.22299.30700.13252.14377: not mounted
Created /tmp/vx.10801.13200.20975.14377/img
44777 blocks
The VxVM root encapsulation upgrade has succeeded.
Please reboot the machine to load the new kernel.

[root@icculus vx]# cat /boot/grub/grub.conf
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/sda2
#          initrd /initrd-version.img
#boot=/dev/sda
#default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu

#vxvm_root_default_START ( do not remove)
# Default menu entry number has been set to vxvm_root.
#  - the vxvm_root default entry number is: 1
#  - the original  default entry number is: 0
#  - the selected  default entry number is: 0
#  - the original grub configuration is in: /boot/grub/menu.lst.b4vxvm
default=6
#vxvm_root_default_END ( do not remove)

title Red Hat Enterprise Linux Server (2.6.18-274.el5)
       root (hd0,0)
        kernel /vmlinuz-2.6.18-274.el5 root=/dev/sda2 ro rhgb quiet
       initrd /initrd-2.6.18-274.el5.img

title Red Hat Enterprise Linux Server (2.6.18-194.el5)
       root (hd0,0)
        kernel /vmlinuz-2.6.18-194.el5 ro root=/dev/sda2 rhgb quiet crashkernel=128M@16M
       initrd /initrd-2.6.18-194.el5.img

#vxvm_backup_root_START ( do not remove)
title vxvm_root_backup
       root (hd0,0)
        kernel /vmlinuz-2.6.18-194.el5 root=/dev/vx/dsk/bootdg/rootvol ro rhgb quiet
       initrd /VxVM_initrd-2.6.18-194.el5.img
#vxvm_backup_root_END ( do not remove)

#recover_vxvm_root_START ( do not remove )
title recover_vxvm_root
       root (hd0,0)
        kernel /vmlinuz-2.6.18-194.el5 root=/dev/sda2 ro rhgb quiet
       initrd /VxVM_initrd.img.b4vmupgrade
#recover_vxvm_root_END ( do not remove )

#recover_vxvm_root_START ( do not remove )
title recover_vxvm_root
       root (hd0,0)
        kernel /vmlinuz-2.6.18-194.el5 root=/dev/sda2 ro rhgb quiet
       initrd /VxVM_initrd.img.b4vmupgrade
#recover_vxvm_root_END ( do not remove )

#recover_vxvm_root_START ( do not remove )
title recover_vxvm_root
       root (hd0,0)
        kernel /vmlinuz-2.6.18-194.el5 root=/dev/sda2 ro rhgb quiet
       initrd /VxVM_initrd.img.b4vmupgrade
#recover_vxvm_root_END ( do not remove )
#vxvm_root_START (do not remove)
title vxvm_root
       root (hd0,0)
        kernel /vmlinuz-2.6.18-274.el5 root=/dev/vx/dsk/bootdg/rootvol ro rhgb quiet
       initrd /VxVM_initrd.img

#vxvm_root_END (do not remove)


<Shutdown -r now>
Errors about /sbin/nash upon reboot.

/sbin/nash is again corrupted.

root@icculus ~]# ls -l /sbin/nash
-rwxr-xr-x 1 root root 101952 Dec  9 10:20 /sbin/nash

Fixed it up again and rebooted.



This worked, but manualy intervention was required to fix the /sbin/nash that was corrupted by the 5.1SP1RP2P2 patch.

 

 




Article URL http://www.symantec.com/docs/HOWTO62220


Terms of use for this information are found in Legal Notices