Veritas File System 4.1 and 4.1 MP1 may cause Linux kernel panic on IA-64 systems, most likely during installation.

Article:TECH46739  |  Created: 2006-01-15  |  Updated: 2008-01-03  |  Article URL http://www.symantec.com/docs/TECH46739
Article Type
Technical Solution

Product(s)

Environment

Issue



Veritas File System 4.1 and 4.1 MP1 may cause Linux kernel panic on IA-64 systems, most likely during installation.

Solution



When a Veritas File System kernel module is being loaded, it attempts to find the last NUMA node using Linux kernel macro NODE_DATA. On IA-64 architecture, the return values from NODE_DATA for non existing NUMA nodes are not guaranteed to be set to NULL, contrary to the assumption made by Veritas File System (VxFS) 4.1 and 4.1M P1.  When the return value is not NULL, VxFS will not recognize that it has already seen the last NUMA node and will continue to look for one. As a result, it will eventually reference an incorrect memory address, causing a kernel panic.

This problem is most likely to occur during VxFS installation. After RPM files installation, the installer script will load the vxfs kernel module for the first time on that specific machine. If the false assumption is broken for this machine/configuration, a crash will occur. On the other hand, if the data structures are set up in a way that does not break the false assumption, then a crash will not be observed (possibly until a future activity, e.g. a configuration change, influences the kernel data structures in such a way that it ultimately breaks the assumption).

Panic strings from the crash dump:

crash> log -m
(snip)
<1>Unable to handle kernel paging request at virtual address 000000007fc9e798
<4>insmod[8987]: Oops 8813272891392 [1]
<4>Modules linked in: vxfs(U) vxspec(U) vxio(U) vxdmp(U) nls_utf8 loop iptable_nat ip_conntrack iptable_filter ip_tables md5 ipv6 autofs4 sunrpc sadump(U) ipmi_si ipmi_devintf ipmi_msghandler scsi_dump diskdump vfat fat dm_mod button uhci_hcd ehci_hcdshpchp tg3 e100 mii sg ext3 jbd lpfc scsi_transport_fc mptscsih mptbase sd_mod scsi_mod
<4>
<4>Pid: 8987, CPU 3, comm:               insmod
<4>psr : 0000101008126030 ifs : 8000000000000002 ip  : [<a00000020127ff40>]    Tainted: P
<4>ip is at vx_find_start_node_id+0xa0/0x160 [vxfs]
(snip)


This problem occurs only if all of the following conditions are present:

  • Veritas File System 4.1 or  4.1 MP1
  • IA-64 machine
  • Linux RHEL4 or later
  • System configuration results in an arrangement of Linux kernel data structure (NUMA nodes) which breaks an incorrect assumption made by Veritas File System (see detailed description above)

Given the last condition, a crash is most likely to occur during a File System 4.1 or 4.1 MP1 installation and is much less likely (though possible) to occur on systems which already have it installed and running.


Recommended Course of Action:

Action varies depending on whether the system already has Veritas File System 4.1 or 4.1 MP1 installed or not.

In case File System  4.1 or 4.1 MP1 is not installed:

Since this problem is most likely to occur during the installation of File System 4.1 or 4.1 MP1, this point patch should be installed without first installing 4.1 or 4.1 MP1.

1. Symantec Engineering has provided a point patch for this issue, 4.1 MP1 RP2, which can be downloaded by clicking on the Download Now link below.
a. Confirm that the check sum of the downloaded file corresponds to the following:

$ cksum vxfs4.1MP1RP2.tar_282282.gz
1075972406      7122590 vxfs4.1MP1RP2.tar_282282.gz

b. Extract the RPM files contained and confirm check sum values:

$ gzcat vxfs4.1MP1RP2.tar_282282.gz | tar xf -
$ cksum *.rpm
3199343768      3909047 VRTSvxfs-common-4.1.10.20-MP1RP2_RHEL4.i686.rpm
172905493       3286913 VRTSvxfs-platform-4.1.10.20-MP1RP2_RHEL4.ia64.rpm

2. Install the point patch following these steps:
a. Copy the VRTSvlic-3.02-012.ia64.rpm file from Storage Foundation 4.1 for Linux to the system

b. Install RHEL4 Update 2 including IA-32 Emulation Layer ("ia32el" package). See OS manual for details

c. Install RPMs manually:

# rpm -U VRTSvlic-3.02-012.ia64.rpm
# rpm -U VRTSvxfs-common-4.1.10.20-MP1RP2_RHEL4.i686.rpm
# rpm -U VRTSvxfs-platform-4.1.10.20-MP1RP2_RHEL4.ia64.rpm

d. Licence the products:

# vxlicinst -k "license-key"

e. (Optional) Install other Storage Foundation products following normal installation procedures

The Installer script (installer, installmp, etc.) will print warning messages about VRTSvxfscommon and VRTSvxfsplatform (examples below) when it checks the installed packages. It is safe to ignore them and continue with the installation, as the installer is simply notifying you that a version of VxFS  (this point patch, 4.1 MP1 RP2), newer than the release level of products found in the installation media/image, has already been installed.

Examples of Installer messages:

  "CPI WARNING V-9-0-0 SF version 4.1 includes VRTSvxfscommon version 4.1.00.10.  A more recent version of VRTSvxfscommon, 4.1.10.20, is already installed on sv5."
  "CPI WARNING V-9-0-0 In this situation VRTSvxfscommon version 4.1.10.20 will not be installed or downgraded on sv5."
  "SF version 4.1 may not operate correctly with this more recent rpm."
  "The VRTSvxfscommon rpm must be removed manually before version 4.1.00.10 can be installed."
  "Do you want to continue? [y,n,q,?] (n) y"


In case File System  4.1 or 4.1 MP1 is already installed:

Even if File System 4.1 or 4.1 MP is already installed and this problem has not been observed, it is still highly recommended to apply this point patch to eliminate the possibility of an outage in the future.

1. Follow steps 1-a and 1-b above in "In case File System 4.1 or 4.1 MP1 is not installed"

2. Install the point patch using these commands:

# rpm -U VRTSvxfs-common-4.1.10.20-MP1RP2_RHEL4.i686.rpm
# rpm -U VRTSvxfs-platform-4.1.10.20-MP1RP2_RHEL4.ia64.rpm

3. Reboot the system



Note: When using other Veritas Storage Foundation (tm) products together with Veritas File System, these products need to be of level corresponding to Storage Foundation 4.1 MP1. For example, Veritas Volume Manager (tm) 4.1MP1 should be used, not Volume Manager 4.1.


If you have not received this TechNote from the Symantec Email Notification Service as a Software Alert, please subscribe at the following link:



Download Now - 7122.59 K
File Name: vxfs4.1MP1RP2.tar_282282.gz
File Type: Patch

Attachments

vxfs4.1MP1RP2.tar_282282.gz (7 MBytes)

Supplemental Materials

SourceETrack
Value540263
DescriptionLxRT4.1: IA-64 panicking when discovering largest memory node


Legacy ID



282282


Article URL http://www.symantec.com/docs/TECH46739


Terms of use for this information are found in Legal Notices