Intermittent failures and possible system crashes are known to occur when using Storage Foundation for UNIX, (SF/HA) Veritas Cluster Server, (VCS) Storage Foundation for Oracle RAC, (SFRAC) and Storage Foundation Cluster File Server (SFCFS) with certain AIX platforms.

Article:TECH58172  |  Created: 2008-01-01  |  Updated: 2011-01-23  |  Article URL http://www.symantec.com/docs/TECH58172
Article Type
Technical Solution

Product(s)

Environment

Issue



Intermittent failures and possible system crashes are known to occur when using Storage Foundation for UNIX, (SF/HA) Veritas Cluster Server, (VCS) Storage Foundation for Oracle RAC, (SFRAC) and Storage Foundation Cluster File Server (SFCFS) with certain AIX platforms.


Solution





Problem: System Crashes/Hangs in one of the drivers of the cluster solutions

System crash/hang will occur with any version of clustering products based on 4.0 or 5.0 releases. The Service Pack releases of AIX with the patch levels specified in the following table are the levels of AIX which result in the crash.

 

O/S Level Storage Foundation Veritas Cluster Server Storage Foundation High Availability Storage Foundation Cluster File Server Storage Foundation Oracle RAC
AIX 5.2 TL10 SP5 NO YES YES YES YES
AIX 5.3 TL6 SP3 NO YES YES YES YES
AIX 5.3 TL7 SP2 NO YES YES YES YES
AIX 6.1 Base SP3 NO YES YES YES YES



If your system is upgraded to or installed at the specified levels or above, you are impacted. You must follow the instructions provided below to recover. The specific IBM AIX APAR triggering this LLT panic has the abstract: Bring DLPI driver "to SPEC" applies to AIX - enhance DLPI driver to comply with specifications or normal streams processing. This APAR is installed as:

AIX OS Level AIX APAR
5200-10 IZ19838
5300-06 IZ05430
5300-07 IZ11726
6100-00 IZ13304




Problem and fix description

Changes to the DLPI driver with the latest TLs on AIX 5.3 and 6.1 result in upcalls to LLT in the interrupt context instead of the process context. Some LLT locks are not interrupt safe which caused panics. Some upcalls panic in GAB where it does an xmalloc in interrupt context.

All LLT locks on AIX are now interrupt safe. Out-of order lock releases are handled correctly with the new interrupt safe locks. Link notifications are provided for the client in the thread or process context instead of the interrupt context. This will ensure that no changes are required for LLT's clients like GAB/LMX thus abstracting all the changes to just one driver.


Impacted releases and corresponding LLT patches

Release Patch
4.0MP3 4.0MP3+e1274390_llt_b
4.0MP4 4.0MP4+e1274390_llt
5.0 5.0+e1274390_llt
5.0MP1 5.0MP1_5.0MP1EXT+e1274390_llt
5.0MP1-AIX6.1 5.0MP1_5.0MP1EXT+e1274390_llt



The corresponding LLT patches are found at Patch Central below and under the Related Documents section.

4.0 MP3    https://sort.symantec.com/patch/detail/2106

4.0 MP4    https://sort.symantec.com/patch/detail/2107

5.0            https://sort.symantec.com/patch/detail/2104

5.0 MP1 and 5.0 MP1 Update 1  https://sort.symantec.com/patch/detail/2103


Recommended procedure for an existing setup with panics

- Install and configure the appropriate patch for your MP


Recommended procedure for an existing setup without panics

- Install and configure the patch for the appropriate release
  (follow directions in the README)
- Upgrade to the latest AIX release when ready.


Recommended procedure for a fresh install

- Use installonly to install the base release
- Upgrade to the latest MP
- Install the required llt patch, then reboot the system.
- Use the configure option to configure the cluster-server.


NOTE#
 The removal of IBM temporary patch. (Workaround)

Once the LLT patch is installed as above, the workaround patch can be uninstalled, if it was installed.

To uninstall the workaround patch:

For AIX 6.1:
   # emgr -rL 61TL00dlpi
For AIX 5.3 TL6:
   # emgr -rL 53TL06dlpi
For AIX 5.3 TL7:
   # emgr -rL 53TL07dlpi


 




Legacy ID



300765


Article URL http://www.symantec.com/docs/TECH58172


Terms of use for this information are found in Legal Notices