Replacing an SF-RAC 5.0mp1 Solaris 10 system with an image

Article:TECH146304  |  Created: 2010-12-15  |  Updated: 2014-01-02  |  Article URL http://www.symantec.com/docs/TECH146304
Article Type
Technical Solution


Issue



Replacing an SF-RAC 5.0mp1 Solaris 10 system with an image


Cause



There are several reasons for replacing a system in a SF-RAC cluster   including replacing bad hardware, upgrading, returning loaner equipment,etc. 

Generally, there are two basic methods of replacing the hardware:

  1. Adding the new system hardware as a new system to the cluster, then removing the old system from the cluster.   This has the advantages of maintaining bandwidth and performance which is why it's a very popular method.  However, the new system has a higher LLT node id, and this requires significant edits of the VCS configuration. 
  2. Using an image (whether via NFS Jumpstart, tape backup, or other), replace the system using the same hostname, and configuration.  This requires disabling and taking down the old node first, reducing performance and bandwidth.    However, it has the advantage that it requires very few edits in the VCS configuration.  There are some files which will need to be checked and/or edited.

Solution



In the case of an image replacement,

Consider editing any configuration files needed for differences in hardware before taking the image.  Otherwise, the changes will need to be made after the new system fails to startup.   Here are some examples of things to check and possibly change:

  • Network NIC names may be different. 
    • The public NIC's are configured in /etc/hostname.{_NIC_}
    • The VCS private LLT NIC's are configured in /etc/llttab. 
      •  Please note that the syntax of the llttab links are  "link[-lowpri] {_tag_}  {_device_}  - - ether - " . 
      • The "link" line stars with the text "link" for private links, or "link-lowpri" for public links. 
      • The "tag" is a comment on what LLT calls the link.  If systems use different NIC's, you can use another name, like vlan22, link-22, etc.
      • On Solaris, the "device" is in the form of "/dev/{_nic-type_}:{_n_}" where "nic-type" is like ge, bge, ce, en, etc.  And "n" is the instance of that type.  So, "/dev/qfe1" becomes "/dev/qfe:1" in the llttab file
  • HBA's. Please check the HBA configuration to see if local tuning is required for link failover timeouts, HBA based zoning, etc.  Please check with the VxVM Hardware Compatibility List, HBA and S.A.N. vendors for the requirements. 
  • VxVM excludes maybe based on controller numbers and would need to be updated if different controller numbers are used on the new system.
  • Depending on the S.A.N. configuration and wwn's, add the new system to the S.A.N. hostgroups and Lun configurations to ensure VxFen is supported.

Before booting up the new system,  disable the system to be removed.  This is critical do to because if the old system boots up on the network, it can cause a panic and/or split brain. 

To disable the old system use  one   of the following methods:

  • Uninstall SF-RAC via /opt/VRTS/install/uninstallsfrac
  • Clean the disks and install a new OS fresh.
  • Unconfigure LLT, GAB, and VxFEN:
    • Unconfigure LLT by moving or removing /etc/llttab and /etc/llthosts.
    • Unconfigure GAB by moving or removing /etc/gabtab
    • Unconfigure VxFEN by moving or removing /etc/vxfendg and /etc/vxfenmode
  • Disable LLT, GAB, and VxFEN:
    • Disable LLT:  mv   /etc/rc2.d/S70llt /etc/rc2.d/disabled_S70llt
    • Disable GAB: mv /etc/rc2.d/S92gab /etc/rc2.d/disabled_S92gab
    • Disable VxFEN: mv /etc/rc2.d/S97vxfen /etc/rc2.d/disabled_S97vxfen

From here, bring down the old system, cable in the new system hardware, and bootup the new image.

Here are some additional resources which may be helpful:

  • What to do after changing the host name of the system:  http://www.symantec.com/docs/TECH16816
     
  • How to change a host name for both VCS and VERITAS Volume Manager :  http://www.symantec.com/docs/TECH94036
     
  • The SF-CFS Install Guide has the Add and Remove Node procedures for reference.
    http://ftp.support.veritas.com/pub/support/products/SANPoint_FS_HA/275727.pdf
    Add a Node, Chapter 3, starts on page 45.
     
  • Please note that
    • With CFS, the hostname in /etc/nodename must be used in main.cf and /etc/llthosts.
    • The LLT host id must be in the SystemList and NodeList for the cvm groups.
    • If you replace an existing node, the other existing nodes must reboot to clear the record of the previous nodes mac addresses on LLT.  
      If Adding a node, no reboot is required.
    • LLT and GAB must be at the exact same version for adding or replacing nodes.
       
  • If this is SF-RAC for Oracle RAC, there are additional steps.   Also, the /kernel/drv/vcsmm and lmx.conf files must be the same. (It's a good idea to have the gab, glm, gms, odm, and llt dot.conf files synced with sf-cfs). 


     Please contact Symantec Technical Support if there are any questions or issues the node replacement process.



Article URL http://www.symantec.com/docs/TECH146304


Terms of use for this information are found in Legal Notices