NetBackup 7.1 upgrade from 6.x Unix/Linux Masters hangs during the policies conversion stage.

Article:TECH156810  |  Created: 2011-03-29  |  Updated: 2011-06-07  |  Article URL http://www.symantec.com/docs/TECH156810
Article Type
Technical Solution


Issue



When upgrading NetBackup to 7.1 from 6.x on Unix/Linux, the upgrade may take many minutes and perhaps hours, depends on how many clients in the current policies are defined with supported Windows operating system but an older hardware definition; PC, PC-x64 or PC-IA64.

 

The delay will be observed at this point in the install_trace log.

 

  Updating client hardware definitions:

 (Potential delay of many minutes)

  Hardware update conversion summary:

        Number of policies processed:                         110

        Number of policies with affected clients:             48

        Number of clients processed:                          215

        Number of clients converted:                          48

        Number of Disaster Recovery flags cleared:            0

 


Error



If enabled, the admin log shows that bpplconvert suffers a 5 minute delay when processing some, but not all, policies.

 

22:13:19.836 [11896] <2> bpplconvert: VERBOSE = 5

22:13:19.836 [11896] <2> bpplconvert: INITIATING: version NetBackup 7.1 created: 0

22:13:19.836 [11896] <2> logparams: -allpolicies -new_hardware_definitions

...

22:13:19.905 [11896] <2> update_hardware_convert_all_policies: Looking for clients in policy Test_pol

22:13:19.910 [11896] <2> logconnections: BPDBM CONNECT FROM 127.0.0.1.40803 TO 127.0.0.1.1556 fd = 5

22:13:19.930 [11896] <4> TZ: Send Timezone offset=7200

22:13:19.932 [11896] <2> db_end: Need to collect reply

... (5 minute delay because policy Test_pol contains clients for PC*/Windows* platforms)

22:18:43.152 [11896] <2> db_end: no DONE from db_getreply(): cannot connect on socket

22:18:43.153 [11896] <2> db_CLASS: db_end_sts() failed: cannot connect on socket (25)

22:18:43.153 [11896] <2> update_hardware_convert_all_policies: In policy Test_pol: Changed hardware for 1 clients

...

22:24:06.496 [11896] <2> logconnections: BPDBM CONNECT FROM 127.0.0.1.41410 TO 127.0.0.1.1556 fd = 5

22:24:06.519 [11896] <4> TZ: Send Timezone offset=7200

22:24:06.530 [11896] <2> db_end: Need to collect reply

... (No delay because policy online-catalog-pol does not contain any clients for PC*/Windows* platforms)

22:24:06.530 [11896] <2> update_hardware_convert_all_policies: Looking for clients in policy online-catalog-pol

22:24:06.530 [11896] <2> bpplconvert: EXIT status = 0

 

 

For each policy to be converted, the bpdbm log shows a CORBA timeout connecting to nbemm (to get the AUDIT setting), but nbemm has not yet been started during the upgrade.

 

22:13:19.917 [11898] <2> logconnections: BPDBM ACCEPT FROM 127.0.0.1.40803 TO 127.0.0.1.1556 fd = 10

...

22:13:19.929 [11898] <2> class_db: ?

...

22:13:19.985 [11898] <2> Orb::init: initializing ORB EMMlib_Orb with: bpdbm -ORBSvcConfDirective ...

22:13:20.021 [11898] <2> Orb::init: caching EndpointSelectorFactory(../Orb.cpp:839)

22:14:19.678 [11898] <16> emmlib_initializeEx: (-) Exception! CORBA::TRANSIENT

22:14:19.679 [11898] <16> emmlib_initializeEx: (-) system exception, ID 'IDL:omg.org/CORBA/TRANSIENT:1.0'

22:15:24.273 [11898] <16> emmlib_initializeEx: (-) Exception! CORBA::TRANSIENT

22:15:24.273 [11898] <16> emmlib_initializeEx: (-) system exception, ID 'IDL:omg.org/CORBA/TRANSIENT:1.0'

22:16:28.863 [11898] <16> emmlib_initializeEx: (-) Exception! CORBA::TRANSIENT

22:16:28.863 [11898] <16> emmlib_initializeEx: (-) system exception, ID 'IDL:omg.org/CORBA/TRANSIENT:1.0'

22:17:33.523 [11898] <16> emmlib_initializeEx: (-) Exception! CORBA::TRANSIENT

22:17:33.523 [11898] <16> emmlib_initializeEx: (-) system exception, ID 'IDL:omg.org/CORBA/TRANSIENT:1.0' 22:18:38.138 [11898] <16> emmlib_initializeEx: (-) Exception! CORBA::TRANSIENT

22:18:38.138 [11898] <16> emmlib_initializeEx: (-) system exception, ID 'IDL:omg.org/CORBA/TRANSIENT:1.0'

22:18:43.151 [11898] <16> AuditPolicy::get_audit_setting: (-) Translating EMM_ERROR_CorbaTransient(3000001) to 25 in the NetBackup context

22:18:43.152 [11898] <2> class_db: Failed to get AUDIT setting: cannot connect on socket (25)

22:18:43.152 [11898] <2> process_request: request complete: exit status 25 cannot connect on socket; query type: 189

 


Environment



NetBackup 6.x UNIX/Linux master server with multiple policies containing supported Windows clients, but with PC, PC-x64, or PC-IA64 hardware definitions.


Cause



Before saving each updated policy to the class database, bpplconvert checks to see if policy change auditing is enabled.  The audit flag is stored in the EMM database, but nbemm has not yet been started by the upgrade process, so the connection times out; once for each policy to be updated.

NetBackup 7.1 upgrade from 6.x Unix/Linux Masters upgrades where the following Windows client platforms are currently being backed up:

 

       Hardware    Operating system

       ---------   ---------------

   1)  Windows-IA64, Windows2003

   2)  Windows-IA64, Windows2008

   3)  Windows-IA64, WindowsXP

   4)  Windows-x64, Windows2003

   5)  Windows-x64, Windows2008

   6)  Windows-x64, Windows7

   7)  Windows-x64, WindowsVista

   8)  Windows-x64, WindowsXP

   9)  Windows-x86, Windows2003

  10)  Windows-x86, Windows2008

  11)  Windows-x86, Windows7

  12)  Windows-x86, WindowsVista

  13)  Windows-x86, WindowsXP

 


Solution



If the UNIX/Linux master server has already been upgraded to NetBackup 7.1, the class database conversion did not occur and the existing policy clients will not have the updated hardware types.  Run this command on the master server to complete the conversion.

 

   # cd /usr/openv/netbackup/bin/goodies

   # bpplconvert -allpolicies -new_hardware_definitions

 

Customers who are preparing to upgrade to 7.1 can avoid the delay during upgrade by performing either of these options.

 

Option A

 

When you see:

 

Policy conversion summary:

Number of original policies:                                     xx

Number of non-snapshot policies skipped  :      xx

Number of policies not needing conversion:      xx

Number of policies converted to

     'auto' snapshot method:                                      xx

Number of policies converted:                                xx


Updating client hardware definitions:

 

Open another terminal session to the master server and simply start up nbemm by running:

 

/usr/openv/netbackup/bin/nbemm

will allow the upgrade to proceed. 

Option B

Upgrade to NetBackup 7.0 on the master server.  This will successfully convert policies and an upgrade to 7.1 can then be executed.

 

Option C

Apply the attached NetBackup Hotfix (Etrack 2334190) which can be applied during the NetBackup 7.1 master server during the upgrade process.  The hotfix installation instructions are as follows:

 

1. Execute the normal install script to upgrade to NB_71

 

2. When prompted for the license key, skip to step 3.

 

   Do you want to continue the installation

   with the current active NetBackup license key? [y,n] (y)

 

3. Open a new terminal session on the master server.

 

4. Execute the eebinstaller to apply the hotfix.

 

5. Once the hotfix is installed, close the second session.

 

6. Resume the upgrade by answering the license key prompt.

 

Hotfix Details:

 

Installation Instructions: 
Please follow the EEB Installer instructions available on  http://www.symantec.com/docs/TECH64620

Package Contents:
Please choose the appropriate platform after download:

eebinstaller.2334190.1.hpia64             HP-IA64 Installation
eebinstaller.2334190.1.linuxS_x86         x64 SuSE Linux Enterprise Installation
eebinstaller.2334190.1.linuxR_x86         x64 RedHat Enterprise Linux Installation
eebinstaller.2334190.1.solaris_x86        Solaris 10 x86/x64 Installation
eebinstaller.2334190.1.solaris            Solaris SPARC Installation
eebinstaller.2334190.1.rs6000             AIX Installation

Deliverables:
--
/usr/openv/lib/libVdbMT.so
/usr/openv/lib/libVdb.so
/usr/openv/netbackup/bin/bpdbm
/usr/openv/netbackup/bin/goodies/bpplconvert

File checksum:
--
3294552010 5344192 solaris/bpdbm
1599729431 62640 solaris/bpplconvert
3930416403 1384040 solaris/libVdbMT.so
3118649976 1317152 solaris/libVdb.so
4115721188 3956727 linuxR_x86/bpdbm
3280753131 45777 linuxR_x86/bpplconvert
1000300316 1194167 linuxR_x86/libVdbMT.so
1792567746 1169549 linuxR_x86/libVdb.so
3023947833 3709197 linuxS_x86/bpdbm
1162555316 44208 linuxS_x86/bpplconvert
1576213871 940396 linuxS_x86/libVdbMT.so
3291528246 925127 linuxS_x86/libVdb.so
1445673496 12670720 hpia64/bpdbm
3855621791 157496 hpia64/bpplconvert
4049044911 5216256 hpia64/libVdbMT.so
1830113131 4776976 hpia64/libVdb.so
322656103 5851104 solaris_x86/bpdbm
3168208945 60232 solaris_x86/bpplconvert
599852735 1737472 solaris_x86/libVdbMT.so
3873215455 1665944 solaris_x86/libVdb.so
4080967543 9205606 rs6000/bpdbm
3346037871 73441 rs6000/bpplconvert
272694175 2227816 rs6000/libVdbMT.so
1350226582 2174421 rs6000/libVdb.so


Attachments

NB_7.1_ET2334190_1 is a NetBackup Hotfix for Unix/Linux Master Servers
NB_7.1_ET2334190_1.zip (18.4 MBytes)

Supplemental Materials

SourceETrack
Value2334706
Description

(primary) NB 7.1 upgrade delayed for hours while bpplconvert repeatedly attempts to connect to nbemm which has not yet been started



Article URL http://www.symantec.com/docs/TECH156810


Terms of use for this information are found in Legal Notices