XBSA client <-> server backup failure
Created: 06 Feb 2013 | Updated: 06 Feb 2013 | 7 comments
Hi,
Since we upgraded to Netbackup v7.1.0.3, the Netbackup client fails to communicate to the server via XBSA (AIX platform). Or more precisely, the backup fails, since the communication seems first successful (C_GET_FEATURES command OK) but fails after that. There is no firewall involved. Strangely, all backup demands via XBSA are now frozen (the last log in exten_client is "connect in progress") and no answer is ever received. However, a standard client backup works. When the Netbackup client is in v6.x, it works also !
End of log of bprd (server side) :
16:44:37.683 [13642] <2> vnet_pbxAcceptSocket: Accepted sock[9] from IP9:34428 16:44:37.683 [13642] <2> bprd: socket fd from accept() is 9 16:44:37.684 [13642] <2> listen_loop: request complete 16:44:37.685 [17454] <2> logconnections: BPRD ACCEPT FROM IP9.34428 TO IP129.1556 fd = 9 16:44:37.686 [17454] <2> process_request: setsockopt SO_LINGER on 9 succeeded. 16:44:37.686 [17454] <2> connected_peer: Connection from host CLIENTNAME_HIDDEN, IP9, on non-reserved port 34428 16:44:37.688 [17454] <2> process_request: command C_GET_FEATURES (72) received 16:44:37.691 [17454] <2> process_request: EXIT STATUS 0 16:44:37.697 [13642] <2> vnet_pbxAcceptSocket: Accepted sock[9] from IP9:34430 16:44:37.697 [13642] <2> bprd: socket fd from accept() is 9 16:44:37.697 [13642] <2> listen_loop: request complete 16:44:37.698 [17455] <2> logconnections: BPRD ACCEPT FROM IP9.34430 TO IP129.1556 fd = 9 16:44:37.699 [17455] <2> process_request: setsockopt SO_LINGER on 9 succeeded. 16:44:37.699 [17455] <2> connected_peer: Connection from host CLIENTNAME_HIDDEN, IP9, on non-reserved port 34430 16:44:37.811 [17455] <2> get_long: (2) premature end of file (byte 1) 16:44:37.816 [17455] <32> process_request: get_string() failed - premature end of file encountered (5) 16:44:37.820 [17455] <32> process_request: could not process request from CLIENTNAME_HIDDEN 16:44:38.829 [13642] <2> ConnectionCache::connectAndCache: Acquiring new connection for host SERVERNAME_HIDDEN, query type 98 16:44:38.829 [13642] <2> logconnections: BPDBM CONNECT FROM IP68.46871 TO IP68.13721 fd = 9 16:44:38.829 [13642] <2> db_end: Need to collect reply
End of log of exten_client (client side):
<2> vnet_cached_getaddrinfo_and_update: ../../libvlibs/vnet_addrinfo.c.1373: 0: found in cache service: NULL <2> get_pref_netconnection: ../../libvlibs/vnet_addrinfo.c.5032: 0: prefnet = : REQIF <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1200: 0: connect in progress: 1 0x00000001
The bpclntcmd command on the client seems successful :
bpclntcmd -hn SERVERNAME_HIDDEN host SERVERNAME_HIDDEN: SERVERNAME_HIDDEN at IP129 aliases: SERVERNAME_HIDDEN IP129 bpclntcmd -hn CLIENTNAME_HIDDEN host CLIENTNAME_HIDDEN: CLIENTNAME_HIDDEN at IP9 aliases: CLIENTNAME_HIDDEN IP9 bpclntcmd -self yp_get_default_domain failed: (6) internal yp server or client error NIS does not seem to be running: (1) args to yp function are bad gethostname() returned: CLIENTNAME_HIDDEN host CLIENTNAME_HIDDEN: CLIENTNAME_HIDDEN at IP9 aliases: CLIENTNAME_HIDDEN IP9 bpclntcmd -ip IP129 host IP129: SERVERNAME_HIDDEN at IP129 aliases: SERVERNAME_HIDDEN IP129
Any idea ? I have attached logs files.
Thanks for your help.
Discussion Filed Under:
Comments 7 Comments • Jump to latest comment
It looks like this may be a Symantec Private Branch Exchange (pbx or pbx_exchange) issue
As you see in your log file the connection goes over port 1556 which is the pbx port
It may be that your client is having an issue with running pbx_exchange on it when at V7.1
Check that it is running, it is located at /opt/VRTSpbx/bin/
You can check its status using this doc http://www.symantec.com/docs/TECH44074
To check its logs use:
vxlogview -o 103 -t 02:00:00
This will pipe out the pbx logs for the last 2 hours so if it is not running start it and then take a look at its logs
7.1 also does a lot more with reverse lookups so hosts files can be useful to make sure the client can communicate correctly with the Master and Media Servers and vice versa, using short and FQDN
Hope this helps
Authorised Symantec Consultant
Don't forget to give a "Thumbs Up" or mark as "Solution" if someones advice has helped you.
I realize I didn't properly "reply" to your answer, but answered in another comment. In case you were hence not notified of my answer, please find it later in this thread.
Thanks.
Thanks, we are going to check that, but why, if it is really a problem of pbx, is the following log present in extenclient :
and why "normal" client backups (not using XBSA) work without pbx working correctly ?
Hi,
We have performed the checks on pbx. The pbx process is running normally on the master and on the client, the /etc/services contains the service on port 1556. The pbxcfg seems to be normal, both on server and client.
However, when archiving through XBSA, there is no log on client side generated by pbx. The process seems inactive :
What should we do now ?
Thanks,
Hi,
Some last details, if it can help. We have shutdown the pbx service on client side, it changes nothing. The command bptestnetconn on the client returns right results. A telnet to 1556 port from the client to the server works also. It seems really, from the log of exten_client, that PBX hangs : a first connection is done
vnet_async_connect: ../../libvlibs/vnet_connect.c.1200: 0: connect in progress: 1 0x00000001
vnet_pbxConnect: pbxConnectEx Succeeded
but a second one is blocked :
vnet_async_connect: ../../libvlibs/vnet_connect.c.1200: 0: connect in progress: 1 0x00000001
and nothing after that.
We see the traces of two connections in the PBX logs server side :
03/01/2013 09:37:08.294 [Info] PBX_Manager:: handle_input with fd = 7
03/01/2013 09:37:08.294 [Info] PBX_Client_Proxy::parse_line, line = ack=1 From IP9
03/01/2013 09:37:08.294 [Info] PBX_Client_Proxy::parse_line, line = extension=bprd From IP9
03/01/2013 09:37:08.294 [Info] hand_off looking for proxy for = bprd
03/01/2013 09:37:08.294 [Info] is_accepting about to return true
03/01/2013 09:37:08.294 [Info] Client Expects an ACK, sent one.
03/01/2013 09:37:08.294 [Info] Proxy found.
03/01/2013 09:37:08.294 [Info] PBX_Client_Proxy::handle_close
03/01/2013 09:37:08.305 [Info] PBX_Manager:: handle_input with fd = 7
03/01/2013 09:37:08.306 [Info] PBX_Client_Proxy::parse_line, line = ack=1 From IP9
03/01/2013 09:37:08.306 [Info] PBX_Client_Proxy::parse_line, line = extension=bprd From IP9
03/01/2013 09:37:08.306 [Info] hand_off looking for proxy for = bprd
03/01/2013 09:37:08.306 [Info] is_accepting about to return true
03/01/2013 09:37:08.306 [Info] Client Expects an ACK, sent one.
03/01/2013 09:37:08.306 [Info] Proxy found.
03/01/2013 09:37:08.306 [Info] PBX_Client_Proxy::handle_close
I've seen http://clientui-kb.symantec.com/kb/index?page=content&pmv=print&impressions=&viewlocale=&id=TECH174379 but it doesn't seem to be related (there are not so many connections).
Still no idea ?
Thanks,
Check that this command on client returns expected client name from master:
bpclntcmd -pn
Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links
Hi,
Yes, this seems the case :
What should we check now ? Do you know PBX-related fixes from 7.0.3 to 7.5.x that could be linked to this problem, apart from 2525096 ?
Thanks for your help,
Would you like to reply?
Login or Register to post your comment.