Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Backup failing with error code 23.

Created: 23 Apr 2013 | 17 comments

Master server BPTESTBPCD Output :-

=====================================

 

chev1bkp2{root}# bptestbpcd -client cspa1v12 -verbose
1 1 1
194.51.183.81:52825 -> 10.6.74.26:13724
194.51.183.81:52826 -> 10.6.74.26:13724
PEER_NAME = chev1bkp2
HOST_NAME = cspa1v12
CLIENT_NAME = cspa1v12
VERSION = 0x06500000
PLATFORM = hpia64
PATCH_VERSION = 
SERVER_PATCH_VERSION = 
MASTER_SERVER = chev1bkp2
EMM_SERVER = chev1bkp2
194.51.183.81:52827 -> 10.6.74.26:13724
chev1bkp2{root}# bptestbpcd -client cspa1v12 -debug
08:51:29.137 [27473] <2> bptestbpcd: VERBOSE = 0
08:51:29.164 [27473] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2048: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
08:51:29.164 [27473] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2062: service: bpcd
08:51:29.227 [27473] <2> logconnections: BPCD CONNECT FROM 194.51.183.81.52854 TO 10.6.74.26.13724
08:51:29.229 [27473] <2> vnet_connect_to_vnetd_extra: vnet_vnetd.c.182: msg: VNETD CONNECT FROM 194.51.183.81.52855 TO 10.6.74.26.13724 fd = 5
08:51:29.249 [27473] <2> vnet_vnetd_connect_forward_socket_begin: vnet_vnetd.c.535: VN_REQUEST_CONNECT_FORWARD_SOCKET: 10 0x0000000a
08:51:29.310 [27473] <2> vnet_vnetd_connect_forward_socket_begin: vnet_vnetd.c.552: ipc_string: /tmp/vnet-09604366786303751396000000000-a09604
1 1 1
194.51.183.81:52854 -> 10.6.74.26:13724
194.51.183.81:52855 -> 10.6.74.26:13724
08:51:29.411 [27473] <2> vnet_connect_to_vnetd_extra: vnet_vnetd.c.182: msg: VNETD CONNECT FROM 194.51.183.81.52856 TO 10.6.74.26.13724 fd = 6
08:51:29.429 [27473] <2> vnet_vnetd_connect_forward_socket_begin: vnet_vnetd.c.535: VN_REQUEST_CONNECT_FORWARD_SOCKET: 10 0x0000000a
08:51:29.490 [27473] <2> vnet_vnetd_connect_forward_socket_begin: vnet_vnetd.c.552: ipc_string: /tmp/vnet-09605366786303931418000000000-a09605
194.51.183.81:52856 -> 10.6.74.26:13724
<2>bptestbpcd: EXIT status = 0
08:51:29.521 [27473] <2> bptestbpcd: EXIT status = 0
========================================================================
 
Media server BPTESTBPCD output:-
=============================================

Master server BPTESTBPCD Output :-

root@csbkpms1 # ./bptestbpcd -client cspa1v12 -verbose
<16>bptestbpcd main: Function ConnectToBPCD(cspa1v12) failed: 23
socket read failed
root@csbkpms1 # ./bptestbpcd -client cspa1v12 -debug
08:57:09.142 [27668] <2> bptestbpcd: VERBOSE = 1
08:57:09.204 [27668] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2048: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
08:57:09.204 [27668] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2062: service: bpdbm
08:57:09.314 [27668] <2> logconnections: BPDBM CONNECT FROM 10.6.74.124.59878 TO 194.51.183.81.13724
08:57:09.633 [27668] <2> db_getCLIENT: db_CLIENTreceive: no entity was found 227
08:57:09.654 [27668] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2048: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
08:57:09.654 [27668] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2062: service: bpcd
08:57:09.707 [27668] <2> logconnections: BPCD CONNECT FROM 10.6.74.124.59879 TO 10.6.74.26.13724
08:57:09.708 [27668] <2> vnet_connect_to_vnetd_extra: vnet_vnetd.c.182: msg: VNETD CONNECT FROM 10.6.74.124.59880 TO 10.6.74.26.13724 fd = 5
08:57:09.727 [27668] <2> vnet_vnetd_connect_forward_socket_begin: vnet_vnetd.c.535: VN_REQUEST_CONNECT_FORWARD_SOCKET: 10 0x0000000a
08:57:09.789 [27668] <2> vnet_vnetd_connect_forward_socket_begin: vnet_vnetd.c.552: ipc_string: /tmp/vnet-11637366786778076381000000000-a11637
08:57:09.792 [27668] <2> get_short: (2) premature end of file (byte 1)
08:57:09.792 [27668] <2> bpcr_put_vnetd_forward_socket: get_short failed: 5
08:57:09.792 [27668] <2> local_bpcr_connect: bpcr_put_vnetd_forward_socket failed: 23
08:57:09.792 [27668] <2> ConnectToBPCD: bpcd_connect_and_verify(cspa1v12, cspa1v12) failed: 23
<16>bptestbpcd main: Function ConnectToBPCD(cspa1v12) failed: 23
08:57:09.793 [27668] <16> bptestbpcd main: Function ConnectToBPCD(cspa1v12) failed: 23
<2>bptestbpcd: socket read failed
08:57:09.793 [27668] <2> bptestbpcd: socket read failed
<2>bptestbpcd: EXIT status = 23
08:57:09.793 [27668] <2> bptestbpcd: EXIT status = 23
socket read failed
root@csbkpms1 # cat /etc/hosts | grep cspa1v12
10.6.74.26      cspa1v12
==============================================================
 
BPCD logs on client :-
------------------------------------[root@cspa1v12 - /usr/openv/netbackup/logs/bpcd ] # cat log.042413
08:51:10.035 [9413] <2> bpcd main: offset to GMT -3600
08:51:10.041 [9413] <2> logconnections: BPCD ACCEPT FROM 194.51.183.81.52825 TO 10.6.74.26.13724
08:51:10.041 [9413] <2> bpcd main: setup_sockopts complete
08:51:10.097 [9413] <2> bpcd peer_hostname: Connection from host chev1bkp2 (194.51.183.81) port 52825
08:51:10.097 [9413] <2> bpcd valid_server: comparing chev1bkp2 and chev1bkp2
08:51:10.098 [9413] <4> bpcd valid_server: hostname comparison succeeded
08:51:10.100 [9413] <2> bpcd main: output socket port number = 1
08:51:10.101 [9413] <2> bpcd main: Duplicated vnetd socket on stderr
08:51:10.101 [9413] <2> bpcd main: <---- NetBackup 6.5 0 ------------initiated
08:51:10.101 [9413] <2> bpcd main: VERBOSE = 0
08:51:10.101 [9413] <2> bpcd main: Not using VxSS authentication with chev1bkp2
08:51:10.182 [9413] <2> bpcd main: BPCD_PEERNAME_RQST
08:51:10.227 [9413] <2> bpcd peer_hostname: Connection from host chev1bkp2 (194.51.183.81) port 52825
08:51:10.292 [9413] <2> bpcd main: BPCD_HOSTNAME_RQST
08:51:10.402 [9413] <2> bpcd main: BPCD_CLIENTNAME_RQST
08:51:10.512 [9413] <2> bpcd main: BPCD_GET_VERSION_RQST
08:51:10.622 [9413] <2> bpcd main: BPCD_GET_PLATFORM_RQST
08:51:10.732 [9413] <2> bpcd main: BPCD_GET_VERSION_RQST
08:51:10.842 [9413] <2> bpcd main: BPCD_GET_VERSION_RQST
08:51:10.952 [9413] <2> bpcd main: BPCD_GET_VERSION_RQST
08:51:11.062 [9413] <2> bpcd main: BPCD_READ_HOST_CONFIG_RQST
08:51:11.472 [9413] <2> bpcd main: BPCD_GET_STDOUT_SOCKET_RQST
08:51:11.472 [9413] <2> bpcd main: socket port number = 1
08:51:11.557 [9413] <2> bpcd main: Connected on output socket
08:51:11.557 [9413] <2> bpcd main: Skipping shutdown of send side of stdout.
08:51:11.557 [9413] <2> bpcd main: Duplicated socket on stdout
08:51:11.558 [9413] <2> bpcd main: BPCD_DISCONNECT_RQST
08:51:11.558 [9413] <2> bpcd exit_bpcd: exit status 0  ----------->exiting
08:51:43.705 [9603] <2> bpcd main: offset to GMT -3600
08:51:43.705 [9603] <2> logconnections: BPCD ACCEPT FROM 194.51.183.81.52854 TO 10.6.74.26.13724
08:51:43.705 [9603] <2> bpcd main: setup_sockopts complete
08:51:43.761 [9603] <2> bpcd peer_hostname: Connection from host chev1bkp2 (194.51.183.81) port 52854
08:51:43.761 [9603] <2> bpcd valid_server: comparing chev1bkp2 and chev1bkp2
08:51:43.762 [9603] <4> bpcd valid_server: hostname comparison succeeded
08:51:43.765 [9603] <2> bpcd main: output socket port number = 1
08:51:43.765 [9603] <2> bpcd main: Duplicated vnetd socket on stderr
08:51:43.765 [9603] <2> bpcd main: <---- NetBackup 6.5 0 ------------initiated
08:51:43.765 [9603] <2> bpcd main: VERBOSE = 0
08:51:43.765 [9603] <2> bpcd main: Not using VxSS authentication with chev1bkp2
08:51:43.855 [9603] <2> bpcd main: BPCD_GET_STDOUT_SOCKET_RQST
08:51:43.855 [9603] <2> bpcd main: socket port number = 1
08:51:43.938 [9603] <2> bpcd main: Connected on output socket
08:51:43.938 [9603] <2> bpcd main: Skipping shutdown of send side of stdout.
08:51:43.938 [9603] <2> bpcd main: Duplicated socket on stdout
08:51:43.938 [9603] <2> bpcd main: BPCD_DISCONNECT_RQST
08:51:43.938 [9603] <2> bpcd exit_bpcd: exit status 0  ----------->exiting
08:59:03.079 [11392] <2> bpcd main: offset to GMT -3600
08:59:03.079 [11392] <2> logconnections: BPCD ACCEPT FROM 10.6.74.124.59872 TO 10.6.74.26.13724
08:59:03.079 [11392] <2> bpcd main: setup_sockopts complete
08:59:03.085 [11392] <8> bpcd peer_hostname: gethostbyaddr failed : HOST_NOT_FOUND (1)
08:59:03.085 [11392] <16> bpcd peer_hostname: gethostbyaddr failed to return peer host, herrno = 1
08:59:03.085 [11392] <16> bpcd main: Couldn't get peer hostname
08:59:38.027 [11636] <2> bpcd main: offset to GMT -3600
08:59:38.028 [11636] <2> logconnections: BPCD ACCEPT FROM 10.6.74.124.59879 TO 10.6.74.26.13724
08:59:38.028 [11636] <2> bpcd main: setup_sockopts complete
08:59:38.030 [11636] <8> bpcd peer_hostname: gethostbyaddr failed : HOST_NOT_FOUND (1)
08:59:38.030 [11636] <16> bpcd peer_hostname: gethostbyaddr failed to return peer host, herrno = 1
08:59:38.030 [11636] <16> bpcd main: Couldn't get peer hostname
=================================================================
 
Job detailed status :-
==========================
04/23/2013 18:00:01 - requesting resource csbkpms1-hcart3-robot-tld-0
04/23/2013 18:00:01 - requesting resource chev1bkp2.NBU_CLIENT.MAXJOBS.cspa1v12
04/23/2013 18:00:01 - requesting resource chev1bkp2.NBU_POLICY.MAXJOBS.sgm_val_sapsysunix
04/23/2013 18:00:15 - granted resource  chev1bkp2.NBU_CLIENT.MAXJOBS.cspa1v12
04/23/2013 18:00:15 - granted resource  chev1bkp2.NBU_POLICY.MAXJOBS.sgm_val_sapsysunix
04/23/2013 18:00:15 - granted resource  002490
04/23/2013 18:00:15 - granted resource  Drive08
04/23/2013 18:00:15 - granted resource  csbkpms1-hcart3-robot-tld-0
04/23/2013 18:00:32 - estimated 373981 kbytes needed
04/23/2013 18:00:32 - started process bpbrm (pid=2030)
04/23/2013 18:07:53 - Error bpbrm (pid=2030) bpcd on cspa1v12 exited with status 23: socket read failed
04/23/2013 18:07:54 - Error bpbrm (pid=2030) cannot send mail because BPCD on cspa1v12 exited with status 23: socket read failed
04/23/2013 18:10:06 - mounting 002490
04/23/2013 18:10:10 - end writing
04/23/2013 18:20:43 - Error bpbrm (pid=1976) bpcd on cspa1v12 exited with status 23: socket read failed
04/23/2013 18:20:44 - Error bpbrm (pid=1976) cannot send mail because BPCD on cspa1v12 exited with status 23: socket read failed
socket read failed (23)
========================================================
Client End initial troubleshooting :-
=------------------------------------------------------------
[root@cspa1v12 - /usr/openv/netbackup/bin ] # ./bpps -x
NB Processes
------------
 
 
Shared Symantec Processes
-------------------------
    root 11871     1  0 05:15:29 ?         0:00 /opt/VRTSpbx/bin/pbx_exchange
[root@cspa1v12 - /usr/openv/netbackup/bin ] # netstat -a | grep bpcd
tcp        0      0  *.bpcd                 *.*                     LISTEN
[root@cspa1v12 - /usr/openv/netbackup/bin ] # netstat -a | grep vnetd
tcp        0      0  *.vnetd                *.*                     LISTEN
[root@cspa1v12 - /usr/openv/netbackup/bin ] # uname -a
HP-UX cspa1v12 B.11.23 U ia64 2231644394 unlimited-user license
[root@cspa1v12 - /usr/openv/netbackup/bin ] # cat version
NetBackup-HP-UX11.23 6.5
======================================================================
 
 
I have restarted the services on client but no luck.
==========================
Master server end troubleshooting :-
===========================chev1bkp2{root}# /usr/openv/netbackup/bin/admincmd/bptestbpcd -client cspa1v12
1 1 1
194.51.183.81:53325 -> 10.6.74.26:13724
194.51.183.81:53326 -> 10.6.74.26:13724
194.51.183.81:53327 -> 10.6.74.26:13724
chev1bkp2{root}# /usr/openv/netbackup/bin/admincmd/bpgetconfig -g cspa1v12 -L
Client/Master = Client of chev1bkp2
NetBackup Client Platform = HP-UX-IA64, HP-UX11.23
NetBackup Client Protocol Level = 6.5.0
Product = NetBackup
Version Name = 6.5
Version Number = 650000
NetBackup Installation Path = /usr/openv/netbackup/bin
Client OS/Release = HP-UX B.11.23
===================================================
 
Any suggestions ???
 
 
Operating Systems:

Comments 17 CommentsJump to latest comment

Mack Disouza's picture

Connectivity test from master server to client :-

===================================

 

chev1bkp2{root}# ping cspa1v12
cspa1v12 is alive
chev1bkp2{root}# /usr/openv/netbackup/bin/bpclntcmd -hn cspa1v12
host cspa1v12: cspa1v12 at 10.6.74.26 (0xa064a1a)
aliases:
chev1bkp2{root}# /usr/openv/netbackup/bin/bpclntcmd -ip 10.6.74.26
checkhaddr: host   : cspa1v12: cspa1v12 at 10.6.74.26 (0xa064a1a)
checkhaddr: aliases:
============================================
Connectivity test from media server to client :-
=====================
root@csbkpms1 # /usr/openv/netbackup/bin/bpclntcmd -hn cspa1v12
host cspa1v12: cspa1v12 at 10.6.74.26 (0xa064a1a)
aliases:
root@csbkpms1 #  /usr/openv/netbackup/bin/bpclntcmd -ip 10.6.74.26
checkhaddr: host   : cspa1v12: cspa1v12 at 10.6.74.26 (0xa064a1a)
checkhaddr: aliases
=====================
 
From client to master server :-
=================================
[root@cspa1v12 - /root ] # /usr/openv/netbackup/bin/bpclntcmd -hn chev1bkp2
host chev1bkp2: chev1bkp2 at 194.51.183.81 (0xc233b751)
aliases:
[root@cspa1v12 - /root ] # /usr/openv/netbackup/bin/bpclntcmd -ip 194.51.183.81
checkhaddr: host   : chev1bkp2: chev1bkp2 at 194.51.183.81 (0xc233b751)
checkhaddr: aliases:
==================================
 
From Client to Media server :-
====================================
[root@cspa1v12 - /root ] # /usr/openv/netbackup/bin/bpclntcmd -hn csbkpms1
host csbkpms1: csbkpms1 at 10.6.74.124 (0xa064a7c)
aliases:
[root@cspa1v12 - /root ] # /usr/openv/netbackup/bin/bpclntcmd -ip 10.6.74.124
host 10.6.74.124: not found.
========================================

 

Giri_S's picture

Hello,

Here is what I understood from your issue :

Master : chev1bkp2 194.51.183.81
Media : csbkpms1 10.6.74.124
Client : cspa1v12 10.6.74.26

And The Error Points To :

************************************************************

08:59:03.085 [11392] <8> bpcd peer_hostname: gethostbyaddr failed : HOST_NOT_FOUND (1)

08:59:03.085 [11392] <16> bpcd peer_hostname: gethostbyaddr failed to return peer host, herrno = 1

08:59:03.085 [11392] <16> bpcd main: Couldn't get peer hostname

[root@cspa1v12 - /root ] # /usr/openv/netbackup/bin/bpclntcmd -ip 10.6.74.124

host 10.6.74.124: not found.

***************************************************************

The issue is client server is failing for the reverse name lookup.
Please help to ensure the /etc/hosts file contains below entry correct.

10.6.74.124 csbkpms1

Thanks.

Netbackup Admin (Unix)

sazz.'s picture

As you see it is failing on media server and can't resolve media server ip to a hostname.

Please add /etc/hosts entry for media server.
Ensure that hostname in hosts file have valid SERVER entry in bp.conf.

08:59:38.030 [11636] <8> bpcd peer_hostname: gethostbyaddr failed : HOST_NOT_FOUND (1)
08:59:38.030 [11636] <16> bpcd peer_hostname: gethostbyaddr failed to return peer host, herrno = 1
08:59:38.030 [11636] <16> bpcd main: Couldn't get peer hostname
Mack Disouza's picture

The media server IP is already in the client hosts file. I have already checked it.:-

======================================================

 

[root@cspa1v12 - /root ] # cat /etc/hosts | grep -i csbkpms1
10.6.74.124      csbkpms1 
[root@cspa1v12 - /root ] # ping csbkpms1
PING csbkpms1: 64 byte packets
64 bytes from 10.6.74.124: icmp_seq=0. time=0. ms
64 bytes from 10.6.74.124: icmp_seq=1. time=0. ms
64 bytes from 10.6.74.124: icmp_seq=2. time=0. ms
64 bytes from 10.6.74.124: icmp_seq=3. time=0. ms
64 bytes from 10.6.74.124: icmp_seq=4. time=0. ms
64 bytes from 10.6.74.124: icmp_seq=5. time=0. ms
^C
----csbkpms1 PING Statistics----
6 packets transmitted, 6 packets received, 0% packet loss
round-trip (ms)  min/avg/max = 0/0/0
[root@cspa1v12 - /root ] # ping 10.6.74.124
PING 10.6.74.124: 64 byte packets
64 bytes from 10.6.74.124: icmp_seq=0. time=0. ms
64 bytes from 10.6.74.124: icmp_seq=1. time=0. ms
64 bytes from 10.6.74.124: icmp_seq=2. time=0. ms
64 bytes from 10.6.74.124: icmp_seq=3. time=0. ms
64 bytes from 10.6.74.124: icmp_seq=4. time=0. ms
64 bytes from 10.6.74.124: icmp_seq=5. time=0. ms
^C
----10.6.74.124 PING Statistics----
6 packets transmitted, 6 packets received, 0% packet loss
round-trip (ms)  min/avg/max = 0/0/0
==========================================================
 
[root@cspa1v12 - /usr/openv/netbackup ] # cat bp.conf
SERVER = chev1bkp2
SERVER = csbkpms1
SERVER = csisov01
SERVER = csedcp01
CLIENT_NAME = cspa1v12

 

sazz.'s picture

can you please try the nslookup from csbkpms1 to cspa1v12 with name and ip address and vice versa. I still think it is not resolving, please show us the output of nslookup from media server to client and vice versa with ip and hostname
 

Mack Disouza's picture

NSLOOKUP from Media to client :-

=============================

 

root@csbkpms1 # cat /etc/hosts | grep -i cspa1v12
10.6.74.26      cspa1v12
root@csbkpms1 # nslookup cspa1v12
;; connection timed out; no servers could be reached
 
root@csbkpms1 # nslookup 10.6.74.26
;; connection timed out; no servers could be reached
 
root@csbkpms1 # ping cspa1v12
cspa1v12 is alive
root@csbkpms1 # ping 10.6.74.26
10.6.74.26 is alive
root@csbkpms1 # traceroute cspa1v12
traceroute: Warning: Multiple interfaces found; using 10.6.74.124 @ aggr262001
traceroute to cspa1v12 (10.6.74.26), 30 hops max, 40 byte packets
 1  cspa1v12 (10.6.74.26)  0.430 ms  0.329 ms  0.183 ms
root@csbkpms1 # traceroute 10.6.74.26
traceroute: Warning: Multiple interfaces found; using 10.6.74.124 @ aggr262001
traceroute to 10.6.74.26 (10.6.74.26), 30 hops max, 40 byte packets
 1  cspa1v12 (10.6.74.26)  0.299 ms  0.191 ms  0.183 ms
=========================================================
 
NSLOOKUP from Client tp media server
-----------------------------------------------------------------------------
[root@cspa1v12 - /root ] # cat /etc/hosts | grep -i csbkpms1
10.6.74.124      csbkpms1 
[root@cspa1v12 - /root ] # nslookup csbkpms1
Name Server:  cs-ad-prd01.ads.local
Address:  10.67.1.65
 
Trying DNS
Trying NIS
looking up FILES
Name:    csbkpms1
Address:  10.6.74.124
 
[root@cspa1v12 - /root ] # nslookup 10.6.74.124
Name Server:  cs-ad-prd01.ads.local
Address:  10.67.1.65
 
Trying DNS
Trying NIS
looking up FILES
Name:    csbkpms1
Address:  10.6.74.124
=========================================
 
 
sazz.'s picture

Why nslookup is not working from media to client. Your client is on 6.5, what is the nbu on media/master?

Was this client working before or first time configuration.

Mack Disouza's picture

Yes, the client backup was working before.

It is not a first time configuration.

Master version :- chev1bkp2{root}# cat /usr/openv/netbackup/bin/version

NetBackup-Solaris10 6.5.5

Media Version :- root@csbkpms1 # cat /usr/openv/netbackup/bin/version

NetBackup-Solaris10 6.5.5
CRZ's picture
Going back to the original bpcd log on the client:
08:59:03.079 [11392] <2> logconnections: BPCD ACCEPT FROM 10.6.74.124.59872 TO 10.6.74.26.13724
08:59:03.079 [11392] <2> bpcd main: setup_sockopts complete
08:59:03.085 [11392] <8> bpcd peer_hostname: gethostbyaddr failed : HOST_NOT_FOUND (1)
08:59:03.085 [11392] <16> bpcd peer_hostname: gethostbyaddr failed to return peer host, herrno = 1
08:59:03.085 [11392] <16> bpcd main: Couldn't get peer hostname
Next step was to figure out why gethostbyaddr wasn't working:
[root@cspa1v12 - /root ] # nslookup 10.6.74.124
Name Server:  cs-ad-prd01.ads.local
Address:  10.67.1.65
 
Trying DNS
Trying NIS
looking up FILES
Name:    csbkpms1
Address:  10.6.74.124
=========================================
Well, it explains the gethostbyaddr failing - 10.6.74.124 apparently isn't in DNS so it fails over to NIS, then to FILES, where nslookup finally succeeds in grabbing the /etc/hosts entry (I'm assuming).  But I'm not sure why NetBackup isn't going there as well...

Would you expect reverse DNS to work for 10.6.74.124, normally?  Could that have changed recently?

Check resolv.conf (has IT changed recently?) and confim you have something like "dns nis files" configured for lookup?

Can you move "files" before "dns" in the resolv.conf entry and see if that fixes it?

Try another bptestbpcd from the media server to the client and show us client's bpcd and vnetd logs.

DISCLAIMER: If this turns out to be some 6.5 thing I've forgotten over the past six months, my only excuse is we don't support it anymore.  ;-)


bit.ly/76LBN | APPLBN | 75LBN

Mack Disouza's picture

BPTESTBPCD  from Media server to client :-

======================================

 

root@csbkpms1 # ./bptestbpcd -client cspa1v12 -verbose
<16>bptestbpcd main: Function ConnectToBPCD(cspa1v12) failed: 23
socket read failed
---------------------------------------------
root@csbkpms1 # ./bptestbpcd -client cspa1v12 -debug
05:14:30.634 [12048] <2> bptestbpcd: VERBOSE = 1
05:14:30.700 [12048] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2048: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
05:14:30.701 [12048] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2062: service: bpdbm
05:14:30.812 [12048] <2> logconnections: BPDBM CONNECT FROM 10.6.74.124.33636 TO 194.51.183.81.13724
05:14:31.151 [12048] <2> db_getCLIENT: db_CLIENTreceive: no entity was found 227
05:14:31.173 [12048] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2048: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
05:14:31.173 [12048] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2062: service: bpcd
05:14:31.233 [12048] <2> logconnections: BPCD CONNECT FROM 10.6.74.124.33637 TO 10.6.74.26.13724
05:14:31.235 [12048] <2> vnet_connect_to_vnetd_extra: vnet_vnetd.c.182: msg: VNETD CONNECT FROM 10.6.74.124.33638 TO 10.6.74.26.13724 fd = 5
05:14:31.254 [12048] <2> vnet_vnetd_connect_forward_socket_begin: vnet_vnetd.c.535: VN_REQUEST_CONNECT_FORWARD_SOCKET: 10 0x0000000a
05:14:31.315 [12048] <2> vnet_vnetd_connect_forward_socket_begin: vnet_vnetd.c.552: ipc_string: /tmp/vnet-21133366859822168274000000000-a21133
05:14:31.318 [12048] <2> get_short: (2) premature end of file (byte 1)
05:14:31.318 [12048] <2> bpcr_put_vnetd_forward_socket: get_short failed: 5
05:14:31.318 [12048] <2> local_bpcr_connect: bpcr_put_vnetd_forward_socket failed: 23
05:14:31.318 [12048] <2> ConnectToBPCD: bpcd_connect_and_verify(cspa1v12, cspa1v12) failed: 23
<16>bptestbpcd main: Function ConnectToBPCD(cspa1v12) failed: 23
05:14:31.319 [12048] <16> bptestbpcd main: Function ConnectToBPCD(cspa1v12) failed: 23
<2>bptestbpcd: socket read failed
05:14:31.319 [12048] <2> bptestbpcd: socket read failed
<2>bptestbpcd: EXIT status = 23
05:14:31.319 [12048] <2> bptestbpcd: EXIT status = 23
socket read failed
root@csbkpms1 # uname -a
SunOS csbkpms1 5.10 Generic_144488-17 sun4v sparc SUNW,SPARC-Enterprise-T5220
root@csbkpms1 # date
Thu Apr 25 05:14:46 CEST 2013
===============================================================
root@csbkpms1 # /usr/openv/netbackup/bin/admincmd/bptestbpcd -client cspa1v12
<16>bptestbpcd main: Function ConnectToBPCD(cspa1v12) failed: 23
socket read failed
=================

On client :-

---------------

 

root@cspa1v12 - /etc ] # ls -l | grep resolv.conf
-rw-r--r--   1 root       sys            105 Apr 23 12:46 resolv.conf
[root@cspa1v12 - /etc ] # cat resolv.conf
domain ads.local
nameserver 10.67.1.65
nameserver 10.67.1.67
nameserver 10.67.1.81
nameserver 10.67.1.83
[root@cspa1v12 - /etc ] # uname -a
HP-UX cspa1v12 B.11.23 U ia64 2231644394 unlimited-user license
[root@cspa1v12 - /etc ] # cat nsswitch.conf
ipnodes : dns files
 
[root@cspa1v12 - /etc ] # cat nsswitch.files
#
# /etc/nsswitch.files:
#
# @(#)B11.23_LR
#
# An example file that could be copied over to /etc/nsswitch.conf; it
# does not use any name services.
#
passwd:       files
group:        files
hosts:        files dns
ipnodes:      files
services:     files
networks:     files
protocols:    files
rpc:          files
publickey:    files
netgroup:     files
automount:    files
aliases:      files
==================================================
 
 
Dan@NB's picture

This is related to connectivity issue between master, media and client servers.If all other backups are going fine using the same master and media servers, please check connectivity from master to client server and from media to client server using ping, telnet (using port 13782, 13724, 1556), nslookup, traceroute. Please perform teh same from client server for master and media servers.

Possible firewall or port block issue. Its also possible teh connectivity is not bi-directional from client, media and master server.

Also ensure bp.conf file of client server is having the same client name defined as the one defined in policy in master server. Ensure master and media server IP, FQDN and short name (if any) is listed in bp.conf and hosts file of client server. Same is valid for master and media server bp.conf and hosts file entry for client server.

Do let us know if that resolves the issue.

 

+ Dan

Mack Disouza's picture

 

When i am trying to nslookup the media server csbkpms1 from cspa1v12:-
 
=====================================================================
[root@cspa1v12 - /root ] # nslookup csbkpms1
Name Server:  cs-ad-prd01.ads.local
Address:  10.67.1.65
 
Trying DNS
Trying NIS
looking up FILES
Name:    csbkpms1
Address:  10.6.74.124
--------------------------------------------------------------
 
I checked the resolv.conf file on client cspa1v12 and found below :-
-----------------
 
[root@cspa1v12 - /etc ] # cat resolv.conf
domain ads.local
nameserver 10.67.1.65
nameserver 10.67.1.67
nameserver 10.67.1.81
nameserver 10.67.1.83
------------------------------------------------------------
[root@cspa1v12 - /etc ] # uname -a
HP-UX cspa1v12 B.11.23 U ia64 2231644394 unlimited-user license
================================================================================================
 
Whereas i have another client ( csea1v01) with the same configuration in the same policy and backup for this client is successful without any issue.
 
When i am trying to nslookup the media server csbkpms1 from csea1v01
 

 ==================================================================================================
[root@csea1v01 - /root ] # nslookup csbkpms1
Using /etc/hosts on:  csea1v01

looking up FILES
Name:    csbkpms1
Address:  10.6.74.124
----------------------------------

I checked the resolv.conf file on client csea1v01 and found below :-

[root@csea1v01 - /etc ] # cat resolv.conf
domain  
--------------------------------
[root@csea1v01 - /etc ] # uname -a
HP-UX csea1v01 B.11.23 U ia64 3111162404 unlimited-user license

=======================================================================================================

Netbackup Connectivity test from media server to client :-

=====================
root@csbkpms1 # /usr/openv/netbackup/bin/bpclntcmd -hn cspa1v12
host cspa1v12: cspa1v12 at 10.6.74.26 (0xa064a1a)
aliases:
root@csbkpms1 #  /usr/openv/netbackup/bin/bpclntcmd -ip 10.6.74.26
checkhaddr: host   : cspa1v12: cspa1v12 at 10.6.74.26 (0xa064a1a)
checkhaddr: aliases
=====================
Netbackup Connectivity test from Client to Media server :-
====================================
[root@cspa1v12 - /root ] # /usr/openv/netbackup/bin/bpclntcmd -hn csbkpms1
host csbkpms1: csbkpms1 at 10.6.74.124 (0xa064a7c)
aliases:
[root@cspa1v12 - /root ] # /usr/openv/netbackup/bin/bpclntcmd -ip 10.6.74.124
host 10.6.74.124: not found.
========================================
 
 
Mack Disouza's picture

I have changed the resolv.conf file on the client cspa1v12 as it is on other servers

It almost resolve the connectivity issue.

=========================================================

 

[root@cspa1v12 - /root ] # /usr/openv/netbackup/bin/bpclntcmd -hn csbkpms1
host csbkpms1: csbkpms1 at 10.6.74.124 (0xa064a7c)
aliases:
[root@cspa1v12 - /root ] # /usr/openv/netbackup/bin/bpclntcmd -ip 10.6.74.124
checkhaddr: host   : csbkpms1: csbkpms1 at 10.6.74.124 (0xa064a7c)
checkhaddr: aliases:
[root@cspa1v12 - /root ] # cat /etc/resolv.conf
domain
===========================================================
 
Backup restarted and it is working ok . Finger crossed:-
=========================
04/25/2013 12:19:15 - requesting resource csbkpms1-hcart3-robot-tld-0
04/25/2013 12:19:15 - requesting resource chev1bkp2.NBU_CLIENT.MAXJOBS.cspa1v12
04/25/2013 12:19:15 - requesting resource chev1bkp2.NBU_POLICY.MAXJOBS.sgm_val_sapsysunix
04/25/2013 12:19:16 - Waiting for scan drive stop Drive03, Media server: csbkpms1
04/25/2013 12:19:47 - granted resource  chev1bkp2.NBU_CLIENT.MAXJOBS.cspa1v12
04/25/2013 12:19:47 - granted resource  chev1bkp2.NBU_POLICY.MAXJOBS.sgm_val_sapsysunix
04/25/2013 12:19:47 - granted resource  006328
04/25/2013 12:19:47 - granted resource  Drive03
04/25/2013 12:19:47 - granted resource  csbkpms1-hcart3-robot-tld-0
04/25/2013 12:19:48 - estimated 373981 kbytes needed
04/25/2013 12:19:49 - started process bpbrm (pid=9128)
04/25/2013 12:21:19 - connecting
04/25/2013 12:21:19 - connected; connect time: 0:00:00
04/25/2013 12:21:20 - mounting 006328
04/25/2013 12:21:56 - mounted 006328; mount time: 0:00:36
04/25/2013 12:21:57 - positioning 006328 to file 38
04/25/2013 12:23:22 - positioned 006328; position time: 0:01:25
04/25/2013 12:23:22 - begin writing
=========================================

 

Dan@NB's picture

Since its connected and data is moving ... its gonna be fine.. no worries ;)

+ Dan

CRZ's picture
root@cspa1v12 - /etc ] # ls -l | grep resolv.conf
-rw-r--r--   1 root       sys            105 Apr 23 12:46 resolv.conf
Hmmm, changed recently?  ;-)
 
I did mean nsswitch.conf up there, good of you to read my mind!
 
So you've probably cracked it and the problem most likely had/has something to do with the resolution of csbkpms1 vs. csbkpms1.ads.local and expecting one but getting the other (or nothing).
 
I notice that your nslookup goes straight to FILES on the "working" client - also, no domain or name servers set.  I have a hunch that whoever helped you out with the new resolv.conf also broke the backups, so you will have to put your heads together to figure out how to get whatever they want done without also affecting the backups.  It will probably involve getting the unqualified hostname and the fully qualified (with ".ads.local") and the ultimate resolution may involve switching your clients within the NetBackup configuration to use their fully qualified names instead of short ones.  Just something to think about for the future - if it's working NOW, maybe don't touch it until somebody complains about you changing their resolv.conf :)


bit.ly/76LBN | APPLBN | 75LBN