Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

bptestbpcd just hangs

Created: 12 Jan 2013 • Updated: 12 Feb 2013 | 24 comments
This issue has been solved. See solution.
-bash-3.2$ sudo ./bptestbpcd -client ukunix12  -debug
07:46:29.949 [12028] <2> bptestbpcd: VERBOSE = 0
07:46:29.949 [12028] <2> read_client: dname=., offline=0, online_at=0
07:46:29.949 [12028] <2> read_client: dname=.., offline=0, online_at=0
07:46:29.949 [12028] <2> read_client: dname=CO_0, offline=0, online_at=0
07:46:29.949 [12028] <2> read_client: dname=host_info, offline=0, online_at=0
07:46:29.949 [12028] <2> read_client: dname=OA_1319112873, offline=0, online_at=1319112873
07:46:29.950 [12028] <2> db_freeEXDB_INFO: ?
07:46:29.951 [12028] <2> vnet_pbxConnect: pbxConnectEx Succeeded
07:46:29.952 [12028] <2> logconnections: BPCD CONNECT FROM 13.11.31.32.40707 TO 13.11.39.95.1556 fd = 3
07:46:29.953 [12028] <2> vnet_pbxConnect: pbxConnectEx Succeeded
 
 
This script just hangs.Please let me know what to do.This does does comes out of its own.I have to press ctrl c to come out of this.

Comments 24 CommentsJump to latest comment

RamNagalla's picture

hey,

what is your netbackup versions and OS for master and the client?

let me know the below outputs form Client

netstat -a | grep 1556

netstat -a | grep 13782

netstat -a | grep 13724

bpclntcmd -pn

bpclntcmd -self

Marianne's picture

The server is waiting for Connect-back from the client.

It will wait until Client Connect Timeout expires.

Before you try again, check CLIENT_CONNECT_TIMEOUT value in this server's bp.conf. 
If there is no such entry it means that the timeout is the default of 300 (5 minutes).
If there is a big timeout value, change it to 300 while you are testing.
Best to wait for timeout to see error message.

Hope you have created bpcd log folder on the client?

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

nike_new's picture
-bash-3.00$ netstat -an |grep -i bpcd
600d3de21d8 stream-ord 600b2ac4c40 00000000 /usr/openv/var/vnetd/terminate_bpcd.uds
6010255c3b0 stream-ord 6009f851500 00000000 /usr/openv/var/vnetd/bpcd.uds
-bash-3.00$ netstat -an |grep -i vnetd
600d3de21d8 stream-ord 600b2ac4c40 00000000 /usr/openv/var/vnetd/terminate_bpcd.uds
6010255c3b0 stream-ord 6009f851500 00000000 /usr/openv/var/vnetd/bpcd.uds
600e085b420 stream-ord 600b2ac5e40 00000000 /usr/openv/var/vnetd/terminate_vnetd.uds
-bash-3.00$ netstat -an |grep -i pbx
 
 
There is no CLIENT_CONNECT_TIMEOUT value in this server's bp.conf. 
 
NBU MASTER :7.1
client :7.1
 
-bash-3.00$ cat version
NetBackup-Solaris10 7.1.0.2
 
 
client OS:Sun 5.10 Generic_147440-12 sun4u sparc SUNW,SPARC-Enterprise
 
 
 
-bash-3.00$ sudo ./bpclntcmd -pn
expecting response from server uv23
ukunix12   ukunix12   13.129.39.95
-bash-3.00$ sudo ./bpclntcmd -self
yp_get_default_domain failed: (6) internal yp server or client error
NIS does not seem to be running: (1) args to yp function are bad
gethostname() returned: ukunix12   
host ukunix12   : ukunix12.mit.apne.com    at 13.11.39.95
aliases:     ukunix12.mit.apne.com     ukunix12        13.11.39.95
nike_new's picture

Nothing comes in bpcd logs after bptestbpcd command was completed with below error after 30 minutes.

<16>bptestbpcd main: Function ConnectToBPCD(ukunix12  ) failed: 25

09:57:21.226 [16414] <16> bptestbpcd main: Function ConnectToBPCD(ukunix12  ) failed: 25

<2>bptestbpcd: cannot connect on socket

09:57:21.226 [16414] <2> bptestbpcd: cannot connect on socket

<2>bptestbpcd: EXIT status = 25

09:57:21.226 [16414] <2> bptestbpcd: EXIT status = 25

cannot connect on socket

Do i need to run the backup manually for bpcd logs to be created or just run bptestbpcd from my master server?

RamNagalla's picture
-bash-3.00$ sudo ./bpclntcmd -pn
expecting response from server uv23
ukunix12   ukunix12   13.129.39.95
-bash-3.00$ sudo ./bpclntcmd -self
yp_get_default_domain failed: (6) internal yp server or client error
NIS does not seem to be running: (1) args to yp function are bad
gethostname() returned: ukunix12   
host ukunix12   : ukunix12.mit.apne.com    at 13.11.39.95
aliases:     ukunix12.mit.apne.com     ukunix12        13.11.39.95
 
why there are 2 different IP address ,
does your client ukunix12 have multiple  IP address?
 
show us ifconfig -a from client
 
and also below commands from both master and client
 
nslookup 13.129.39.95
nslookup 13.11.39.95
nslookup ukunix12
 
grep 13.129.39.95 /etc/hosts
grep 13.11.39.95 /etc/hosts
grep ukunix12 /etc/hosts
 
 
nike_new's picture

Thats a typo.

It has only one ip address

nike_new's picture

13.129.39.95 is the ip address of client.

nike_new's picture
-bash-3.00$ netstat -an |grep -i pbx
-bash-3.00$
 
 
It is showing no output.Why?
 
Currently backup is not running.It will show when my backup will run
?
RamNagalla's picture

I don't think pbx would show in netstat.

i just reliased that i have asked that also, dont mind

I dont think IP address is Typo..

see your intial post its showing the IP 13.11.39.95., you just have to make sure whats wrong with your IPs

07:46:29.952 [12028] <2> logconnections: BPCD CONNECT FROM 13.11.31.32.40707 TO 13.11.39.95.1556 fd = 3

07:46:29.953 [12028] <2> vnet_pbxConnect: pbxConnectEx Succeeded
 
 
you did not provide the below output from both master and client yet.
 
nslookup 13.129.39.95
nslookup 13.11.39.95
nslookup ukunix12
 
grep 13.129.39.95 /etc/hosts
grep 13.11.39.95 /etc/hosts
grep ukunix12 /etc/hosts
Marianne's picture

Nothing comes in bpcd logs

Firewall between server and client?

Check/verify that the following processes are running on your 7.1 client:
bpcd -standalone
vnetd -standalone
pbx_exchange

Use this command to check:

ps -ef | egrep 'vnetd|bpcd|pbx'
 
You should see output like this:
 
root     28682     1  0 09:28 ?        00:00:00 /opt/VRTSpbx/bin/pbx_exchange
root     28709     1  0 09:28 ?        00:00:00 /usr/openv/netbackup/bin/vnetd -standalone
root     28712     1  0 09:28 ?        00:00:00 /usr/openv/netbackup/bin/bpcd -standalone

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

nike_new's picture

bpcd logs comes when i fire the backup or when i run bptestbpcd from master server?

RamNagalla's picture

it should come for both,

it logs any activity that communicates with bpcd process..

nike_new's picture

@Nagalla:Why pbx is not showing in netstat?

Pbx runs everytime on client or when the backup runs?

RamNagalla's picture

becuase its a process, and it runs everytime.

what about your IP address issue? 

why its showing 2 IP address?

CRZ's picture

Can we not go off on tangents and try resolving your issue?

You still have not explained the IP address discrepancy Nagalla has brought up.

Could you please provide output of `ifconfig -a` run from ukunix12 in addition to the other nslookup and hosts commands previously requested by Nagalla?

(For what it's worth, when *I* nslookup ukunix12.mit.apne.com, I get 66.246.235.44 and THAT takes me to a domain name landing page where it says "apne.com is for sale!" ...so I question whether this is a real hostname or something you've just made up for some reason.)


bit.ly/76LBN | APPLBN | 75LBN

nike_new's picture
-bash-3.00$ ifconfig -a
lo0:1: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
bge0:2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet 13.129.39.95 netmask ffffff00 broadcast 13.129.39.255
bge0:3: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet 13.129.39.155 netmask ffffff00 broadcast 13.129.39.255
bge0:4: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet 13.129.39.175 netmask ffffff00 broadcast 13.129.39.255
bge0:5: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet 13.129.39.154 netmask ffffff00 broadcast 13.129.39.255
nxge1:2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 5
        inet 10.5.103.38 netmask ffff0000 broadcast 10.5.255.255
-bash-3.00$
 
 
W are using below ip
13.129.39.255
 
 
 
 
 
RamNagalla's picture

your Previous post saying its has only one IP

 https://www-secure.symantec.com/connect/forums/bptestbpcd-just-hangs#comment-8191701

Marianne's picture

It is not possible to be using 13.129.39.255

IP addresses on this machine are:

13.129.39.95
13.129.39.155
13.129.39.175
13.129.39.154
10.5.103.38

This means that the server where you are running bptestbpcd has the wrong IP address for client ukunix12:

BPCD CONNECT FROM 13.11.31.32.40707 TO 13.11.39.95.1556 fd = 3

This is why nothing is logged in bpcd on ukunix12. Connection request is going to wrong IP address.
Check /etc/hosts entries for ukunix12 on the server where you are running bptestbpcd: 

grep ukunix12 /etc/hosts

It seems that the correct IP address for this client should be 13.129.39.95.

Connection should be fine if correct IP address in /etc/hosts is used or maybe DNS.

Nagalla has pointed all of this out 3 days ago: https://www-secure.symantec.com/connect/forums/bptestbpcd-just-hangs#comment-8191691

This issue could have been resolved on the same day....

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

nike_new's picture

13.129.39.95

This is the ip address

bpcd logs:

00:00:47.146 [27119] <2> setup_debug_log: switched debug log file for bpcd

00:00:47.153 [9769] <2> unix_daemonize: No close(3) for DOOR file mode 0x0000d124
00:00:47.153 [9769] <2> unix_daemonize: 0 = close(4) errno = 2 mode = 0x00008180
00:00:47.153 [9769] <2> unix_daemonize: 0 = close(5) errno = 2 mode = 0x0000c1b6
00:00:47.153 [9769] <2> unix_daemonize: 0 = close(6) errno = 2 mode = 0x0000c1b6
00:00:47.153 [9769] <2> unix_daemonize: 0 = close(7) errno = 2 mode = 0x0000c1b6
00:00:47.153 [9769] <2> unix_daemonize: 0 = close(8) errno = 2 mode = 0x0000c1b6
00:00:47.153 [9769] <2> unix_daemonize: 0 = close(9) errno = 2 mode = 0x00001180
00:05:56.337 [27119] <2> vnet_pbxAcceptSocket: Accepted sock[11] from 13.129.31.32:44447
00:05:56.338 [27119] <2> bpcd main: accept sock = 11
00:05:56.366 [10756] <2> process_requests: offset to GMT 18000
00:05:56.366 [10756] <2> logconnections: BPCD ACCEPT FROM 13.129.31.32.44447 TO 13.129.39.95.1556 fd = 0
00:05:56.367 [10756] <2> process_requests: setup_sockopts complete
 
00:05:56.369 [10756] <4> bpcd valid_server: hostname comparison succeeded
 

00:05:56.369 [10756] <2> process_requests: output socket port number = 1

00:05:56.574 [10756] <2> process_requests: Duplicated vnetd socket on stderr
00:05:56.574 [10756] <2> process_requests: <---- NetBackup 7.1 0 ------------initiated
00:05:56.574 [10756] <2> process_requests: VERBOSE = 0
00:05:56.574 [10756] <2> process_requests: Not using VxSS authentication with uv23
00:05:56.626 [10756] <2> process_requests: BPCD_GET_STDOUT_SOCKET_RQST
00:05:56.626 [10756] <2> process_requests: socket port number = 1
 
 
 
 
 
Marianne's picture

This means IP address for client ukunix12 is correct on server uv23 (IP address 13.129.31.32)

The server where you did bptestbpcd in your opening post has IP address of 13.11.31.32.

You need to carefully check IP addresses and hostnames in /etc/hosts for ALL servers and clients.

If you have a backup network, you need to ensure that each IP address has a different hostname in /etc/hosts and/or DNS.

Ensure that correct hostnames are used in NBU.

You said 3 days ago that nothing is logged in bpcd log.
Now we see connection attempt in bpcd.

Question:

WHAT HAS CHANGED??
 

Have you followed any of the advice above?

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

SOLUTION
RamNagalla's picture

what is this IP address 13.129.31.32?

does it master server IP?

nike_new's picture

Hi ALL,

IP was not changed.I am not able to telnet the bpcd port as per bptestbpcd o/p

What needs to be done for this

Marianne's picture

Seems you are battling to understand what we are trying to tell you. Please read through my previous post again.

I have nothing else to say...

Good luck!

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

RamNagalla's picture

Nike,

its looks like its time for you to go for a support call,  symantec support able to take the webex and directly see whats wrong and where.

symantec support call is best for you.

good luck.