Video Screencast Help
Give us your opinion and win with Symantec! Please help us by taking this survey to tell us about your experience with Symantec Connect, so that we can continue to grow and improve.  Take the survey.

Windows 2003 R2 Client server backup failing w/Netbackup code (58)

Created: 06 Feb 2013 | 15 comments

Hi

We have a Windows 2003 R2 Client server backup failing  w/Netbackup code (58) here are the commands and their output.

 NetBackup Client Level 7.5

fyi - I have been told the firewall is turned off

 Master Server side commands

 [root@nbcom4 admincmd]# bpclntcmd -hn dakota.nzes.xxx.xx

host dakota.nzes.xxx.xx: dakota.nzes.xxx.xx at xx.xx.xx.xx

aliases:     dakota.nzes.xxx.xx     xx.xx.xx.xx

[root@nbcom4 admincmd]# bptestbpcd -client dakota.nzes.xxx.xx -verbose

<16>bptestbpcd main: Function ConnectToBPCD(dakota.nzes.xxx.xx) failed: 25

cannot connect on socket

[root@nbcom4 admincmd]# ./bptestbpcd -client dakota.nzes.xxx.xx

<16>bptestbpcd main: Function ConnectToBPCD(dakota.nzes.xxx.xx) failed: 25

cannot connect on socket

[root@nbcom4 admincmd]# nslookup dakota.nzes.xxx.xx

Server:         xx.xx.xx.xx

Address:        xx.xx.xx.xx#53

Name:   dakota.nzes.xxx.xx

Address: xx.xx.xx.xx

[root@nbcom4 admincmd]# ping dakota.nzes.xxx.xx

PING dakota.nzes.xxx.xx (xx.xx.xx.xx) 56(84) bytes of data.

64 bytes from dakota.nzes.xxx.xx (xx.xx.xx.xx): icmp_seq=1 ttl=125 time=0.368 ms

64 bytes from dakota.nzes.xxx.xx (xx.xx.xx.xx): icmp_seq=2 ttl=125 time=0.323 ms

64 bytes from dakota.nzes.xxx.xx (xx.xx.xx.xx): icmp_seq=3 ttl=125 time=0.294 ms

[root@nbcom4 admincmd]# bptestbpcd -client dakota.nzes.xxx.xx -verbose -debug

13:06:47.933 [76544] <2> bptestbpcd: VERBOSE = 0

13:06:47.933 [76544] <2> ConnectionCache::connectAndCache: Acquiring new connection for host nbcom4.my.company.com, query type 223

13:06:47.934 [76544] <2> logconnections: BPDBM CONNECT FROM xx.xx.xx.xx.58375 TO xx.xx.xx.xx.13721 fd = 3

13:06:47.935 [76544] <8> vnet_get_user_credential_path: [vnet_vxss.c:1472] status 35 0x23

13:06:47.935 [76544] <8> vnet_get_credential_path: [vnet_vxss.c:1099] vnet_get_user_credential_path failed 35 0x23

13:06:47.935 [76544] <8> vnet_determine_vxss_socket: [vnet_vxss.c:1754] non-fatal vnet_get_credential_path() failed 35 0x23

13:06:47.935 [76544] <8> vnet_get_oc: [vnet_vxss_helper.c:3122] Invalid Argument: vxss_coninfo

13:06:47.935 [76544] <8> vnet_check_vxss_client_magic_with_info: [vnet_vxss_helper.c:922] VxSS magic 1450726227 0x56785353

13:06:47.935 [76544] <8> vnet_check_vxss_client_magic_with_info: [vnet_vxss_helper.c:923] *use_vxss 2 0x2

13:06:47.937 [76544] <2> db_CLIENTsend: reset client protocol version from 0 to 8

13:06:47.978 [76544] <2> db_getCLIENT: db_CLIENTreceive: no entity was found 227

13:07:51.038 [76544] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_ERROR returned 110 0x6e

13:08:03.050 [76544] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_ERROR returned 110 0x6e

13:08:15.063 [76544] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_ERROR returned 110 0x6e

13:08:56.039 [76544] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_ERROR returned 110 0x6e

13:09:08.052 [76544] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_ERROR returned 110 0x6e

13:09:21.065 [76544] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_ERROR returned 110 0x6e

13:10:02.109 [76544] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_ERROR returned 110 0x6e

13:10:16.112 [76544] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_ERROR returned 110 0x6e

13:10:31.125 [76544] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_ERROR returned 110 0x6e

13:11:09.112 [76544] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_ERROR returned 110 0x6e

13:11:27.124 [76544] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_ERROR returned 110 0x6e

13:11:46.137 [76544] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_ERROR returned 110 0x6e

13:12:21.167 [76544] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_ERROR returned 110 0x6e

13:12:39.188 [76544] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_ERROR returned 110 0x6e

13:12:59.139 [76544] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_ERROR returned 110 0x6e

13:12:59.140 [76544] <16> connect_to_service: connect failed STATUS (18) CONNECT_FAILED

        status: FAILED, (42) CONNECT_REFUSED; system: (110) Connection timed out; FROM 0.0.0.0 TO dakota.nzes.xxx.xx xx.xx.xx.xx bpcd VIA pbx

        status: FAILED, (42) CONNECT_REFUSED; system: (110) Connection timed out; FROM 0.0.0.0 TO dakota.nzes.xxx.xx xx.xx.xx.xx bpcd VIA vnetd

        status: FAILED, (42) CONNECT_REFUSED; system: (110) Connection timed out; FROM 0.0.0.0 TO dakota.nzes.xxx.xx xx.xx.xx.xx bpcd

13:12:59.140 [76544] <8> vnet_connect_to_bpcd: [vnet_connect.c:279] connect_to_service() failed 18 0x12

13:12:59.140 [76544] <2> local_bpcr_connect: Can't connect to client dakota.nzes.xxx.xx

13:12:59.140 [76544] <2> ConnectToBPCD: bpcd_connect_and_verify(dakota.nzes.xxx.xx, dakota.nzes.xxx.xx) failed: 25

<16>bptestbpcd main: Function ConnectToBPCD(dakota.nzes.xxx.xx) failed: 25

13:12:59.140 [76544] <16> bptestbpcd main: Function ConnectToBPCD(dakota.nzes.xxx.xx) failed: 25

<2>bptestbpcd: cannot connect on socket

13:12:59.140 [76544] <2> bptestbpcd: cannot connect on socket

<2>bptestbpcd: EXIT status = 25

13:12:59.140 [76544] <2> bptestbpcd: EXIT status = 25

cannot connect on socket

[root@nbcom4 admincmd]#

Netbackup Client side commands

C:\Program Files\VERITAS\NetBackup\bin>bpclntcmd -hn nbcom4.my.company.com

host nbcom4.my.company.com: nbcom4.my.company.com at xx.xx.xx.xx

aliases:     nbcom4.my.company.com     xx.xx.xx.xx

C:\Program Files\VERITAS\NetBackup\bin>bpclntcmd -self

gethostname() returned: dakota.nzes.xxx.xx

host dakota.nzes.xxx.xx: dakota.nzes.xxx.xx at xx.xx.xx.xx

aliases:     dakota.nzes.xxx.xx     xx.xx.xx.xx

getfqdn(dakota.nzes.xxx.xx) returned: dakota.nzes.xxx.xx

C:\Program Files\VERITAS\NetBackup\bin>bpclntcmd -pn

expecting response from server nbcom4.my.company.com

dakota.nzes.xxx.xx dakota.nzes.xxx.xx xx.xx.xx.xx 3049

C:\Program Files\VERITAS\NetBackup\bin>bpclntcmd -ip nbcom4.my.company.com

host nbcom4.my.company.com: nbcom4.my.company.com at xx.xx.xx.xx

aliases:     nbcom4.my.company.com     xx.xx.xx.xx

C:\Program Files\VERITAS\NetBackup\bin>ping nbcom4.my.company.com

Reply from xx.xx.xx.xx: bytes=32 time<1ms TTL=61

Reply from xx.xx.xx.xx: bytes=32 time<1ms TTL=61

Reply from xx.xx.xx.xx: bytes=32 time<1ms TTL=61

Reply from xx.xx.xx.xx: bytes=32 time<1ms TTL=61

Ping statistics for xx.xx.xx.xx:

    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

Approximate round trip times in milli-seconds:

    Minimum = 0ms, Maximum = 0ms, Average = 0ms

Thanks

Comments 15 CommentsJump to latest comment

RamNagalla's picture

13:12:59.140 [76544] <16> connect_to_service: connect failed STATUS (18) CONNECT_FAILED

        status: FAILED, (42) CONNECT_REFUSED; system: (110) Connection timed out; FROM 0.0.0.0 TO dakota.nzes.xxx.xx xx.xx.xx.xx bpcd VIA pbx

        status: FAILED, (42) CONNECT_REFUSED; system: (110) Connection timed out; FROM 0.0.0.0 TO dakota.nzes.xxx.xx xx.xx.xx.xx bpcd VIA vnetd

        status: FAILED, (42) CONNECT_REFUSED; system: (110) Connection timed out; FROM 0.0.0.0 TO dakota.nzes.xxx.xx xx.xx.xx.xx bpcd

why its showing IP address as 0.0.0.0?

does it ever working? or new configuration?

RamNagalla's picture

another one,

13:06:47.934 [76544] <2> logconnections: BPDBM CONNECT FROM xx.xx.xx.xx.58375 TO xx.xx.xx.xx.13721 fd = 3

why its trying to connect bprd port in destination (client) which is not correct.

what are the connect options enalbe for this client?

please let us know the output of below command

bptestbpcd -client <Clientname> -debug -verbose -connect_options 0 1 2

Stumpr2's picture

Are the NetBackup services running on the client? What are their properties and who is login?

VERITAS ain't it the truth?

Mark_Solutions's picture

"fyi - I have been told the firewall is turned off"

Dont believe everything you are told!

Do check it manually - Domain, public and private

Also check for things like McAfee Access Protection being active on the client - if you open the console up and take a look at the access protection part - logs - you will see if it is blocking your communications

Hope this helps

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

Stumpr2's picture

"fyi - I have been told the firewall is turned off"

Dont believe everything you are told!

VERITAS - ain't it the truth?

One thing I like to try is telnet to the bpcd port on master/media to client and viceversa.

and dont forget traceroute

VERITAS ain't it the truth?

Marianne's picture

The source IP address of 0.0.0.0 is perfectly normal - have seen it while doing tests in our own environment.

Important fact is that ALL connection attempts are failing:
to pbx (port 1556)
to vnetd (port 13724)
to bpcd (port 13782)

Have you created bpcd log folder on client?
If so, was anything logged after connection attempt?
If not - check firewall settings. Either external firewall between server and client or either Windows firewall on client.
If log file created, please post log as File attachment.

PS: Great to have you back, Bob!

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

nbustarter380's picture

Thanks,

Nagalla, Stumpr2, Mark and Marianne

Nagalla,

Yes, this backup was working before we had a power outage.

Here is the output of the bptestbpcd -client -debug -verbose -connect_options 0 1 2 command

[root@nbcom4 admincmd]# bptestbpcd -client dakota.nzes.xxx.xx -debug -verbose -connect_options 0 1 2
14:47:52.552 [26451] <2> bptestbpcd: VERBOSE = 0
14:47:52.552 [26451] <2> ConnectionCache::connectAndCache: Acquiring new connection for host nbucore4.oit.state.nj.us, query type 223
14:47:52.553 [26451] <2> logconnections: BPDBM CONNECT FROM 10.26.21.37.36469 TO 10.26.21.37.13721 fd = 3
14:47:52.554 [26451] <8> vnet_get_user_credential_path: [vnet_vxss.c:1472] status 35 0x23
14:47:52.554 [26451] <8> vnet_get_credential_path: [vnet_vxss.c:1099] vnet_get_user_credential_path failed 35 0x23
14:47:52.554 [26451] <8> vnet_determine_vxss_socket: [vnet_vxss.c:1754] non-fatal vnet_get_credential_path() failed 35 0x23
14:47:52.554 [26451] <8> vnet_get_oc: [vnet_vxss_helper.c:3122] Invalid Argument: vxss_coninfo
14:47:52.554 [26451] <8> vnet_check_vxss_client_magic_with_info: [vnet_vxss_helper.c:922] VxSS magic 1450726227 0x56785353
14:47:52.554 [26451] <8> vnet_check_vxss_client_magic_with_info: [vnet_vxss_helper.c:923] *use_vxss 2 0x2
14:47:52.557 [26451] <2> db_CLIENTsend: reset client protocol version from 0 to 8
14:47:52.597 [26451] <2> db_getCLIENT: db_CLIENTreceive: no entity was found 227
14:47:52.597 [26451] <8> file_to_cache_item: [vnet_addrinfo.c:6555] fopen() failed ERRNO=0 FILE=/usr/openv/var/host_cache/0e3/550556e3+0,1,402,0,1,0+dakota.nzes.xxx.xx .txt
14:47:52.598 [26451] <8> file_to_cache_item: [vnet_addrinfo.c:6555] fopen() failed ERRNO=2 FILE=/usr/openv/var/host_cache/0e3/550556e3+bpcd,1,20,2,1,0+dakota.nzes.xxx.xx .txt
14:47:52.599 [26451] <8> file_to_cache_item: [vnet_addrinfo.c:6555] fopen() failed ERRNO=2 FILE=/usr/openv/var/host_cache/0e2/e53f6ce2+0,1,402,0,1,0+10.35.40.10.txt
14:48:55.599 [26451] <8> async_connect: [vnet_connect.c:1653] getsockopt SO_ERROR returned 110 0x6e

Stumpr2,
Yes, the Netback client servers are enabled and running.They are all set to automatic

Mark,
Not sure how to check windows firewall manually? I did check it through Windows is that the same?

Marianne,

On the the client server when I enter the command >> bptestbpcd -client dakota.nzes.xxx.xx -debug –verbose
To create the bpcd log I get the following

'bptestbpcd' is not recognized as an internal or external command,
operable program or batch file.
C:\Program Files\VERITAS\NetBackup\logs>

That is the correct path on the client?

Marianne's picture

Just create a folder called bpcd under C:\Program Files\VERITAS\NetBackup\logs>

bptestbpcd is a server-only command. 
Connection attempt from server (using bptestbpcd, Host Properties or backup) will write to log file in bpcd folder if NBU Client service is running and connection allowed by firewall.

 

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

Stumpr2's picture

Have you tried from the client
telnet $MEDIASERVER bpcd
telnet $MEDIASERVER vnetd

telnet $MEDIASERVER 1556

VERITAS ain't it the truth?

nbustarter380's picture

Marianne,

Thanks,

I created the folder under >>C:\Program Files\VERITAS\NetBackup\logs

Then ran a backup (failed w/58) and tried to connect using host properties. It did not create a log the bpcd is empty

Stumpr2,

I got response from the commands

C:\Program Files\VERITAS>telnet $nbcom2 1556
Connecting To $nbucore4...Could not open connection to the host, on port 1556: C
onnect failed

C:\Program Files\VERITAS>telnet $nbcom4 vnetd
Connecting To $nbucore4...Could not open connection to the host, on port vnetd:
Connect failed

C:\Program Files\VERITAS>telnet $nbcom4 bpcd
Connecting To $nbucore4...Could not open connection to the host, on port bpcd: C
onnect failed

Stumpr2's picture

on client  check static routes to $MEDIASERVER

VERITAS ain't it the truth?

Marianne's picture

Telnet attempts plus the fact that no log is created in bpcd is enough proof that 'something' is preventing port connection - either a company firewall, Windows firewall, or AV software.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

nbustarter380's picture

Stumpr2, The command would be route -a ($MEDIASERVER)?

Also, I believe someone is going to check the server itself.

Thanks

Stumpr2's picture

I have not set them up, I have just identified that connection isn't happening.
Perhaps this would help.

Configuring Ethernet settings and routes
Article:HOWTO15599 | Created: 2009-01-30 | Updated: 2010-01-10 | Article URL http://www.symantec.com/docs/HOWTO15599

VERITAS ain't it the truth?

nbustarter380's picture

Fyi -This server is currently been looked at by the system admins

Will update soon