Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

NetBackup Storage Server - VxSS Access Denied(117)

Created: 24 Oct 2012 | 19 comments
mrmadej's picture

Hello All,

 

I have a strange problem with creating the Storage Server on AIX media server.

I am getting the message: "VxSS Access Denied(117)". I have checked that the space on filesystems is enough. The user which is doing this is root with all rights. And on the others AIX media servers I can create the Storag Server without problems. The AIX configuration is the same on all AIX servers.

I don't know what can be the problem and where I should looking the reason.

Can You help me what should be checked? Or any others idea what can be the cause of my problem?

 

The environment is:

NetBackup 7.0.1 - master, media and clients

Master Server is RedHat 5.5

Media Server is AIX 7.1

 

Thanks & regards

Jakub Madejski

Comments 19 CommentsJump to latest comment

Marianne's picture

Storage Server for which feature?

Usin GUI or cmd?

Please show us command and output or else screenshot.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

mrmadej's picture

Hi,

Thx for reply.

Storage Server for AdvancedDisk.

I have little screenshot from GUI.

Administrator will send me command output in the near future.

 

Regards

Jakub Madejski

sazz.'s picture

Have you checked that you have got the same licese on this media server as well.

From this media server are you able to resolve the master server and vice versa? Make sure all the communication to master server are working properly from this server. Use bpclntcmd to check everything resolves fine.

Make sure you have full rights while running the command.

mrmadej's picture

I am sure that the license is OK - permanent trial license, so, all features are enabled.

Name resolving is working properly based on /etc/hosts file. That was checked few times.

On all media servers all activities are performing by root user with full rights. On master server also.

Can You advice which logs on which server I should check? Logs which was checked shows nothing.

 

Regards

mrmadej's picture

Below is output form CLI:

 

[root@eufrat ~]# nbdevconfig -creatests -storage_server enklawa3 -stype AdvancedDisk -media_server enklawa3 -st 5

Failed to create storage server enklawa3, cannot connect on socket

[root@eufrat ~]# nbdevconfig -creatests -storage_server enklawa3.xxxxx.pl -stype AdvancedDisk -media_server enklawa3.xxxxx.pl -st 5

Failed to create storage server enklawa3.xxxxx.pl, cannot connect on socket

[root@eufrat ~]# telnet enklawa3 1556

Trying xx.xx.xx.xx....

Connected to enklawa3.

Escape character is '^]'.

^]

telnet> quit

Connection closed.

[root@eufrat ~]# telnet enklawa3 13724

Trying xx.xx.xx.xx....

Connected to enklawa3.

Escape character is '^]'.

^]

telnet> quit

Connection closed.

[root@eufrat ~]# telnet enklawa3.xxxxx.pl 1556

Trying xx.xx.xx.xx....

Connected to enklawa3.xxxxx.pl.

Escape character is '^]'.

^]

telnet> quit

Connection closed.

[root@eufrat ~]# telnet enklawa3.xxxxx.pl 13724

Trying xx.xx.xx.xx...

Connected to enklawa3.xxxxx.pl.

Escape character is '^]'.

^]

telnet> quit

Connection closed.

[root@eufrat ~]#

 

Regards

watsons's picture

Enable nbrmms log on the media server:

vxlogcfg -a -p NB -o 222 -s DebugLevel=6

Re-run the creatests either on GUI or command line, and check the nbrmms log in /usr/openv/logs/nbrmms/* for more details.

Marianne's picture

Please create bpcd log folder on enklawa3.xxxxx.pl.

Run the following on master:

bptestbpcd -host enklawa3.xxxxx.pl -debug -verbose

Please post output of command as well as bpcd log file.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

mrmadej's picture

Here is an output form master server:

[root@eufrat ~]# bptestbpcd -host enklawa3 -debug -verbose
14:02:56.479 [7197] <2> bptestbpcd: VERBOSE = 0
14:02:56.481 [7197] <2> ParseConfigExA: Option (OPS_CENTER_SERVER_NAME) redefined on line 126, discarding definition on line 125.
14:02:56.482 [7197] <2> file_to_addrinfo: ../../libvlibs/vnet_addrinfo.c.5663: 0: fopen() failed: 2 0x00000002
14:02:56.482 [7197] <2> file_to_addrinfo: ../../libvlibs/vnet_addrinfo.c.5664: 0: fopen() failed: /usr/openv/var/host_cache/0f1/364b10f1+0,1,402,2,1,0+enklawa3.txt
14:02:56.485 [7197] <2> file_to_addrinfo: ../../libvlibs/vnet_addrinfo.c.5663: 0: fopen() failed: 2 0x00000002
14:02:56.485 [7197] <2> file_to_addrinfo: ../../libvlibs/vnet_addrinfo.c.5664: 0: fopen() failed: /usr/openv/var/host_cache/0f1/364b10f1+veritas_pbx,1,20,2,1,0+enklawa3.txt
14:02:56.488 [7197] <2> file_to_addrinfo: ../../libvlibs/vnet_addrinfo.c.5663: 0: fopen() failed: 2 0x00000002
14:02:56.488 [7197] <2> file_to_addrinfo: ../../libvlibs/vnet_addrinfo.c.5664: 0: fopen() failed: /usr/openv/var/host_cache/0f1/364b10f1+vnetd,1,20,2,1,0+enklawa3.txt
14:02:56.490 [7197] <2> file_to_addrinfo: ../../libvlibs/vnet_addrinfo.c.5663: 0: fopen() failed: 2 0x00000002
14:02:56.490 [7197] <2> file_to_addrinfo: ../../libvlibs/vnet_addrinfo.c.5664: 0: fopen() failed: /usr/openv/var/host_cache/0f1/364b10f1+bpcd,1,20,2,1,0+enklawa3.txt
14:02:56.495 [7197] <2> vnet_pbxConnect: pbxConnectEx Succeeded
14:02:56.496 [7197] <2> logconnections: BPCD CONNECT FROM xx.xx.xx.xx.44206 TO xx.xx.xx.xx.1556 fd = 3
14:02:56.502 [7197] <2> vnet_pbxConnect: pbxConnectEx Succeeded
14:02:56.504 [7197] <2> do_pbx_service: ../../libvlibs/vnet_connect.c.1716: 0: via PBX: VNETD CONNECT FROM xx.xx.xx.xx.47604 TO xx.xx.xx.xx.1556 fd = 4
14:02:56.504 [7197] <2> vnet_vnetd_connect_forward_socket_begin: ../../libvlibs/vnet_vnetd.c.540: 0: VN_REQUEST_CONNECT_FORWARD_SOCKET: 10 0x0000000a
14:02:56.716 [7197] <2> vnet_vnetd_connect_forward_socket_begin: ../../libvlibs/vnet_vnetd.c.557: 0: ipc_string: /tmp/vnet-24260351166576697644000000001-8Laoqb
14:02:56.761 [7197] <2> local_bpcr_connect: expected reserved port, got 1556
1 1 1
xx.xx.xx.xx:44206 -> xx.xx.xx.xx:1556
xx.xx.xx.xx:47604 -> xx.xx.xx.xx:1556
14:02:56.809 [7197] <2> bpcr_get_peername_rqst: Server peername length = 17
14:02:56.825 [7197] <2> bpcr_get_hostname_rqst: Server hostname length = 19
14:02:56.827 [7197] <2> bpcr_get_clientname_rqst: Server client name length = 19
14:02:56.898 [7197] <2> bpcr_get_version_rqst: bpcd version: 07010000
14:02:56.899 [7197] <2> bpcr_get_platform_rqst: Server client platform length = 9
14:02:56.902 [7197] <2> bpcr_get_version_rqst: bpcd version: 07010000
14:02:57.101 [7197] <2> bpcr_patch_version_rqst: theRest == > <
14:02:57.102 [7197] <2> bpcr_get_version_rqst: bpcd version: 07010000
14:02:57.154 [7197] <2> bpcr_patch_version_rqst: theRest == > <
14:02:57.155 [7197] <2> bpcr_get_version_rqst: bpcd version: 07010000
PEER_NAME = eufrat.xxxxx.pl
HOST_NAME = enklawa3.xxxxx.pl
CLIENT_NAME = enklawa3.xxxxx.pl
VERSION = 0x07010000
PLATFORM = rs6000_53
PATCH_VERSION = 7.0.1.0
SERVER_PATCH_VERSION = 7.0.1.0
MASTER_SERVER = ilomilo.xxxxx.pl
EMM_SERVER = ilomilo.xxxxx.pl
<2>bptestbpcd: EXIT status = 0
14:02:57.298 [7197] <2> bptestbpcd: EXIT status = 0

 

And here is a piece of bpcd log:

 

14:02:56.495 [5439696] <2> vnet_pbxAcceptSocket: Accepted sock[10] from xx.xx.xx.xx:44206
14:02:56.495 [5439696] <2> bpcd main: accept sock = 10
14:02:56.498 [10092630] <2> process_requests: offset to GMT -3600
14:02:56.498 [10092630] <2> logconnections: BPCD ACCEPT FROM xx.xx.xx.xx.44206 TO xx.xx.xx.xx.1556 fd = 0
14:02:56.498 [10092630] <2> process_requests: setup_sockopts complete
14:02:56.504 [10092630] <2> bpcd peer_hostname: Connection from host eufrat.xxxxx.pl (xx.xx.xx.xx) port 44206
14:02:56.504 [10092630] <2> bpcd valid_server: comparing ilomilo.xxxxx.pl and eufrat.xxxxx.pl
14:02:56.531 [10092630] <2> bpcd valid_server: comparing eufrat.xxxxx.pl and eufrat.xxxxx.pl
14:02:56.531 [10092630] <4> bpcd valid_server: hostname comparison succeeded
14:02:56.536 [10092630] <2> process_requests: output socket port number = 1
14:02:56.759 [10092630] <2> process_requests: Duplicated vnetd socket on stderr
14:02:56.759 [10092630] <2> process_requests: <---- NetBackup 7.0 0 ------------initiated
14:02:56.759 [10092630] <2> process_requests: VERBOSE = 0
14:02:56.759 [10092630] <2> process_requests: Not using VxSS authentication with eufrat.xxxxx.pl
14:02:56.805 [10092630] <2> process_requests: BPCD_PEERNAME_RQST
14:02:56.808 [10092630] <2> bpcd peer_hostname: Connection from host eufrat.xxxxx.pl (xx.xx.xx.xx) port 44206
14:02:56.811 [10092630] <2> process_requests: BPCD_HOSTNAME_RQST
14:02:56.827 [10092630] <2> process_requests: BPCD_CLIENTNAME_RQST
14:02:56.897 [10092630] <2> process_requests: BPCD_GET_VERSION_RQST
14:02:56.898 [10092630] <2> process_requests: BPCD_GET_PLATFORM_RQST
14:02:56.901 [10092630] <2> process_requests: BPCD_GET_VERSION_RQST
14:02:56.902 [10092630] <2> process_requests: BPCD_PATCH_VERSION_RQST
14:02:57.098 [10092630] <2> retrieveLocalPatchVersion: Reading from /usr/openv/netbackup/bin/version
14:02:57.099 [10092630] <2> parsePatchVersionString: parsing = >7.0.1
<
14:02:57.099 [10092630] <2> parsePatchVersionString: theRest = ><
14:02:57.101 [10092630] <2> process_requests: BPCD_GET_VERSION_RQST
14:02:57.102 [10092630] <2> process_requests: BPCD_PATCH_VERSION_RQST
14:02:57.102 [10092630] <2> retrieveLocalPatchVersion: Reading from /usr/openv/netbackup/version
14:02:57.112 [10092630] <2> parsePatchVersionString: parsing = >7.0.1
<
14:02:57.112 [10092630] <2> parsePatchVersionString: theRest = ><
14:02:57.155 [10092630] <2> process_requests: BPCD_GET_VERSION_RQST
14:02:57.156 [10092630] <2> process_requests: BPCD_READ_HOST_CONFIG_RQST
14:02:57.298 [10092630] <2> process_requests: BPCD_DISCONNECT_RQST
14:02:57.298 [10092630] <2> bpcd exit_bpcd: exit status 0  ----------->exiting
14:03:07.892 [5439696] <2> vnet_pbxAcceptSocket: Accepted sock[10] from xx.xx.xx.xx:51222
14:03:07.892 [5439696] <2> bpcd main: accept sock = 10
14:03:07.895 [20512834] <2> process_requests: offset to GMT -3600
14:03:07.895 [20512834] <2> logconnections: BPCD ACCEPT FROM xx.xx.xx.xx.51222 TO xx.xx.xx.xx.1556 fd = 0
14:03:07.895 [20512834] <2> process_requests: setup_sockopts complete
14:03:07.900 [20512834] <2> bpcd peer_hostname: Connection from host eufrat.xxxxx.pl (xx.xx.xx.xx) port 51222
14:03:07.900 [20512834] <2> bpcd valid_server: comparing ilomilo.xxxxx.pl and eufrat.xxxxx.pl
14:03:07.912 [20512834] <2> bpcd valid_server: comparing eufrat.xxxxx.pl and eufrat.xxxxx.pl
14:03:07.912 [20512834] <4> bpcd valid_server: hostname comparison succeeded
14:03:07.916 [20512834] <2> process_requests: output socket port number = 1
14:03:07.976 [20512834] <2> process_requests: Duplicated vnetd socket on stderr
14:03:07.976 [20512834] <2> process_requests: <---- NetBackup 7.0 0 ------------initiated
14:03:07.976 [20512834] <2> process_requests: VERBOSE = 0
14:03:07.976 [20512834] <2> process_requests: Not using VxSS authentication with eufrat.xxxxx.pl
14:03:08.020 [20512834] <2> process_requests: BPCD_FORK_CMD_RQST
14:03:08.021 [20512834] <2> process_requests: fork cmd = /usr/openv/netbackup/bin/bptm bptm -unload -dn Drive012 -dp /dev/rmt2.1 -dk 2000196 -m 402253 -mk 4000208 -mds 0 -alocid 265078219 -jobid -1350809727 -jm
14:03:08.021 [20512834] <2> process_requests: filter type = -1
14:03:08.021 [20512834] <2> process_requests: backuptime before fork = 0
14:03:08.023 [19726350] <2> process_requests: Converting args string into an argv
14:03:08.023 [19726350] <2> process_requests: Done converting args string into an argv
14:03:08.023 [19726350] <2> process_requests: child_args[0] = /usr/openv/netbackup/bin/bptm
14:03:08.023 [19726350] <2> process_requests: child_args[1] = bptm
14:03:08.023 [19726350] <2> process_requests: child_args[2] = -unload
14:03:08.023 [19726350] <2> process_requests: child_args[3] = -dn
14:03:08.023 [19726350] <2> process_requests: child_args[4] = Drive012
14:03:08.023 [19726350] <2> process_requests: child_args[5] = -dp
14:03:08.023 [19726350] <2> process_requests: child_args[6] = /dev/rmt2.1
14:03:08.023 [19726350] <2> process_requests: child_args[7] = -dk
14:03:08.023 [19726350] <2> process_requests: child_args[8] = 2000196
14:03:08.023 [19726350] <2> process_requests: child_args[9] = -m
14:03:08.023 [19726350] <2> process_requests: child_args[10] = 402253
14:03:08.023 [19726350] <2> process_requests: child_args[11] = -mk
14:03:08.023 [19726350] <2> process_requests: child_args[12] = 4000208
14:03:08.023 [19726350] <2> process_requests: child_args[13] = -mds
14:03:08.023 [19726350] <2> process_requests: child_args[14] = 0
14:03:08.023 [19726350] <2> process_requests: child_args[15] = -alocid
14:03:08.023 [19726350] <2> process_requests: child_args[16] = 265078219
14:03:08.023 [19726350] <2> process_requests: child_args[17] = -jobid
14:03:08.023 [19726350] <2> process_requests: child_args[18] = -1350809727
14:03:08.023 [19726350] <2> process_requests: child_args[19] = -jm
14:03:08.023 [19726350] <2> process_requests: Before execvp of command
14:03:08.032 [20512834] <2> bpcd exit_bpcd: exit status 0  ----------->exiting

 

Regards

Jakub

watsons's picture

VxSS sounds to be a NBAC permission issue.

Have you done "bpnbat -login" before running that command?

Mark_Solutions's picture

Just a check here ... enklawa3.xxxxx.pl is the media server you are trying to create as a storage server,  eufrat.xxxxx.pl is where you are running the command from and ilomilo.xxxxx.pl is the Master Server.

Does ilomilo.xxxxx.pl have rights to do that on the Master and Media Server (additional server)

Have you tried configuring it from the Master Server?

Do ensure that you do have the right license on the media server.

Hope this helps

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

mrmadej's picture

Hi,

ilomilo and eufrat is the same server - master server. This is cluster.

I have ran the command from Master Server as root.

License was checked and is ok.

Regards

Jakub

Marianne's picture

Please show us output of 'nbemmcmd -listhosts -verbose'.

Does media server enklawa3 function normally as media server (e.g. basic STU)?

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

mrmadej's picture

This command lists ~800 machines :-)

Yes, enklawa is workiing as media server properly.

 

Regards

Jakub

Marianne's picture

~800? This sounds like production environment with 'permanent trial license' in place?

Are you using your partner NFR key in production?

Please show output for this media server:

nbemmcmd -listhosts -verbose -display_server -machinename <media-server> -machinetype media

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

mrmadej's picture

The license is a special license dedicated for this client and is compliant with licensing rules.

Below is output.

 

[root@eufrat ~]# nbemmcmd -listhosts -verbose -display_server -machinename enklawa3 -machinetype media

NBEMMCMD, Version:7.0.1

enklawa3.xxxxx.pl

        ClusterName = ""

        MachineName = "enklawa3.xxxxx.pl"

        FQName = "enklawa3.xxxxx.pl"

        LocalDriveSeed = ""

        MachineDescription = ""

        MachineFlags = 0x17

        MachineNbuType = media (1)

        MachineState = active for tape and disk jobs (14)

        MasterServerName = "ilomilo.xxxxx.pl"

        NetBackupVersion = 7.0.1.0 (701000)

        OperatingSystem = rs6000 (5)

        ScanAbility = 5

Command completed successfully.

 

Regards

Jakub

Marianne's picture

I cannot see any reason why command is failing with 'cannot connect on socket'.
All comms tests are fine.

Similar issue in this post, but I think we have done all comms tests: https://www-secure.symantec.com/connect/forums/creating-media-server-existing-master-server  

Maybe time to log a Support Call? 

 

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

mrmadej's picture

Yes, I know, this is very strange behavior.

The support case is logged but they are working very slow on this issue.

Thanks a lot for suggestions.

 

Ragards

Jakub