Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

MSSQL 2008 server suddenly stopped backing up logs

Created: 15 Jan 2013 • Updated: 17 Jan 2013 | 14 comments
This issue has been solved. See solution.

I have a Microsoft MSSQL 2008 server that has backed up without incident for months.  Without any known changes and none from Netbackup's perspective MSSQL stopped backing up on Saturday 1/12/2013 and hasn't been successful since. 

Job Overvies:

1: (59) access to the client was not allowed
2: (59) access to the client was not allowed

Detail Staus:

01/13/2013 14:42:14 - Info nbjm (pid=31190) requesting MEDIA_SERVER_WITH_ATTRIBUTES resources from RB for backup job (jobid=960795, request id:{B622B352-5DC1-11E2-ACDF-56450F757AAE})
01/13/2013 14:42:14 - requesting resource su_dtc_dd670dtc
01/13/2013 14:42:14 - requesting resource utlp1227.NBU_CLIENT.MAXJOBS.RDBP5060
01/13/2013 14:42:14 - requesting resource utlp1227.NBU_POLICY.MAXJOBS.MSSQL-General-LOGS
01/13/2013 14:42:14 - granted resource  utlp1227.NBU_CLIENT.MAXJOBS.RDBP5060
01/13/2013 14:42:14 - granted resource  utlp1227.NBU_POLICY.MAXJOBS.MSSQL-General-LOGS
01/13/2013 14:42:14 - granted resource  su_dtc_dd670dtc
01/13/2013 14:42:14 - estimated 0 kbytes needed
01/13/2013 14:42:14 - Info nbjm (pid=31190) started backup job for client RDBP5060, policy MSSQL-General-LOGS, schedule BiHourlyLogs on storage unit su_dtc_dd670dtc
01/13/2013 14:42:14 - started process bpbrm (pid=7580)
01/13/2013 14:42:19 - end writing
01/13/2013 14:42:20 - Error bpbrm (pid=7580) bpcd on RDBP5060 exited with status 59: access to the client was not allowed
01/13/2013 14:42:20 - Error bpbrm (pid=7580) cannot send mail to operations@xxxxx
access to the client was not allowed  (59)

However, the file level backups on the same server backs up perfectly.  Also Netbackup can connect to the client and I can see that the Master, Media and OpsCenter server entries are all correct.

Have any of you seen this behaviour and importantly know a cause / fix for it.

Comments 14 CommentsJump to latest comment

Will Restore's picture

see this old thread for solution:

https://www-secure.symantec.com/connect/forums/status-code-59-access-client-was-not-allowed-0#comment-7173071

 

client must have both Master and Media servers listed in its Server list

 

 

Will Restore -- where there is a Will there is a way

trs06's picture

Please see the attached document. 

From the first post above:  "

However, the file level backups on the same server backs up perfectly.  Also Netbackup can connect to the client and I can see that the Master, Media and OpsCenter server entries are all correct."

AttachmentSize
Amaebs10 is the master server.docx 100.42 KB
Will Restore's picture

OK, thanks, what about server  utlp1227  as shown in your first post ?

 

Will Restore -- where there is a Will there is a way

Marianne's picture

Please post client's bpcd log as File attachment.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

Yasuhisa Ishikawa's picture

As per job detail in first post, utlp1227 is master server of NetBackup domain, but is not listed in client's server list.

Be sure in which domain you are trying to backup SQL Server DB, and add all the servers in that domain into client's server list.

Authorized Symantec Consultant(ASC) Data Protection in Tokyo, Japan

Will Restore's picture

thank you, my point exactly

Will Restore -- where there is a Will there is a way

trs06's picture

amaebs10 is simply an alias for utlp1227.  Either entry in the Servers List is completely satisfactory.

amaebs11 is an alias for utlp5178 and amaebs12 is an alias for utlp5179

utlp2010 and utlp2011 are remote Media Servers that do client side dedup and backup across wan connections.

 

Please focus otherwise.  This is not the issue.  Also again I refer back to my fist post -> the client has backed up successfully for many months.  Nothing in the Client or Netbackup configuration changed.  The file level backup runs flawlessly (thus verifying the Master and Media Server entries).

In addition, we have tens of other MSSQL servers using the same policy, same credentials, same storage group, same SLP, Same backup  hardware and network - all without errors or problems.

Marianne's picture

Fact is - SOMETHING has changed, maybe not in NBU, maybe reverse lookup in your DNS. Process flow is different for database backups - authentication with master and media server is needed. Filesystem backup needs only media server <-> client connectivity.

We will ONLY know what the problem is when we see client's bpcd log that contains evidence of failed connection attempt.

Please post client's bpcd log as File attachment.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

trs06's picture

Thank You.  Attached are some bpcd logs zipped up.

 

Today the File level backup also failed with status 59.

AttachmentSize
utlp5060_bpcd.zip 175.32 KB
trs06's picture

10.193.97.60 is our Master Server

10.193.80.31 and .32 are two windows 2008 Media Servers, backup interface

10.193.161.13 is RDBP5060 and backup interface in our DMZ

10.193.142.13 is RDBP5060 production interface in our DMZ

On the Master and Media Servers RDBP5060 resolves correctly to the DMZ backup interface.

Ping and nslookup

All servers reply correctly to bpclntcmd -hs ... and other appropriate options.

Will Restore's picture

now there's another hostname?

 

AMA_NT is not a master server

AMA_NT is not a media server either

 

 

Will Restore -- where there is a Will there is a way

trs06's picture

AMA_NT is a Windows Domain not a server.  I hadn't looked for that in the bpcd log.  Now I'm really confused at why it is looing at the domain as if it were a host.

This is a great find.  I didn't even think to look for such a thing.  It gives us some direction to try and find why the systems think that AMA_NT is a host and, the first entry in the 1/12 log conincides with the first failed MSSQL backup.

   thx

Marianne's picture

Extract from 01/12 bpcd log:

 

02:00:32.730 [4236.4276] <2> logconnections: BPCD ACCEPT FROM 10.193.80.31.59927 TO 10.193.161.13.1556 fd = 404

02:00:32.730 [4236.4276] <2> process_requests: setup_sockopts complete

02:00:37.370 [4236.4276] <2> bpcd peer_hostname: Connection from host AMA_NT (10.193.80.31) port 59927
 
Client is resolving IP address 10.193.80.31 to hostname AMA_NT.
Check client's hosts file as well as 'nslookup 10.193.80.31' 
 
Once you have corrected this incorrect reverse lookup, remember to clear host cache on the client:
bpclntcmd -clear_host_cache

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

SOLUTION
trs06's picture

Thank you.  Our reverse DNS configuration was indeed the problem.