Video Screencast Help
Protect Your POS Environment Against Retail Data Breaches. Learn More.

Sql database completed with status code 1, as per dbclient error log, its failed with 6

Created: 08 Nov 2012 | 17 comments
NBU_13's picture

Hi,

I need help,

we configured FT media server and SAN client in our environment, file system backup jobs are running fine,

but, while try to restart sql database backup, it is completed with status code 1, but while try to restart through LAN it is completed with Status code 0, same sql database, if restart through SAN, its completed with status code 1

as per dbclient log, it is failed with status code 6

14:35:48.796 [7496.8364] <4> closeApi: INF - EXIT STATUS 6: the backup failed to back up the requested files

14:35:48.796 [7496.8364] <16> VxBSACreateObject: ERR - Could not create new image with file /solman-nw7.MSSQL7.SOLMAN-NW7.db.master.~.7.001of001.20121107143531..C.
 
Anyone help me on this issue.
 
Thanks.

 

job details :

04-Nov-12 23:21 - Info dbclient(pid=8316) DBMS MSG - ODBC return code <-1>, SQL State <37000>, SQL Message <3202><[Microsoft][SQL Native Client][SQL Server]Write on "VNBU0-8316-8564-1352082446" failed: 995(error not found)>.
04-Nov-12 23:21 - Info dbclient(pid=8316) ERR - Error in GetCommand: 0x80770004.      
04-Nov-12 23:21 - Info dbclient(pid=8316) DBMS MSG - SQL Message <3271><[Microsoft][SQL Native Client][SQL Server]A nonrecoverable I/O error occurred on file "VNBU0-8316-8564-1352082446:" 995(error not found).>
04-Nov-12 23:21 - Info dbclient(pid=8316)     CONTINUATION: - An abort request is preventing anything except termination actions.
04-Nov-12 23:21 - Info dbclient(pid=8316) DBMS MSG - SQL Message <3013><[Microsoft][SQL Native Client][SQL Server]BACKUP DATABASE is terminating abnormally.>
04-Nov-12 23:21 - Info dbclient(pid=8316) ERR - Error found executing <backup database "master" to VIRTUAL_DEVICE='VNBU0-8316-8564-1352082446' with  stats = 10, blocksize = 65536, maxtransfersize = 4194304, buffercount = 2>.
04-Nov-12 23:21 - Info dbclient(pid=8316) ERR - Error in VDS->Close: 0x80770004.      
04-Nov-12 23:21 - Info dbclient(pid=8316)     CONTINUATION: - An abort request is preventing anything except termination actions.
04-Nov-12 23:21 - Info dbclient(pid=8316) INF - OPERATION #2 of batch C:\Program Files\Veritas\NetBackup\DbExt\MsSql\Full_Backup_Online.bch FAILED with STATUS 1 (0 is normal). Elapsed time = 25(0) seconds.
04-Nov-12 23:21 - Info dbclient(pid=8316) INF - BACKUP STARTED USING       
04-Nov-12 23:21 - Info dbclient(pid=8316) Microsoft SQL Server 2005 - 9.00.5000.00 (Intel X86)    
04-Nov-12 23:21 - Info dbclient(pid=8316) Dec 10 2010 10:56:29        
04-Nov-12 23:21 - Info dbclient(pid=8316) Copyright (c) 1988-2005 Microsoft Corporation       
04-Nov-12 23:21 - Info dbclient(pid=8316) Enterprise Edition on Windows NT 5.2 (Build 3790: Service Pack 2) 
04-Nov-12 23:21 - Info dbclient(pid=8316) Batch = C:\Program Files\Veritas\NetBackup\DbExt\MsSql\Full_Backup_Online.bch, Op# = 3     
04-Nov-12 23:21 - Info dbclient(pid=8316) INF - Using backup image solman-nw7.MSSQL7.SOLMAN-NW7.db.model.~.7.001of001.20121104232753..C      
04-Nov-12 23:21 - Info dbclient(pid=8316) INF - backup database "model" to VIRTUAL_DEVICE='VNBU0-8316-7764-1352082474' with  stats = 10, blocksize = 65536, maxtransfersize = 4194304, buffercount = 2

 

dbclient logs

14:35:33.781 [7496.8364] <4> openProgressFile: cleaning directory: <C:\Program Files\Veritas\NetBackup\Logs\user_ops\dbext\jobs>
14:35:33.781 [7496.8364] <4> delete_old_files: entering delete_old_files.
14:35:33.781 [7496.8364] <4> openProgressFile: progressFD = <5>
14:35:33.781 [7496.8364] <4> openProgressFile: progressFD_en = <-1>
14:35:33.781 [7496.8364] <4> CreateNewImage: INF - backing up File:</solman-nw7.MSSQL7.SOLMAN-NW7.db.master.~.7.001of001.20121107143531..C>
14:35:33.781 [7496.8364] <4> sendRequest: entering sendRequest.
14:35:33.781 [7496.8364] <4> sendRequest: Progress log not for monitoring=<C:\Program Files\Veritas\NetBackup\Logs\user_ops\dbext\logs\8364.0.1352309733>

14:35:33.781 [7496.8364] <4> getOwnerName: entering getOwnerName.
14:35:33.781 [7496.8364] <4> sendRequest: sending BACKUP request to bprd
14:35:33.781 [7496.8364] <4> sendRequest: request = SYSTEM SYSTEM solman-nw7 solman-nw7 solman-nw7 /C/Program��Files/Veritas/NetBackup/Logs/user_ops/dbext/logs/8364.0.1352309733 GlobalCrossing-solmannw7-Online-DB NONE 0 0 15 0 C C C C C 0 0 2982798 0 5
14:35:33.781 [7496.8364] <2> bprd_connect: Ignoring local_name solman-nw7.
14:35:33.781 [7496.8364] <2> bprd_connect: Ignoring owner_name SYSTEM.
14:35:33.781 [7496.8364] <2> vnet_sortaddrs: ../../libvlibs/vnet_addrinfo.c.4067: 0: sorted addrs:: 1 0x00000001
14:35:33.781 [7496.8364] <2> get_pref_netconnection: ../../libvlibs/vnet_addrinfo.c.4932: 0: using interface : ANY
14:35:33.781 [7496.8364] <2> vnet_sortaddrs: ../../libvlibs/vnet_addrinfo.c.4067: 0: sorted addrs:: 1 0x00000001
14:35:33.781 [7496.8364] <2> get_pref_netconnection: ../../libvlibs/vnet_addrinfo.c.4932: 0: using interface : ANY
14:35:33.781 [7496.8364] <2> vnet_sortaddrs: ../../libvlibs/vnet_addrinfo.c.4067: 0: sorted addrs:: 1 0x00000001
14:35:33.781 [7496.8364] <2> get_pref_netconnection: ../../libvlibs/vnet_addrinfo.c.4932: 0: using interface : ANY
14:35:33.781 [7496.8364] <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1200: 0: connect in progress: 1 0x00000001
14:35:33.796 [7496.8364] <2> vnet_pbxConnect: pbxConnectEx Succeeded
14:35:33.796 [7496.8364] <2> do_pbx_service: ../../libvlibs/vnet_connect.c.1784: 0: via PBX: bprd CONNECT FROM 192.168.83.239.4470 TO 192.168.83.11.1556 fd = 1064
14:35:33.796 [7496.8364] <2> vnet_async_connect: ../../libvlibs/vnet_connect.c.1367: 0: connect: async CONNECT FROM 192.168.83.239.4470 TO 192.168.83.11.1556 fd = 1064
14:35:33.796 [7496.8364] <2> logconnections: BPRD CONNECT FROM 192.168.83.239.4470 TO 192.168.83.11.1556 fd = 1064
14:35:33.796 [7496.8364] <2> vnet_check_vxss_client_magic_with_info: ../../libvlibs/vnet_vxss_helper.c.843: 0: Assuming no VxSS for DB Agents: 0 0x00000000
14:35:33.796 [7496.8364] <4> serverResponse: entering serverResponse.
14:35:33.796 [7496.8364] <4> serverResponse: initial client_read_timeout = <900>
14:35:33.796 [7496.8364] <4> readCommMessages: Entering readCommMessages
14:35:37.796 [7496.8364] <4> serverResponse: 14:35:36 Initiating backup
14:35:37.796 [7496.8364] <4> readCommMessages: Entering readCommMessages
14:35:41.796 [7496.8364] <4> serverResponse: 14:35:41 INF - Starting bpbrm
14:35:41.796 [7496.8364] <4> readCommMessages: Entering readCommMessages
14:35:45.796 [7496.8364] <4> serverResponse: 14:35:45 INF - Data buffer size = 65536
14:35:45.796 [7496.8364] <4> serverResponse: INF - Server buffer size = 65536
14:35:45.796 [7496.8364] <4> readCommMessages: Entering readCommMessages
14:35:48.796 [7496.8364] <4> serverResponse: 14:35:47 INF - Server status = 83
14:35:48.796 [7496.8364] <16> serverResponse: ERR - server exited with status 83: media open error

14:35:48.796 [7496.8364] <16> CreateNewImage: ERR - serverResponse() failed
14:35:48.796 [7496.8364] <4> closeApi: entering closeApi.
14:35:48.796 [7496.8364] <4> closeApi: INF - EXIT STATUS 6: the backup failed to back up the requested files

14:35:48.796 [7496.8364] <16> VxBSACreateObject: ERR - Could not create new image with file /solman-nw7.MSSQL7.SOLMAN-NW7.db.master.~.7.001of001.20121107143531..C.
14:35:48.796 [7496.8364] <4> getOwnerName: entering getOwnerName.
14:35:48.796 [7496.8364] <4> dbc_GetServerName: entering dbc_GetServerName.

 

 

Comments 17 CommentsJump to latest comment

Will Restore's picture

status 83: media open error
 

Explanation: The tape manager (bptm) or disk manager (bpdm) did not open the

device or file that the backup or restore must use.

Recommended Action: For additional information, check the following:

■ NetBackup Problems report

■ Event Viewer Application log (Windows)

■ On Windows, make sure that the tapes are not write protected.

■ For detailed troubleshooting information:

■ Create a debug log directory for bpdm (if the device is disk) or bptm (if the

device is tape).

 

 

Will Restore -- where there is a Will there is a way

NBU_13's picture

Hi,

Today, I done some changes in FT media server as below,

NUMBER_DATA_BUFFERS_FT : 32

SIZE_DATA_BUFFERS_FT - 262144

and then I restarted sql database backup from SAN client,

Once database backup job has started, its running 4 child jobs + 1 parent job, 3 child jobs are successful and one child job has failed with erros as below

3-Dec-12 17:26 - Error bptm(pid=19800) Could not open FT Server pipe: pipe open failed (17)  
03-Dec-12 17:26 - Info bptm(pid=19800) EXITING with status 83 <----------       
03-Dec-12 17:26 - Info bpbrm(pid=19795) got ERROR 83 from media manager      
media open error(83)

and parent job has completed with status code 1

as per job details,

SOL, msdb, model database are successful, only master database are failed with error code 83, 17

Note : Linux OS (FT media server) and windows 2008 (SAN Client)

Any can help me on this issue, to resolve?

Thanks

 

 

Will Restore's picture

see Article URL http://www.symantec.com/docs/TECH62007

 

If that doesn't help, I would open a support case with Symantec.

Will Restore -- where there is a Will there is a way

NBU_13's picture

Hi

file system backup is running fine, no issue

only issue with sql database backup... have issue with master db, other db sol, msdb, model are completed successfully.

Now, I restarted test bckup for sql database with below changes.

 

NUMBER_DATA_BUFFERS_FT : 32

SIZE_DATA_BUFFERS_FT - 262144

Maximum concurrent FT connections - 8

 

Lets see..

 

 

 

 

NBU_13's picture

Hi,

sql database has failed again with same error.

Could not open FT Server pipe: pipe open failed (17)  
Info bpbrm(pid=19795) got ERROR 83 from media manager      
media open error(83)

 

Will Restore's picture

Per the technote I linked above

NUMBER_DATA_BUFFERS must be an integer with a value from 4 to 32. NUMBER_DATA_BUFFERS_FT can be used for overriding NUMBER_DATA_BUFFERS for SAN Client jobs. For a Fibre Transport media server, the recommendation is not to exceed 16.

 

If that doesn't help, I would open a support case with Symantec.

 

 

Will Restore -- where there is a Will there is a way

NBU_13's picture

Hi,

Previously, I tried with 16 only, it failed with same error 83, so I tried with 32 and then I restarted again, its failed with same error 83, 17

NBU_13's picture

Hi,

Previously, I tried with 16 only, it failed with same error 83, so I tried with 32 and then I restarted again, its failed with same error 83, 17

note : I have only 1 drive with this FT media server and sql database is running with option 6 as multiplexing,

NBU_13's picture

Hi,

is this neccessory to create both files NUMBER_DATA_BUFFERS and NUMBER_DATA_BUFFERS_FT under config directory ?

 because I created only NUMBER_DATA_BUFFERS_FT with 32, need to create NUMBER_DATA_BUFFERS also ?

please guide me.

Nagalla's picture

NUMBER_DATA_BUFFERS_FT is specifially for FT and NUMBER_DATA_BUFFERS is Global one, as you have created the specific one, does not need to think about the global one, as for as FT media server consern.

go ahead and open case with Symantec and let us know if you find the solution.. 

good luck.. 

Marianne's picture

NBU_13, you have been battling with this for almost a month now.
Time to open a Support call, don't you think?

I notice that you have never shared details of your environment, e.g. NBU version on media server and client, type of STU, no media server logs (bptm, bpdm, bpbrm).

Ensure all of these logs exist before you log a support call (they will need level 5 logs).

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

NBU_13's picture

Hi,

The below is over environment, and also I attached log files with verbose 5

Master server : windows 2008, NBU version 7.1

FT Media server : Redhat Linux, NBU version 7.1

SAN client : windows 2008, NBU version 7.1 

 

 

Job details :

03-Dec-12 16:14 - Info nbjm(pid=5944) starting backup job (jobid=3092048) for client solman-nw7, policy GlobalCrossing-solmannw7-Online-DB, schedule Default-Application-Backup 
03-Dec-12 16:14 - Info nbjm(pid=5944) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=3092048, request id:{680C9002-48A1-42CE-9499-07A799E4B332}) 
03-Dec-12 16:14 - requesting resource media9stgo-hcart3-robot-tld-9
03-Dec-12 16:14 - requesting resource masterstgo.NBU_CLIENT.MAXJOBS.solman-nw7
03-Dec-12 16:14 - requesting resource masterstgo.NBU_POLICY.MAXJOBS.GlobalCrossing-solmannw7-Online-DB
03-Dec-12 16:14 - granted resource masterstgo.NBU_CLIENT.MAXJOBS.solman-nw7
03-Dec-12 16:14 - granted resource masterstgo.NBU_POLICY.MAXJOBS.GlobalCrossing-solmannw7-Online-DB
03-Dec-12 16:14 - granted resource JPL641
03-Dec-12 16:14 - granted resource HP.ULTRIUM5-SCSI.002
03-Dec-12 16:14 - granted resource media9stgo-hcart3-robot-tld-9
03-Dec-12 16:14 - granted resource TRANSPORT
03-Dec-12 16:14 - estimated 0 Kbytes needed
03-Dec-12 16:14 - Info nbjm(pid=5944) started backup job for client solman-nw7, policy GlobalCrossing-solmannw7-Online-DB, schedule Default-Application-Backup on storage unit media9stgo-hcart3-robot-tld-9
03-Dec-12 16:14 - started process bpbrm (19795)
03-Dec-12 16:14 - end writing
03-Dec-12 17:26 - Info bpbrm(pid=19795) starting bptm          
03-Dec-12 17:26 - Info bpbrm(pid=19795) Started media manager using bpcd successfully      
03-Dec-12 17:26 - Info bpbrm(pid=19795) solman-nw7 is the host to backup data from    
03-Dec-12 17:26 - Info bpbrm(pid=19795) telling media manager to start backup on client    
03-Dec-12 17:26 - Info bptm(pid=19800) using 65536 data buffer size       
03-Dec-12 17:26 - Info bptm(pid=19800) using 16 data buffers        
03-Dec-12 17:26 - Error bptm(pid=19800) Could not open FT Server pipe: pipe open failed (17)  
03-Dec-12 17:26 - Info bptm(pid=19800) EXITING with status 83 <----------       
03-Dec-12 17:26 - Info bpbrm(pid=19795) got ERROR 83 from media manager      
media open error(83)

 

Thanks.

AttachmentSize
logs.rar 675.42 KB
Syed saied's picture

HI,

Is there any load balancer in the SQL server, if yes then onece remove the network load balancer and check.

 

Thanks In Advance...

Syed Saied

If the suggestion has helped to solve your problem, please mark the post as a solution

Marianne's picture

Seems you buffer sizes are not getting applied?

03-Dec-12 17:26 - Info bptm(pid=19800) using 65536 data buffer size       
03-Dec-12 17:26 - Info bptm(pid=19800) using 16 data buffers        

You really need to log a Support call. 

I don't know if anybody here has got the time to sift through level 5 logs... Symantec Support engineers have the time and the tools... Sorry...

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

NBU_13's picture

Hi,

Today, I created batch file for each sql database (master, msdb, model and sol), its running fine and completed successfully via SAN,

So, Finally, I create two batch file, one batch file for sol, and another batch file for (msdb, model and master) with 2 seperate policies, now all four sql database are running fine and getting completed successfully.

 

is this fine ? or need to do any changes ?