Video Screencast Help
Symantec Appoints Michael A. Brown CEO. Learn more.

Operation Status Error (-1) for SQL Backups

Created: 20 Dec 2012 | 17 comments

We have a Windows 2008 client for which we have configured SQL backups. The backups appear to be completing successfully (status 0 in the NBU admin console) and the SQL DBA is showing successful backups at the database level. The issue that the DBA is seeing is on the client side when he views the status of the SQL backups in the NBU client, they mostly show the Operation Type as Unknown, and the Operation Status as Error (-1) (see attached). Is this an aberration, or is there an actual issue with the SQL backups for this client? What logs would I need to collect for troubleshooting?

Master: NBU 7.5.0.4 running on SunOS 5.10

Media: NBU 7.5.0.4 running on Windows 2008 Server

Client: Windows 2008 Server running SQL 2008 Server and NBU client 7.1

 

Thanks,

Wayne

Comments 17 CommentsJump to latest comment

RamNagalla's picture

hi ,

you need to check the dbclient logs form the client located in installpath\veritas\netbackup\logs\dbclient.

could you post the dbclient logs from the client and the batch file that is being used for backups.

WayneLackey's picture

Still waiting for the DBA to send me this information, will post up as it becomes available.

WayneLackey's picture

Please see attached zip file for dbclient logs and .bch files. Please let me know if you need any other information.

Thanks,

Wayne

AttachmentSize
NBU_files.zip 36.47 KB
RamNagalla's picture

hi,

I am looking into the log file 010113.log for the the job excuted for the script UserFullDB.bch, and I am not seeing any errors for this , but your initial screenshot was showing Error -1 for this script only but for the different date.

what is the status of the job for the script UserFullDB.bch on 01/01/13.?

 

and I am seeing some communication errors for 

21:03:44.660 [64576.67104] <16> CODBCaccess::LogODBCerr: DBMS MSG - ODBC return code <-1>, SQL State <37000>, SQL Message <3202><[Microsoft][ODBC SQL Server Driver][SQL Server]Write on "VNBU0-64576-67104-1357092157" failed: 995(The I/O operation has been aborted because of either a thread exit or an application request.)>.

21:03:44.809 [64576.67104] <16> CODBCaccess::LogODBCerr: DBMS MSG - SQL Message <3271><[Microsoft][ODBC SQL Server Driver][SQL Server]A nonrecoverable I/O error occurred on file "VNBU0-64576-67104-1357092157:" 995(The I/O operation has been aborted because of either a thread exit or an application request.).>

i understand that its some I/O error, but i am not sure what it is trying to say.. 

Will Restore's picture

Seem the real trouble is intermittent Status 25. See this older thread for advice:
https://www-secure.symantec.com/connect/forums/sql...

Increase Timeout values on the Media Server
Ensure no other database activities are occuring during the backup window

Will Restore -- where there is a Will there is a way

WayneLackey's picture

All,

Thanks for your input. I will take a look at the media server timeout settings and adjust as necessary.

So, does this indicate an actual problem? The NBU admin console is showing these backups are successful, as is the SQL side according to the DBA. The NBU client is the only thing showing anything different. I guess my main concern is - which one is right, and should I be concerned that these DBs aren't backing up when everything else is telling me that they are?

Thanks,

Wayne

WayneLackey's picture

@Nagalla,

The DBA reported that the NBU client is showing -1 for all jobs on 1/1/13.

All SQL jobs for this client are showing status 0 in the NBU admin console for 1/1/13, with one exception. There is one job with a status 6 - job detail follows:

01/01/2013 21:00:00 - Info dbclient (pid=64576) INF - BACKUP STARTED USING
01/01/2013 21:00:00 - Info dbclient (pid=64576) Microsoft SQL Server 2008 R2 (SP2) - 10.50.4263.0 (X64)
01/01/2013 21:00:00 - Info dbclient (pid=64576) Aug 23 2012 15:56:56
01/01/2013 21:00:00 - Info dbclient (pid=64576) Copyright (c) Microsoft Corporation
01/01/2013 21:00:00 - Info dbclient (pid=64576) Enterprise Edition (64-bit) on Windows NT 6.1 <X64> (Build 7601: Service Pack 1) (Hypervisor)
01/01/2013 21:00:00 - Info dbclient (pid=64576) Batch = C:\Program Files\Veritas\NetBackup\DbExt\MsSql\VCIS001_SystemFullDB.bch, Op# = 1
01/01/2013 21:00:00 - Info dbclient (pid=64576) INF - Using backup image USOLGWVCIS001.MSSQL7.USOLGWVCIS001.db.master.~.7.001of001.20130101210004..C
01/01/2013 21:00:00 - Info dbclient (pid=64576) INF - backup database "master" to VIRTUAL_DEVICE='VNBU0-64576-65712-1357092006' with  compression,  stats = 10, checksum, stop_on_error, blocksize = 65536, maxtransfersize = 4194304, buffercount = 2
01/01/2013 21:00:00 - Info dbclient (pid=64576) INF - Number of stripes: 1, Number of buffers per stripe 2.
01/01/2013 21:00:00 - Info dbclient (pid=64576) INF - Created VDI object for SQL Server instance <USOLGWVCIS001>. Connection timeout is <300> seconds.
01/01/2013 21:01:13 - Info dbclient (pid=64576) ERR - Error in VxBSACreateObject: 3.
01/01/2013 21:01:13 - Info dbclient (pid=64576)     CONTINUATION: - System detected error, operation aborted.
01/01/2013 21:01:13 - Info dbclient (pid=64576) ERR - Error in GetCommand: 0x80770004.
01/01/2013 21:01:13 - Info dbclient (pid=64576)     CONTINUATION: - An abort request is preventing anything except termination actions.
01/01/2013 21:01:16 - Info dbclient (pid=64576) ERR - Error in VDS->Close: 0x80770004.
01/01/2013 21:01:16 - Info dbclient (pid=64576)     CONTINUATION: - An abort request is preventing anything except termination actions.
01/01/2013 21:01:16 - Info dbclient (pid=64576) INF - OPERATION #1 of batch C:\Program Files\Veritas\NetBackup\DbExt\MsSql\VCIS001_SystemFullDB.bch FAILED with STATUS 1 (0 is normal). Elapsed time = 78(78) seconds.
01/01/2013 21:01:20 - Info dbclient (pid=64576) INF - BACKUP STARTED USING
01/01/2013 21:01:20 - Info dbclient (pid=64576) Microsoft SQL Server 2008 R2 (SP2) - 10.50.4263.0 (X64)
01/01/2013 21:01:20 - Info dbclient (pid=64576) Aug 23 2012 15:56:56
01/01/2013 21:01:20 - Info dbclient (pid=64576) Copyright (c) Microsoft Corporation
01/01/2013 21:01:20 - Info dbclient (pid=64576) Enterprise Edition (64-bit) on Windows NT 6.1 <X64> (Build 7601: Service Pack 1) (Hypervisor)
01/01/2013 21:01:20 - Info dbclient (pid=64576) Batch = C:\Program Files\Veritas\NetBackup\DbExt\MsSql\VCIS001_SystemFullDB.bch, Op# = 2
01/01/2013 21:01:20 - Info dbclient (pid=64576) INF - Using backup image USOLGWVCIS001.MSSQL7.USOLGWVCIS001.db.model.~.7.001of001.20130101210125..C
01/01/2013 21:01:20 - Info dbclient (pid=64576) INF - backup database "model" to VIRTUAL_DEVICE='VNBU0-64576-65180-1357092086' with  compression,  stats = 10, checksum, stop_on_error, blocksize = 65536, maxtransfersize = 4194304, buffercount = 2
01/01/2013 21:01:20 - Info dbclient (pid=64576) INF - Number of stripes: 1, Number of buffers per stripe 2.
01/01/2013 21:01:21 - Info dbclient (pid=64576) INF - Created VDI object for SQL Server instance <USOLGWVCIS001>. Connection timeout is <300> seconds.
01/01/2013 21:02:26 - Info dbclient (pid=64576) ERR - Error in VxBSACreateObject: 3.
01/01/2013 21:02:27 - Info dbclient (pid=64576)     CONTINUATION: - System detected error, operation aborted.
01/01/2013 21:02:27 - Info dbclient (pid=64576) ERR - Error in GetCommand: 0x80770004.
01/01/2013 21:02:27 - Info dbclient (pid=64576) DBMS MSG - ODBC return code <-1>, SQL State <37000>, SQL Message <3202><[Microsoft][ODBC SQL Server Driver][SQL Server]Write on "VNBU0-64576-65180-1357092086" failed: 995(The I/O operation has been aborted because of either a thread exit or an application request.)>.
01/01/2013 21:02:27 - Info dbclient (pid=64576)     CONTINUATION: - An abort request is preventing anything except termination actions.
01/01/2013 21:02:27 - Info dbclient (pid=64576) DBMS MSG - SQL Message <3271><[Microsoft][ODBC SQL Server Driver][SQL Server]A nonrecoverable I/O error occurred on file "VNBU0-64576-65180-1357092086:" 995(The I/O operation has been aborted because of either a thread exit or an application request.).>
01/01/2013 21:02:27 - Info dbclient (pid=64576) DBMS MSG - SQL Message <3013><[Microsoft][ODBC SQL Server Driver][SQL Server]BACKUP DATABASE is terminating abnormally.>
01/01/2013 21:02:27 - Info dbclient (pid=64576) ERR - Error found executing <backup database "model" to VIRTUAL_DEVICE='VNBU0-64576-65180-1357092086' with  compression,  stats = 10, checksum, stop_on_error, blocksize = 65536, maxtransfersize = 4194304, buffercount = 2>.
01/01/2013 21:02:27 - Info dbclient (pid=64576) ERR - Error in VDS->Close: 0x80770004.
01/01/2013 21:02:27 - Info dbclient (pid=64576)     CONTINUATION: - An abort request is preventing anything except termination actions.
01/01/2013 21:02:28 - Info dbclient (pid=64576) INF - OPERATION #2 of batch C:\Program Files\Veritas\NetBackup\DbExt\MsSql\VCIS001_SystemFullDB.bch FAILED with STATUS 1 (0 is normal). Elapsed time = 69(69) seconds.
01/01/2013 21:02:31 - Info dbclient (pid=64576) INF - BACKUP STARTED USING
01/01/2013 21:02:31 - Info dbclient (pid=64576) Microsoft SQL Server 2008 R2 (SP2) - 10.50.4263.0 (X64)
01/01/2013 21:02:31 - Info dbclient (pid=64576) Aug 23 2012 15:56:56
01/01/2013 21:02:31 - Info dbclient (pid=64576) Copyright (c) Microsoft Corporation
01/01/2013 21:02:31 - Info dbclient (pid=64576) Enterprise Edition (64-bit) on Windows NT 6.1 <X64> (Build 7601: Service Pack 1) (Hypervisor)
01/01/2013 21:02:31 - Info dbclient (pid=64576) Batch = C:\Program Files\Veritas\NetBackup\DbExt\MsSql\VCIS001_SystemFullDB.bch, Op# = 3
01/01/2013 21:02:31 - Info dbclient (pid=64576) INF - Using backup image USOLGWVCIS001.MSSQL7.USOLGWVCIS001.db.msdb.~.7.001of001.20130101210236..C
01/01/2013 21:02:32 - Info dbclient (pid=64576) INF - backup database "msdb" to VIRTUAL_DEVICE='VNBU0-64576-67104-1357092157' with  compression,  stats = 10, checksum, stop_on_error, blocksize = 65536, maxtransfersize = 4194304, buffercount = 2
01/01/2013 21:02:32 - Info dbclient (pid=64576) INF - Number of stripes: 1, Number of buffers per stripe 2.
01/01/2013 21:02:32 - Info dbclient (pid=64576) INF - Created VDI object for SQL Server instance <USOLGWVCIS001>. Connection timeout is <300> seconds.
01/01/2013 21:03:38 - Info dbclient (pid=64576) ERR - Error in VxBSACreateObject: 3.
01/01/2013 21:03:38 - Info dbclient (pid=64576)     CONTINUATION: - System detected error, operation aborted.
01/01/2013 21:03:38 - Info dbclient (pid=64576) ERR - Error in GetCommand: 0x80770004.
01/01/2013 21:03:38 - Info dbclient (pid=64576)     CONTINUATION: - An abort request is preventing anything except termination actions.
01/01/2013 21:03:38 - Info dbclient (pid=64576) DBMS MSG - ODBC return code <-1>, SQL State <37000>, SQL Message <3202><[Microsoft][ODBC SQL Server Driver][SQL Server]Write on "VNBU0-64576-67104-1357092157" failed: 995(The I/O operation has been aborted because of either a thread exit or an application request.)>.
01/01/2013 21:03:38 - Info dbclient (pid=64576) DBMS MSG - SQL Message <3271><[Microsoft][ODBC SQL Server Driver][SQL Server]A nonrecoverable I/O error occurred on file "VNBU0-64576-67104-1357092157:" 995(The I/O operation has been aborted because of either a thread exit or an application request.).>
01/01/2013 21:03:38 - Info dbclient (pid=64576) DBMS MSG - SQL Message <3013><[Microsoft][ODBC SQL Server Driver][SQL Server]BACKUP DATABASE is terminating abnormally.>
01/01/2013 21:03:39 - Info dbclient (pid=64576) ERR - Error found executing <backup database "msdb" to VIRTUAL_DEVICE='VNBU0-64576-67104-1357092157' with  compression,  stats = 10, checksum, stop_on_error, blocksize = 65536, maxtransfersize = 4194304, buffercount = 2>.
01/01/2013 21:03:39 - Info dbclient (pid=64576) ERR - Error in VDS->Close: 0x80770004.
01/01/2013 21:03:39 - Info dbclient (pid=64576)     CONTINUATION: - An abort request is preventing anything except termination actions.
01/01/2013 21:03:39 - Info dbclient (pid=64576) INF - OPERATION #3 of batch C:\Program Files\Veritas\NetBackup\DbExt\MsSql\VCIS001_SystemFullDB.bch FAILED with STATUS 1 (0 is normal). Elapsed time = 69(69) seconds.
01/01/2013 21:03:41 - Info dbclient (pid=64576) INF - Results of executing <C:\Program Files\Veritas\NetBackup\DbExt\MsSql\VCIS001_SystemFullDB.bch>:
01/01/2013 21:03:41 - Info dbclient (pid=64576) <0> operations succeeded. <3> operations failed.
01/01/2013 21:03:41 - Info dbclient (pid=64576) INF - The following object(s) were not backed up successfully.
01/01/2013 21:03:42 - Info dbclient (pid=64576) INF - master
01/01/2013 21:03:42 - Info dbclient (pid=64576) INF - model
01/01/2013 21:03:42 - Info dbclient (pid=64576) INF - msdb

What does that mean?

Thanks,

Wayne

WayneLackey's picture

@wr,

See attached screen shot for timeout values on the media server. Which ones should I be concerned with?

Thanks,

Wayne

Media server timeouts.jpg
Will Restore's picture

OK, per the older thread 'Check Client Connect and Client Read Timeouts on the media server'

Looks like they have already been bumped up.  Good. 

 

Will Restore -- where there is a Will there is a way

RamNagalla's picture

01/01/2013 21:03:39 - Info dbclient (pid=64576) ERR - Error in VDS->Close: 0x80770004.
01/01/2013 21:03:39 - Info dbclient (pid=64576)     CONTINUATION: - An abort request is preventing anything except termination actions.

 

I just find the related tech note for the same error.

http://www.symantec.com/business/support/index?pag...

<<copy from the T/N

11:27:45 [2704,2740] <32> CDBbackrec::FreeDeviceSet(): ERR - Error in VDS->Close: 0x80770004. The text follows:
11:27:45 [2704,2740]     CONTINUATION: - An abort request is preventing anything except termination actions.

Resolution: 
Application of SQL 7 Service Pack 3 resolved these issues.

 

hope this helps,.

 
Will Restore's picture

That's a really old technote.  Doubt that particular version applies here but the concept is good.

Make sure you have the latest service packs applied for current product. 

 

Will Restore -- where there is a Will there is a way

Marianne's picture

Please post full dbclient log as well as SQL Error log as File attachments.
Also check Event Viewer Application log for clues.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

Will Restore's picture

Agree with Marianne -- the latest log above from the Activity Monitor shows something went wrong and the logs on the Client should help pinpoint what.

Will Restore -- where there is a Will there is a way

WayneLackey's picture

Not sure if it makes a difference, but the DBA told me they are using the following version of SQL:

SQL 2008 R2 64bit SP2 CU2  (10.50.4263)

Wayne

Will Restore's picture

Wayne, did you get this resolved?  You never did attach the client logs per Marianne's request. 

Will Restore -- where there is a Will there is a way

WayneLackey's picture

My apologies for the slow response time, I have been out of the office for the past several days. I went ahead and opened a support case with Symantec for this, so you can disregard this posting. My thanks to everyone for your assistance and willingness to help.

Wayne

Marianne's picture

Please let us know the outcome of the Support call.

The rest of the community can also benefit from the solution...

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links