Video Screencast Help

SQLVDI Errors

Created: 13 May 2013 | 5 comments

SQL 2008 R2, Windows 2008.

Just noticed some SQLVDI errors cropping up in the event viewer.  One from yesterday is within a couple minutes of 2 database backup failures.  Both backups were at 1.8TB+

 

SQLVDI: Loc=TriggerAbort. Desc=invoked. ErrorCode=(0). Process=4596. Thread=9820. Server. Instance=SODA. VD=Global\VNBU0-15744-10288-1368203414_SQLVDIMemoryName_0

Error details from activity monitor attached:

Operating Systems:

Comments 5 CommentsJump to latest comment

Mark_Solutions's picture

Looks like timeouts are causing your errors - but take a very long time to do these backups.

How large are these databases?

I think you need to look at what is holding them up as they are taking hours to process / backup - maybe antivirus slowing things down - or a lack of system resources (usually memory on SQL Servers)

You could increase the timeouts but these look to be very slow so see if you can pin that down first.

 

Hope this helps

5/12/2013 2:51:13 PM - begin writing
5/12/2013 7:38:50 PM - Error bpbrm(pid=11344) could not write FILE ADDED message to stderr 

 

5/11/2013 4:03:02 PM - Info dbclient(pid=19308) INF - ODBC return code <2>, SQL State <01000>, SQL Message <3211><[Microsoft][ODBC SQL Server Driver][SQL Server]10 percent processed.>.
5/11/2013 9:38:44 PM - Info dbclient(pid=19308) INF - ODBC return code <2>, SQL State <01000>, SQL Message <3211><[Microsoft][ODBC SQL Server Driver][SQL Server]20 percent processed.>.
5/12/2013 1:02:39 AM - Info dbclient(pid=19308) INF - ODBC return code <2>, SQL State <01000>, SQL Message <3211><[Microsoft][ODBC SQL Server Driver][SQL Server]30 percent processed.>.
5/12/2013 4:41:45 AM - Info dbclient(pid=19308) INF - ODBC return code <2>, SQL State <01000>, SQL Message <3211><[Microsoft][ODBC SQL Server Driver][SQL Server]40 percent processed.>.
5/12/2013 7:24:13 AM - Info dbclient(pid=19308) INF - ODBC return code <2>, SQL State <01000>, SQL Message <3211><[Microsoft][ODBC SQL Server Driver][SQL Server]50 percent processed.>.
5/12/2013 9:45:39 AM - Info dbclient(pid=19308) INF - ODBC return code <2>, SQL State <01000>, SQL Message <3211><[Microsoft][ODBC SQL Server Driver][SQL Server]60 percent processed.>.
5/12/2013 11:38:51 AM - Info dbclient(pid=19308) INF - ODBC return code <2>, SQL State <01000>, SQL Message <3211><[Microsoft][ODBC SQL Server Driver][SQL Server]70 percent processed.>.
5/12/2013 1:32:13 PM - Info dbclient(pid=19308) INF - ODBC return code <2>, SQL State <01000>, SQL Message <3211><[Microsoft][ODBC SQL Server Driver][SQL Server]80 percent processed.>.
5/12/2013 3:45:46 PM - Info dbclient(pid=19308) INF - ODBC return code <2>, SQL State <01000>, SQL Message <3211><[Microsoft][ODBC SQL Server Driver][SQL Server]90 percent processed.>.
5/12/2013 7:12:02 PM - Info dbclient(pid=19308) INF - ODBC return code <2>, SQL State <01000>, SQL Message <4035><[Microsoft][ODBC SQL Server Driver][SQL Server]Processed 81000048 pages for database 'SODA_PROD_DDR2', file 'CSU_P43Production_I2' on file 1.>.
5/12/2013 7:12:03 PM - Info dbclient(pid=19308) INF - SQL Message <4035><[Microsoft][ODBC SQL Server Driver][SQL Server]Processed 133106416 pages for database 'SODA_PROD_DDR2', file 'CSU_P43Production_I2_REPORTS' on file 1.>
5/12/2013 7:12:04 PM - Info dbclient(pid=19308) INF - SQL Message <4035><[Microsoft][ODBC SQL Server Driver][SQL Server]Processed 5922680 pages for database 'SODA_PROD_DDR2', file 'P41_Phase3_IndexObjs' on file 1.>
5/12/2013 7:12:04 PM - Info dbclient(pid=19308) INF - SQL Message <4035><[Microsoft][ODBC SQL Server Driver][SQL Server]Processed 8613328 pages for database 'SODA_PROD_DDR2', file 'P41_Phase3_UserObjs' on file 1.>
5/12/2013 7:12:05 PM - Info dbclient(pid=19308) INF - SQL Message <3211><[Microsoft][ODBC SQL Server Driver][SQL Server]100 percent processed.> 
 

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

HankHill's picture

DB Server has SEP installed. 

This particular database is 1.8TB.  I have another that is 2.5TB.  Development and DBAs won't do anything about the size.

I have also discovered there is a permissions problem, possibly, on the C:\Program Files\Veritas\NetBackup\logs\user_ops\MSSQL.  Found some discussion of that looking up a VxBSAInit and VxBSAEndData: 6.  I get a wonderful message about permissions being incorrectly ordered when I look at the security settings for the folder.

 

Mark_Solutions's picture

OK - pretty big then!

Are you using a specific account to run the NetBackup Client Service and NetBackup Legacy Network Service? These will need full rights to that directory

Make sure you have all NetBackup processes and folders excluded from SEP (bpfis, bpbkar32, dbbackex etc.)

You may need to keep your timeout hight, or even tune your keep alive settings on the client and master / media servers as well.

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

HankHill's picture

All services run as Local System. 

TCPIP Keepalive time on the master and media is set to 1800000.  This was set when we started getting timeouts at exactly 2 hours.  Was not set on client.

EDIT:  SEP Exlcusions for program files\veritas\* and program files (x86)\veritas\* are configured.  Also Symantec and SQL Server folders.  Is there anything else that should be excluded?  Programdata? 

Mark_Solutions's picture

I am surprised they all run as local system to be honest - normall needs the client service and legacy network service to run as a sysadmin account to all databases on 2008.

Your exclusions sound OK

Your keep alive timeout is 30 minutes - you could perhaps reduce that - I tend to use 510000 (8.5 minutes) but also add the KeepAliveInterval registry key with a decimal value of 3.

The TcpMaxConnectResponseRetransmissions key has also been know to help here - its default value is 2 so you can add or edit the key and increase that value.

If you still get the errors then maybe disable SEP and run the backup - if it works that time then at least it gives a good starting point

Hope this helps

 

 

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.