Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Backup Exec 2012 - Snapshots failing on VM's

Created: 25 Sep 2012 • Updated: 21 Oct 2012 | 11 comments
This issue has been solved. See solution.

 

Hi All,
 
For some reason I wasn't able to create a support issue, said it was in maintenance window 3 weeks ago :)
 
So here goes I guess
 
Environment is 3 x ESX Servers fibre to SAN.
Multiple VM's hosted on each server.
Symantec Backup Exec 2012 with latest patches
 
My backup strategy is VM snapshot backups of each machine via the SAN connection, Then Granular backups of each machine also.
 
All granular backups work fine.
 
Half of the snapshot backups fail with the same error each time, which is as follows.
The error displayed in the VMWare vSphere Client is:
An error occurred while quiescing the virtual machine. See the virtual machines's event log for details.
NB: I am able to take snapshots successfully of these machines via the VMWare vSphere Client.
 
The error displayed in the Symantec Backup Exec 2012 is:
Job ended: Wednesday, 26 September 2012 at 8:51:50 PM
Completed status: Failed
Final error: 0xe000959a - An attempt to take a snapshot of a virtual machine failed because it was unable to quiesce an application.
Final error category: Resource Errors
For additional information regarding this error refer to link V-79-57344-38298
 
Another odd thing is each time this happens the servers time is put out which I have to sync back to the domheir each time. As you can imagine this is problematic. Unsure as to where that time is coming from, it isn't the time on the hosts but may match up to the timezone displayed in iLo, I have changed that and should be able to determine when I build up the patience to have another go.
 
Any steps, ideas, thoughts or theories much appreciated.
 
Cheers

Comments 11 CommentsJump to latest comment

Jaydeep S's picture

From what you have posted this is what I understood, you have 2 jobs setup - 1st AVVI snapsot backup of VMs without GRT and 2nd a traditional RAWS backup of the VMs with Agent based GRT. Please correct me if I am wrong.

If this is the case, could you please check if at any time are both the jobs runnig simultaneously. Do you have the BE VSS Provider and VMWare VSS Provider installed on the VMs If they are installed, please uninstall them using the links below -

http://www.symantec.com/docs/TECH129864

http://www.symantec.com/docs/TECH130940

Note - Reboot might be required following the uninstall.

Once this is done, go to the vspehere client and attempt to take a manual snapshot. Remove the check from "Snapshot the Virtual machine's memory" and check option "Quiesce guest file system." 

vmds's picture

What ESX Version do you use and what machines to you backup?

What VSS Providers are you using? If you want to use Backup Exec Agent VSS Provider (for GRT), you can't have the Vmware VSS Provider installed as this will conflict.

 

masarost's picture

For Granular Tech to work, all virtual servers have to have an agent on them.

How is the backup server connecting to the vmware hosts?  I have had issues using 2012 (thats understated) make sure you have the isci initiator set up on a windows based system, plus having the vendors integration kit helps as well. 

syb_mpyz's picture

 

Thanks for responses.
I will address all questions and update with what I have done and the results.
 
I have a group which contains 18 jobs (1 RAWS job for each of the 17 servers (1 physical, the rest vm's)). The other job is a VMWare Server backup which contains individually selected snapshot vm backups for each vm. So basically, there are multiple jobs duplicated which one vm selected for each.
I did this so if one failed (which is happending a lot) the other would complete successfully.
They are all for the same time and this works fine, they just queue.
 
The RAWS jobs all succeed no problems.
The vm backups are different although the same servers are failing for the same reason except the SQL vm which has and event in its event viewer relating to sqlvdi. I thought I would leave that to last.
All vm's can be successfully snapshotted via vSphere Client.
 
All VM's are Windows Server 2008 R2.
ESX 5.0 U1
 
The backup server is connected to SAN via 8GB fibre
 
I tend to think it is related to the vss config as some vm's work.
Today, I will uninstall the VMWare vss component and post results.
syb_mpyz's picture

Uninstalled VMWare VSS Provider and attempted again.

It failed with same result. 

It did change the time on the server but because I had changed timezone on ESX to correct it was changed to correct time.

 

syb_mpyz's picture

In addition to above just did the following..

 

http://www.symantec.com/docs/TECH130940

Note - Reboot might be required following the uninstall.

Once this is done, go to the vspehere client and attempt to take a manual snapshot. Remove the check from "Snapshot the Virtual machine's memory" and check option "Quiesce guest file system." 

The snapshot was successful via vSphere Client.

Still failing via Backup Exec.

Jaydeep S's picture

syb_mpyz could you try to perform a AVVI GRT or Non GRT backup of the Virtual machines that seem to be failing. Make sure that there are no other backups running for those VMs at the same time (even RAWS backup) and see if that made any difference.

syb_mpyz's picture

Jaydeep S

This is what I am doing as far as I know?

syb_mpyz's picture

Jaydeep S

What I meant by that is I am currently doing AVVI non GRT backups of the Virtual Machines. I have tested each option with no other backups running.

I am going to throw this in the mix as I moved some Virtual Machines around to test a theory with a pattern I was noticing.

Basically any virtual machines stored on the ESX Servers 2 & 3 are failing whereas any on ESX Server 1 are successful.

Any ideas would be much appreciated.

syb_mpyz's picture

Im 99% sure it is related to the server it is hosted on. At a loss currently as to why tho.

syb_mpyz's picture

I have now resolved this issue.

As suspected the problem was due to the ESX servers that the failing virtual servers were hosted on.

Basically ESX2 and ESX3 had incorrect time settings, once the ESX Servers were configured with correct time zones and valid NTP Settings backups were successful.

SOLUTION