Troubleshooting Hyper-V virtual machine backups that fails with Snapshot provider error "0xE000FED1 - A failure occurred querying the Writer status".

Article:TECH154720  |  Created: 2011-03-03  |  Updated: 2014-05-07  |  Article URL http://www.symantec.com/docs/TECH154720
Article Type
Technical Solution


Issue



Troubleshooting Hyper-V virtual machine backups that fails with Snapshot provider error "0xE000FED1 - A failure occurred querying the Writer status".

 


Error



Final error: 0xe000fed1 - A failure occurred querying the Writer status.
Final error category: Resource Errors
For additional information regarding this error refer to link V-79-57344-65233

Backup- HOST_SERVER
V-79-57344-65233 - AOFO: Initialization failure on: "VRTSRV::\\HOST_SERVER\Hyper-V?Virtual?Machine\GUEST". Advanced Open File Option used: Microsoft Volume Shadow Copy Service (VSS).

Snapshot provider error (0xE000FED1): A failure occurred querying the Writer status.

Check the Windows Event Viewer for details.

Writer Name: Microsoft Hyper-V, Writer ID: {66841CD4-6DED-4F4B-8F17-FD23F8DDC3DE}, Last error: The VSS Writer failed, but the operation can be retried (0x800423f3), State: Waiting for backup complete notification (5).


Environment



WIndows 2008 R2 and Windows 2008 SP2 Hyper-V host.


Solution



Scenario 1:

Snapshot provider error (0xE000FED1): A failure occurred querying the Writer status. State: Waiting for backup complete notification (5) when backing backing up Virtual machine using Hyper-V Agent.

When Backing up Virtual machines using the Backup Exec Hyper-V agent (AMHV) the backup may fail with A failure occurred querying the Writer status in Waiting for backup complete notification (5).

The following may be present on the Hyper-V host in the Microsoft-Windows-Hyper-V-VMMS-Admin Event Log.

Microsoft-Windows-Hyper-V-VMMS-Admin Event Log:

 Log Name:   Microsoft-Windows-Hyper-V-VMMS-Admin
Source:    Microsoft-Windows-Hyper-V-VMMS
Event ID:   10102
Task Category: None
Level:     Error
Keywords:  
User:     SYSTEM
Computer:   HOST_SERVER
Description:
Failed to create the backup of virtual machine 'GUEST'. (Virtual machine ID 2759275E-705B-4D19-9DA9-3282B574D8FD)

 Log Name:   Microsoft-Windows-Hyper-V-VMMS-Admin
Source:    Microsoft-Windows-Hyper-V-VMMS
Event ID:   10102
Task Category: None
Level:     Error
Keywords:  
User:     SYSTEM
Computer:   HOST_SERVER
Description:
Failed to create the backup of virtual machine 'GUEST'. (Virtual machine ID 2759275E-705B-4D19-9DA9-3282B574D8FD)

 

Cause: Typically the VSS Snapshot Notification (5) error can be caused by various issues.

Solution: 

  1. Review the Application and System Logs on BOTH the Hyper-V host and the Virtual machine for any VSS or VOLSNAP errors or low disk conditions. Review the Backup Exec (SCL) Software Compatibility List to ensure the Virtual Machine disk configuration and Operating System are supported.
  2. If no other Application or System Logs errors on both the Virtual Machine Guest or the Hyper-V host are present related to VSS or Volsnap errors review the Microsoft-Windows-Hyper-V-VMMS event log for the above error. 
  3. If this error is present logon the the Virtual Machine and check the Total space and free space of the disk\volumes inside the virtual machine guest
  4. In order for the Hyper-V writer to quiesce the VSS writers inside the Virtual machine guest and perform a snapshot there must be adequate space inside the virtual machine guest volumes. Generally but not limited if there is less than 10% free disk space inside the guest the job may fail.
  5. Please make sure each Virtual Machine is allocated a shadowstorage association for each Volume present inside the guest.  See Technotes http://www.symantec.com/docs/TECH69691  and http://www.symantec.com/docs/TECH75504 
  6. Increase Disk space inside the Virtual Machine.

 

Scenario 2:

Hyper-V backup of Windows SBS 2011 virtual machine fails with error, "0xe000fed1 - A failure occurred querying the Writer status."

Cause: 

Application Event on the Guest :
Log Name:      Application
Source:        VSS
Date:          DATE  &  TIME
Event ID:      8230
Task Category: None
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      computername
Description:
Volume Shadow Copy Service error: Failed resolving account spsearch with status 1376. Check connection to domain controller and VssAccessControl registry key.

Operation:
   Gathering Writer Data
   Executing Asynchronous Operation

Context:
   Execution Context: Requestor
   Current State: GatherWriterMetadata

Error-specific details:
   Error: NetLocalGroupGetMemebers(spsearch), 0x80070560, The specified local group does not exist.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">

 

As per Microsoft [http://support.microsoft.com/kb/2483007], This issue occurs because the account that is configured as the Windows SharePoint Services service account does not have the Local Activation permission to the IIS Wamreg Admin Service. The event ID error message 10016 does not affect the functionality of Windows SharePoint Services.

Solution:

Warning: Incorrect use of the Windows registry editor may prevent the operating system from functioning properly. Great care should be taken when making changes to a Windows registry. Registry modifications should only be carried out by persons experienced in the use of the registry editor application. It is recommended that a complete backup of the registry and workstation be made prior to making any registry changes.

  1. Open the Registry.
  2. Navigate to the following registry key: HKEY_CLASSES_ROOT\AppID\{61738644-F196-11D0-9953-00C04FD919C1}.
  3. Right click on the key {61738644-F196-11D0-9953-00C04FD919C1} and choose permissions.
  4. Click Advanced button.
  5. Change to the Owner Tab.
  6. Change the Owner to the Administrators Group be sure to check the box labeled “Replace owner on subcontainers and objects”.
  7. Once the Owner is changed, go back to the security tab and Grant the Administrators group Full Control.
  8. Close Regedit.
  9. Reboot the server, and try the job again.

Refer to http://support.microsoft.com/kb/2483007

 

Scenario 3:

Backup of Virtual machine hosted on Hyper-V fail with Snapshot provider error 0xe000fed1 - A failure occurred querying the Writer status. The operation cannot be retried (0x800423f4).

On the Hyper-V host server the following warning is observed in the Hyper-V-VMMS event log (Event Viewer - Application and Service Logs - Microsoft -Hyper-V-VMMS)

Hyper-V Virtual Machine Management Service Event Log Error:

Log Name:      Microsoft-Windows-Hyper-V-VMMS-Admin
Source:        Microsoft-Windows-Hyper-V-VMMS-Admin
Event ID:      10137
Level:         Error
Keywords:      Classic
Description: No snapshots to revert were found for virtual machine 'GUEST'. (Virtual machine ID GUID)

The following Warning is display in the System Log on the Hyper-V host moments before the backup job failure:

System Log Warning:

Log Name:      System
Source:        partmgr
Event ID:      58
Level:         Warning
Keywords:      Classic
Description:
The signature of disk <number> because it is equal to the disk signature of disk <number>

Cause:  

This failure occurs due to the Hyper-V host disks and the Virtual machines disks having the same disk signature.  When a Hyper-V agent backup takes place VSS mounts the guest VHD disk files and snapshots for backup.  If the Host machine disk signature is the same as the disk signature inside the Virtual machine a disk signature collision occurs and the Hyper-V writer enters a re-tryable state and fails the job. 

Disk Collision conflict can occur after using 3rd Party utilities to convert a Physical Server to a Virtual Server.

Solution:  

Since this issue occurs due to Disk Signature conflict between the Hyper-V host disk and Virtual machine disk, this conflict needs to be resolved by creating a different disk signature for the Virtual machine disk (VHD). For detailed steps on how to do the same, refer the Microsoft kb article http://support.microsoft.com/kb/975695

Note: Before attempting to fix the disk signature conflict, it is recommended to backup the Virtual Machine using the Backup Exec remote agent (RAWS) by installing the remote agent inside the guest or copy the Virtual machines (VHD files) to another directory or volume.

 REFERENCE:

When you try to back up a virtual machine on a Windows Server 2008-based computer that has multiple virtual hard disks attached, the backup fails, and Hyper-V Writer error "0x800423f4" is generated.

http://support.microsoft.com/kb/975695

  

Scenario 4:

A backup of a Hyper-V guest fails with error: 0xe000fed1 - A failure occurred querying the writer status. Writer Name: Microsoft Hyper-V, Writer ID: {66841CD4-6DED-4F4B-8F17-FD23F8DDC3DE}, Last error: The VSS Writer failed, but the operation can be retried (0x800423f3), State: Failed during post-snapshot operation (11).

 

The following two error events may appear in the System event log on the Hyper-V host.
Event ID 27 from source Volsnap:
The shadow copies of volume \\?\Volume{8dc2a918-2145-11df-a8ff-001f295ae761} were aborted during detection because a critical control file could not be opened.
Event ID 55: The file system structure on the disk is corrupt and unusable. Please run the chkdsk utility on the volume \\?\Volume{<GUID>}
Solution:
A drive is mapped to a volume GUID.  This can be checked using mountvol.exe.  See the following blog post:
If the .vhd file for an affected virtual machine resides on a drive that is mapped to a volume GUID, perform the following steps:
  • Take affected virtual machine(s) offline.
  • Run Chkdsk /r against the affected drive.
  • Bring the affected virtual machine back online.
  • Perform another backup.

 


Supplemental Materials

SourceUMI
ValueV-79-57344-65233
Description

0xe000fed1 - A failure occurred querying the Writer status.


SourceEvent ID
Value10102
Description

Failed to create the backup of virtual machine 'GUEST'. (Virtual machine ID 2759275E-705B-4D19-9DA9-3282B574D8FD).




Article URL http://www.symantec.com/docs/TECH154720


Terms of use for this information are found in Legal Notices