Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Exchange 2010 - Unable to complete the operation for the selected resource using the specified options for the following reason: VFF Open Failure. This can be caused by low memory or disk resources.

Created: 28 Apr 2014 | 21 comments

Hi All,

We are currently running Symantec Backup Exec 2010 R3 (SP3) on our media server (Windows Server 2008 R2) and attempting to backup an Exchange 2010 environment.

Our normal backup schedule includes backing up our Exchange 2010 Mailbox server on a daily basis however over the last couple of weeks we have been receiving the following issue:

Job ended: Monday, April 28, 2014 at 1:08:00 AM
Completed status: Failed
Final error: 0xe00002f7 - Cannot extract mailbox messages from the Exchange backup. Review the job log for more information.
Final error category: Resource Errors

For additional information regarding this error refer to link V-79-57344-759

Backup- \\EXCHANGETEST.com\Microsoft Information Store\Public Folders EXCHANGETEST

V-79-57344-759 - Unable to complete the operation for the selected resource using the specified options for the following reason: VFF Open Failure.  This can be caused by low memory or disk resources.

This incident does not occur every night but it has been happening sporadically over the last 6 weeks.

Has anyone seen this issue before?

Thanks in advance,

Kirk

Operating Systems:

Comments 21 CommentsJump to latest comment

RahulG's picture

refer http://www.symantec.com/business/support/index?page=content&id=TECH66427&actp=search&viewlocale=en_US&searchid=1398692384945

If youa re abcking upto tape refer 

http://www.symantec.com/business/support/index?pag...

 

 

KirkBoyd's picture

Thanks for the prompt replay RahulG but we have tried the suggestions in this document:

We have changed the registry key (below) to 600.  Issue still occurs.

Key : HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VirtFile
Value: VFF Extended Timeout
 
Articles regarding the Windows Server 2003 are irrelevant as we are running Windows Server 2008 r2 x64.
ArulPrasad's picture

is this backup to tape....?

If either it is to disk or Tape , if AV is Present ensure on access scan has a exclusion on the Disk location/ path of the b2d or Temp staging location (i.e c:\temp )

 

 

Arul

KirkBoyd's picture

Thanks Arul. However we do run AV on both the media and remote server and we exclude the temp staging location

This backup is running to tape.

Forgot to mention that when we untick the "Use Backup Exec Granular Recovery Technology (GRT ) option on the job it completes successfully.

ArulPrasad's picture

Yup that is true... The issue happens during the GRT processing... so if this is to tape the the GRT processing is done on the exchange server... , also try setting the registry to a different path than the one exist...

HKLM\SOFTWARE\Symantec\Backup Exec For Windows\Backup Exec\Engine\Exchange

String Value, "OnHostTemp".

set the value of OnHostTemp to "C:\temp"

 

ref:http://www.symantec.com/business/support/index?page=content&id=TECH164075

 

Arul

KirkBoyd's picture

Thanks Arul.

What is the default staging area on the exchange server?  Is it C:\TEMP?

We currently don't have these registry settings on our exchange servers however I can add that entry to see if it helps.

The problems is difficult to diagnose correctly as the error is intermittent.  The backups completed successfully last night for instance without any changes / modifications. However it could fail again on Thursday / Friday which is the infuriating thing.

VJware's picture

When the GRT Exchange backup runs to tape, the staging area is chosen on the Exchange server and typically the volume with the largest amount of free space is chosen for staging. Creating the OnHostTemp registry key on the Exchange server kinda forces the staging to the specified location only.

citybett1's picture

I am having the same problem as described above.  I get the same error when backing up an Exchange 2010 server with 5 databases (4 mailbox databases, 1 public folder database).   Regardless of whether I'm backing up to disk or to tape, randomly one of the mailbox databases will fail with that error.  The public folder database has never failed, but it's much smaller than the mailbox databases.  All of the mailbox databases are in the ballpark of 20GB each.

I've implemented the registry entry for the "VFF Extended Timeout" referenced in the following article on both the BackupExec server and on the Exchange server, and restarted the Backup Exec services on both, but that did not resolve the problem.

http://www.symantec.com/business/support/index?page=content&id=TECH127758

This morning I have also added the "Detach Timeout" registry entry on the BackupExec server referenced in the following article to see if it helps (I haven't tested extensively yet since adding it and restarting the services).

http://www.symantec.com/business/support/index?page=content&id=TECH180039

My environment is Windows 2008 R2 64-bit Backup Exec 2010 R3 SP4 on the tape backup server.  Windows 2008 R2 64-bit on the Exchange 2010 server.  Plenty of RAM, virtual memory allowed is very large, more than 100GB free diskspace on all drives.

This has been happening randomly on my backups for nearly 2 months now.  Looking back through the logs, it looks like the errors started occurring shortly after applying Exchange 2010 SP3 Rollup5 that was released nearly the end of February.  No idea if this is the cause, but I thought I'd mention it.

http://support.microsoft.com/?kbid=2917508

citybett1's picture

Followup to my previous post.  Adding the "Detach Timeout" registry entry referenced in http://www.symantec.com/business/support/index?page=content&id=TECH180039 hasn't fixed it either.

Last night I had the system run an Exchange with GRT backup to disk, and it failed with the same error on mailbox database #4. A few hours later the system ran an Exchange with GRT backup to tape, and it failed with the error on mailbox database #3. The previous night, no errors.

KirkBoyd's picture

The "Detach Timeout" registry entry has not fixed it for me either.  The backup still fails intermittently.

It's becoming frustrating.

 

 

citybett1's picture

Another person in the following thread is having the same problem, so we're not the only ones.

https://www-secure.symantec.com/connect/forums/backup-exec-2010-r3-final-error-0xe00002f7-cannot-extract-mailbox-messages-exchange-backup

I added the "VFF Extended Timeout", "Detach Timeout", and "OnHostTemp" registry keys to both the Backup Exec server and the Exchange server.  None of it fixed the problem.

I ended up opening a support case with Symantec and it's still being worked on.  The support tech ultimately couldn't resolve the issue either, so we ran the debug tool in Backup Exec, had the jobs fail and they handed all of the data off to the next level of support.  And that's where my problem currently sits.

citybett1's picture

No resolution yet.  The next level up of tech support is still working on it.  Based on my experience so far though, I'm guessing that rebooting the Exchange Server will "fix" the problem for you at least for awhile.

Prior to opening the case, I was having the random failures on GRT processing for both backups to disk and backups to tape.  As mentioned above in this thread, backups to tape do the GRT processing on the Exchange Server, and backups to disk do the GRT processing on the BackupExec server.

Anyway, to make a long story short, I ended up rebooting the BackupExec server, and so far since the reboot the backups to disk haven't failed.  But I have no idea if the problem is going to return in the future or not.

Tech support's analysis of the debug logs from the failures seems to indicate that the GRT processing fails because it can't find it's temp files that it created for some reason, which has everyone scratching their heads on why a reboot would fix the problem, but it seems to have fixed it (unknown if the problem is going to return) for the disk backups by rebooting the BackupExec server.   Antivirus isn't the culprit in my case on why the GRT processing can't find it's temp files.   If you think antivirus might be the culprit in your case, add the OnHostTemp registry key to force a specific location to be used for the temp files, and then exclude that location from your antivirus.

I'm sure I'll ultimately end up rebooting the Exchange server, but I haven't yet to allow Symantec to continue work on it since backup to tape still fails.

KirkBoyd's picture

This absolutely sucks...  I'm going to try and implement the OnHostTemp registry key and see if it helps.

Restarting the Exchange Servers fixes the problem for about a day or two.

Any luck at your end?

citybett1's picture

Nothing new really.  The symantec tech got back in touch with me to collect additional information from the servers, but that's about it.

I'm working around the problem by manually running additional backups of the database that failed and having it append the backups to the tape over and over again until the backup succeeds.  Last night, database #4 failed with the error, so this morning I ran a manual backup of just database #4 and had it append to the tape.  Running the manual backup of the single database eventually succeeds.

citybett1's picture

Just a followup.   No solution yet, Symantec's second level of support had to get their next higher up level of support involved and they're still working on it.

In the meantime, I'm still working around the problem by doing a manual backup of whichever database fails, as described in the post above.

KirkBoyd's picture

Thanks.  We were still getting inconsistent failures so we created seperate DB backup jobs as a workaround.

Let me know if you get a result.

citybett1's picture

I'll definitely post back whenever this gets resolved.  Tech support is still looking at it.  I don't think even the debug logs are giving them much to go on.   Basically to me it looks like the debug logs are indicating that the GRT process can't find it's own temp files that it just got done creating, so it gives up and moves onto the next database.

citybett1's picture

Well, here's the end of my story on this problem.  Symantec's developers ended up blaming the problem on the server running out of memory, and suggested I double the RAM from 8GB to 16GB.  Which doesn't make any sense to me, since the server always has free memory available to use, and PLENTY of virtual memory.

Rebooting the server periodically resolves the issue for awhile.