Snapshot error encountered... VIXAPI freeze failed with -1
Hi all,
We've been backing up our VMware clients fine for months/years but when we came in this morning, all of the sanpshot backups had failed. As far as I'm aware, nothing has changed on VCentre, it was upgraded from 4.0 to 5.0 a while ago without any issues. We can snapshot manually through VSphere on some vms' but not all, and when a new client was added to the backup policy that backed up okay. The section of Bpfis that looked like it would tell us what was going on (below) just got me to an article here http://www.symantec.com/business/support/index?page=content&id=TECH192007 but as we don't use chinese names or characters and have never done so I couldn't see how it helped. Any ideas as we're a bit stuck?
The VM's are a mixture of 2003 and 2008, Netbackup is 7.1.0.3
11:51:22.392 [1824.5564] <32> onlfi_freeze_fim_fs: FTL - VfMS error 11; see following messages:
11:51:22.392 [1824.5564] <32> onlfi_freeze_fim_fs: FTL - Fatal method error was reported
11:51:22.392 [1824.5564] <32> onlfi_freeze_fim_fs: FTL - vfm_freeze: method: VMware, type: FIM, function: VMware_freeze
11:51:22.392 [1824.5564] <32> onlfi_freeze_fim_fs: FTL - VfMS method error 7; see following message:
11:51:22.392 [1824.5564] <32> onlfi_freeze_fim_fs: FTL - VMware_freeze: VIXAPI freeze failed with -1
11:51:22.392 [1824.5564] <32> onlfi_freeze: FTL - VfMS error 11; see following messages:
11:51:22.392 [1824.5564] <32> onlfi_freeze: FTL - Fatal method error was reported
11:51:22.392 [1824.5564] <32> onlfi_freeze: FTL - vfm_freeze: method: VMware, type: FIM, function: VMware_freeze
11:51:22.392 [1824.5564] <32> onlfi_freeze: FTL - VfMS method error 7; see following message:
11:51:22.392 [1824.5564] <32> onlfi_freeze: FTL - VMware_freeze: VIXAPI freeze failed with -1
11:51:22.392 [1824.5564] <2> onlfi_thaw: INF - fim=auto
11:51:22.392 [1824.5564] <2> onlfi_thaw: INF - fim=Hyper-V
11:51:22.392 [1824.5564] <2> onlfi_thaw: INF - fim=VSS
11:51:22.392 [1824.5564] <2> onlfi_thaw: INF - fim=VMware
11:51:22.392 [1824.5564] <2> onlfi_thaw: INF - fim=NAS_Snapshot
11:51:22.392 [1824.5564] <2> onlfi_thaw: INF - fim=VSS_Writer
11:51:22.392 [1824.5564] <16> bpfis main: FTL - snapshot creation failed, status 156
Thanks in advance,
Katie
It may be this line that is
It may be this line that is relevant:
11:51:22.392 [1824.5564] <2> onlfi_vfms_logf: INF - vSphere_freeze: snapshots active, aborting due to parameter selection
It seems to indicate that there are exisiting snapshots and your selection in the snapshot options is to abort if an existing snapshot exists
Check for existing snapshots or change your option to Remove NBU or Ignore
Hope this helps
Comments
Could you post as an
Could you post as an attachment the entire bpfis log along with the detailed section of the job failure (parent and child jobs if they both exist)
Thanks
Authorised Symantec Consultant
Don't forget to give a "Thumbs Up" or mark as "Solution" if someones advice has helped you.
Detailed bpfis log. There was
Detailed bpfis log.
There was one vm that completed the snapshot, this was um01, which is not a new vm, and also hasn't had any updates or reboots and is on the same host as others that failed.
Thanks for looking at this
Katie
Sorry Mark, I'm not sure how
Sorry Mark, I'm not sure how to get the detailed section of the job failures unless you mean the below?
26/11/2012 16:40:08 - Info nbjm(pid=6200) starting backup job (jobid=2108447) for client appmonitor, policy Corp_Vmware_NC, schedule Weekly
26/11/2012 16:40:08 - Info nbjm(pid=6200) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=2108447, request id:{F7F56E17-88A5-467E-A510-817E5F3CC75A})
26/11/2012 16:40:08 - requesting resource CHMediaServer05Pool
26/11/2012 16:40:08 - requesting resource appnbu-v.NBU_CLIENT.MAXJOBS.appmonitor
26/11/2012 16:40:08 - requesting resource appnbu-v.NBU_POLICY.MAXJOBS.Corp_Vmware_NC
26/11/2012 16:40:08 - granted resource appnbu-v.NBU_CLIENT.MAXJOBS.appmonitor
26/11/2012 16:40:08 - granted resource appnbu-v.NBU_POLICY.MAXJOBS.Corp_Vmware_NC
26/11/2012 16:40:08 - granted resource MediaID=@aaab0;DiskVolume=I:\;DiskPool=CHMediaServer05Pool;Path=I:\;StorageServer=appnbu5;MediaServer=appnbu5
26/11/2012 16:40:08 - granted resource CHMediaServer05Pool
26/11/2012 16:40:08 - estimated 42667131 Kbytes needed
26/11/2012 16:40:08 - begin Parent Job
26/11/2012 16:40:08 - begin Flash Backup Windows, Start Notify Script
26/11/2012 16:40:08 - Info RUNCMD(pid=7984) started
26/11/2012 16:40:08 - Info RUNCMD(pid=7984) exiting with status: 0
Status 0
26/11/2012 16:40:08 - end Flash Backup Windows, Start Notify Script; elapsed time: 00:00:00
26/11/2012 16:40:08 - begin Flash Backup Windows, Step By Condition
Status 0
26/11/2012 16:40:08 - end Flash Backup Windows, Step By Condition; elapsed time: 00:00:00
26/11/2012 16:40:08 - begin Flash Backup Windows, Read File List
Status 0
26/11/2012 16:40:08 - end Flash Backup Windows, Read File List; elapsed time: 00:00:00
26/11/2012 16:40:08 - begin Flash Backup Windows, Create Snapshot
26/11/2012 16:40:08 - started
26/11/2012 16:40:09 - started process bpbrm (4700)
26/11/2012 16:40:11 - Info bpbrm(pid=4700) appmonitor is the host to backup data from
26/11/2012 16:40:11 - Info bpbrm(pid=4700) reading file list from client
26/11/2012 16:40:11 - Info bpbrm(pid=4700) start bpfis on client
26/11/2012 16:40:11 - begin Create Snapshot
26/11/2012 16:40:11 - Info bpfis(pid=2160) Backup started
26/11/2012 16:40:11 - snapshot backup of client appmonitor using method VMware
26/11/2012 16:40:13 - Critical bpbrm(pid=4700) from client appmonitor: FTL - snapshot creation failed, status 156
26/11/2012 16:40:13 - Warning bpbrm(pid=4700) from client appmonitor: WRN - ALL_LOCAL_DRIVES is not frozen
26/11/2012 16:40:14 - Info bpfis(pid=2160) done. status: 156
26/11/2012 16:40:14 - end Create Snapshot; elapsed time: 00:00:03
26/11/2012 16:40:14 - end writing
Status 156
26/11/2012 16:40:14 - end Flash Backup Windows, Create Snapshot; elapsed time: 00:00:06
26/11/2012 16:40:14 - begin Flash Backup Windows, Stop On Error
Status 0
26/11/2012 16:40:14 - end Flash Backup Windows, Stop On Error; elapsed time: 00:00:00
26/11/2012 16:40:14 - begin Flash Backup Windows, Delete Snapshot On Exit
26/11/2012 16:40:14 - begin Flash Backup Windows, Delete Snapshot On Exit
26/11/2012 16:40:15 - started process bpbrm (4692)
26/11/2012 16:40:17 - end writing
Status 0
26/11/2012 16:40:17 - end Flash Backup Windows, Delete Snapshot On Exit; elapsed time: 00:00:03
26/11/2012 16:40:17 - begin Flash Backup Windows, End Notify Script
26/11/2012 16:40:18 - Info RUNCMD(pid=7772) started
26/11/2012 16:40:18 - Info RUNCMD(pid=7772) exiting with status: 0
Status 0
26/11/2012 16:40:18 - end Flash Backup Windows, End Notify Script; elapsed time: 00:00:01
Status 156
26/11/2012 16:40:18 - end Flash Backup Windows, Delete Snapshot On Exit; elapsed time: 00:00:04
snapshot error encountered(156)
It may be this line that is
It may be this line that is relevant:
11:51:22.392 [1824.5564] <2> onlfi_vfms_logf: INF - vSphere_freeze: snapshots active, aborting due to parameter selection
It seems to indicate that there are exisiting snapshots and your selection in the snapshot options is to abort if an existing snapshot exists
Check for existing snapshots or change your option to Remove NBU or Ignore
Hope this helps
Authorised Symantec Consultant
Don't forget to give a "Thumbs Up" or mark as "Solution" if someones advice has helped you.
Thanks Mark, my eyes were
Thanks Mark, my eyes were getting a bit tired looking at the logs and I didn't see that. I'll let you know tomorrow if this fixes
Regards,
Katie
Thanks Mark, old snapshots
Thanks Mark, old snapshots were preventing the new ones from starting. Don't suppose you'd have any ideas why the snapshots are not being deleted after they are backed up?
Katie
There are various reasons
There are various reasons that this could happen and it will help when you upgrade to 7.5.0.4 as it adds extra events directly into vCenter which may give clues
You would need to take a look at the bpfis logs from the last set of successful backups to see if you can spot anything in those - usually a crash of network glitch somewhere (if the jobs have been running OK up until now)
The clue should be in the last set of successful backups - or at least their bpfis logs (unless you did have a set of failures prior to this set of 156 errors which could have cause the oprhaned snapshots)
Glad to have helped - hope this helps further
Authorised Symantec Consultant
Don't forget to give a "Thumbs Up" or mark as "Solution" if someones advice has helped you.
Would you like to reply?
Login or Register to post your comment.