Video Screencast Help

Cannot browse or back up virtual machines using vSphere in NetBackup 7.5

Created: 15 Mar 2013 • Updated: 18 Mar 2013 | 9 comments
This issue has been solved. See solution.

Hi all,

My systems was running Netbackup 7.0.1 backing up virtual machines using VMware API (backup host), backups were running without any issues at all for about one year os so. recently( yesterday) I upgraded the backup servers (Solaris 10 64 bit) and the backup host to Netbackup 7.5.0.3, after the upgtrade all the VMware backups are failing with codes 13 and 156, I am not able to browse or select any virtual machines.

I have created 3 support cases with Symantec support but have not got anywhere.

Hoping that someone in this group can help me out

 

Thanks

Operating Systems:

Comments 9 CommentsJump to latest comment

Nagalla's picture

hi ,

are you able to validate the Vcenter/ESXi credentials from the GUI---> Credentials ---> virtual machine servers.

as you are not able to browse , I would start from the credentials validation. 

could you give us more info about your Setup

did your backup host is master/media server or just client?

could you let us know the OS and netbackup version of backup Host?

and also the details status of the failed job.

 

evargas's picture

yes, the credential did not changed, I was able to remove and add the Vmcenter credential, no issues there.

backup host is a Windows 2008 standard (virtual machine) I was backing up using the hotadd transport which was working before the upgrade.

Activity monitor these are for the same client, (same schedule two different backup jobs)

3/14/2013 9:23:40 PM - Info bpbrm(pid=2944) Starting create snapshot processing         
3/14/2013 9:23:44 PM - Info bpfis(pid=4264) Backup started           
3/14/2013 9:23:45 PM - snapshot backup of client bwi-relay-01 using method VMware
3/14/2013 9:38:45 PM - Error bpbrm(pid=2944) socket read failed: errno = 131 - Connection reset by peer  
3/14/2013 9:38:46 PM - Info bpfis(pid=0) done. status: 13: file read failed       
3/14/2013 9:38:46 PM - end writing
Status 13

3/14/2013 9:03:59 PM - started process bpbrm (29503)
3/14/2013 9:04:03 PM - Info bpbrm(pid=29503) bwi-tdmgr-01 is the host to backup data from     
3/14/2013 9:04:03 PM - Info bpbrm(pid=29503) reading file list from client        
3/14/2013 9:04:03 PM - Info bpbrm(pid=29503) start bpfis on client         
3/14/2013 9:04:03 PM - Info bpbrm(pid=29503) Starting create snapshot processing         
3/14/2013 9:04:07 PM - Info bpfis(pid=3472) Backup started           
3/14/2013 9:04:08 PM - snapshot backup of client bwi-tdmgr-01 using method VMware
3/14/2013 9:04:13 PM - Critical bpbrm(pid=29503) from client bwi-tdmgr-01: FTL - snapshot creation failed, status 156   
3/14/2013 9:04:13 PM - Warning bpbrm(pid=29503) from client bwi-tdmgr-01: WRN - ALL_LOCAL_DRIVES is not frozen    
3/14/2013 9:04:13 PM - Info bpfis(pid=3472) done. status: 156          
3/14/2013 9:04:13 PM - end Flash Backup Windows, Create Snapshot; elapsed time: 00:00:15
3/14/2013 9:04:13 PM - Info bpfis(pid=0) done. status: 156: snapshot error encountered       
3/14/2013 9:04:13 PM - end writing
Status 156

 

Nagalla's picture

when the backup is running , did you see any errors or notinficationsin Vcener console.?

did you see if there is old snapshots are in ESX hosts?

evargas's picture

There are no errors on the VCenter and there are no left over snapshots.

 

Nagalla's picture

3/14/2013 9:04:08 PM - snapshot backup of client bwi-tdmgr-01 using method VMware
3/14/2013 9:04:13 PM - Critical bpbrm(pid=29503) from client bwi-tdmgr-01: FTL - snapshot creationfailed, status 156   
3/14/2013 9:04:13 PM - Warning bpbrm(pid=29503) from client bwi-tdmgr-01: WRN - ALL_LOCAL_DRIVES is not frozen

bwi-tdmgr-01 is still valid VMserver?

you would probably need to check why Vcenter  is not browsing the VMs?

delete all the entries in the below path of backup host

Install path\Veritas\NetBackup\online_util\fi_cntl

then try browsing again.... 

if it does not work...

login to the Vcenter with the same logins that you provided to the netbckup, and check if you see anything odd with that login

 

 

 

evargas's picture

yes, bwi-tdmgr-01 is a valid VMserver

yes, Vcenter is browsing the VMs

I have already delete all the entries in the below path of backup host > Install path\Veritas\NetBackup\online_util\fi_cntl

I am still can't browse the VMs, GUI times out when browsing the VMs (error code 195)

I am abkle to login to the Vcenter with the same login that the netbackup host is using. nothing wrong with this credentials.  like I mentioned the system was working perfectly before I upraded to 7.5

I believe that there must be a Netbackup bug somewhere, Symantec support still have not provided any resolution to these issues.  this is the second system I upgraded with same problem.  I will not upgrade anything else until I find what the bug is.

 

 

 

CRZ's picture

This TechNote may help you with your 195 statuses:

"Request has timed out (195)" when attempting to Browse for Virtual Machines in new policy for one VMware backup host
 http://symantec.com/docs/TECH190396

I see you're at 7.5.0.3.  Could you be hitting this issue mentioned in the LBN which is resolved in 7.5.0.5?

VMware backups failing with status 13: file read failed.
 http://symantec.com/docs/TECH199087

I'm not sure this matches your issue or not... but I guess I'm trying to prod you into applying 7.5.0.5 as it's got hundreds of other fixes in it as well.

evargas's picture

I upgraded the Netackup servers and the backup host to Netbackup 7.5.0.5 without positive results.

I don't have that may VMs on the system, about 15 that is not that much clients to cause a delay when browsing.  I hit the wall I really don't know could have changed after the upgrade.

 

evargas's picture

I resolved the issue this past weekend.

I really could not find the root cause, so this is how I end up fixing it.

a) Removed the Netbackup software from the backup host (removed directories and all the registry keys)

b) Reboot the backup host

c) Installed Netbackup 7.5

d) Tested connectivity to VCenter and made sure I can browse the Virtual Machines from the backup host

e) Run a test backup to make sure I can backup the VMs

f) Apply the Netbackup 7.5.0.3 patch

g) Run a another test backup

 

backups are running again.

SOLUTION