Nuts and bolts in NetBackup for VMware: Discovery job in VMware Intelligent Policy
There was a request from Rizwan to explain how NetBackup for VMware works. Several votes and comments later, I came to know about it from a technical support engineer who had pointed me to that post. We hear you loud and clear, we are looking into getting more details in NetBackup for VMware System Administrator’s Guide. While we wait for a formal documentation update, I thought I better write a set of blogs on what was requested by the community. So here it goes.
Let me not repeat what is already there in the manual. If you are new to NetBackup for VMware, I would strongly recommend going through NetBackup for VMware System Administrator's Guide for an introduction.
For this discussion let us assume that the NetBackup master server, media server and VMware backup host are three different systems. Also let us assume that the VMware backup host and VMware discovery host are the same system. The request was to provide a flow of communications for backups and restores. In order to get there, first we need to discuss virtual machine discovery as well.
NetBackup for VMware features two kinds of virtual machine discovery. The newer VMware Intelligent Policy (VIP) discovery and original browse and select discovery. The former creates a discovery job in activity monitor; hence let us discuss its process flow before getting into backup.
VMware Intelligent Policy Discovery: This game changing discovery method is part of Symantec’s V-Ray vision; a set of technologies to provide unique visibility into virtual machines. Your virtual machine protection strategy enters auto pilot mode with VIP. In the policy you are simply specifying selection criteria for what needs to be protected (e.g. all the virtual machines in data center X, all virtual machines in cluster Y, all virtual machines with RedHat Linux etc.). NetBackup automatically discovers the VMs matching the criteria and runs backups. How cool is that! Visibility matters! Let us go through a high level overview of the process flow for VIP discovery job in this blog.
- When backup window opens, NetBackup Policy Execution Manager (nbpem) on master server looks to see if there is a VM list xml file in its policy database (we will explain this later) to see if it can use the pre-existing list of virtual machines. If the xml file is present, it examines its age to see if a new one needs to be generated. Let us assume that the xml file is not present or it is older than the time interval specified in the policy attribute named “Reuse VM selection query results for”. Then a discovery job shows up in Activity Monitor.
- nbpem on master contacts NetBackup Client Service (nbcs) on media server*.
- nbcs on media server starts NetBackup Client Service (nbcs) on VMware backup host*.
- nbcs has a plug-in for VMware which gets loaded. VMware APIs for Data Protection (vADP) calls are implemented in this plug-in. NetBackup has credentials to access the vSphere host (vCenter or ESX/ESXi), it logs into the vSphere host and runs VM discovery based on the selection query given in the policy. The vSphere host returns an xml file with the results. This file may or may not be processed further by the plug-in based on the kind of query.
- nbcs returns the VM list xml file to master server through nbcs on media server*.
- nbpem on master server uses the xml file to generate the client list. The discovery job in Activity monitor moves to Done state.
- If the backup is due it is started by nbpem. The xml file persists in NetBackup database until its age exceeds the time specified in the policy attribute named “Reuse VM selection query results for”
*We assumed here that media server and VMware backup host are separate hosts. As NetBackup is an enterprise platform, there could be a firewall between NetBackup master server and VMware backup/discovery host. The nbcs on media server starts nbcs on VMware backup host on behalf of master server in this case. If the VMware backup host and media server are the same, NetBackup is smart enough to use a singe nbcs process.
VMware Intellgent Policy is availble in NetBackup 7.1 and NetBackup 5200 series appliances with software version 2.0.
Nuts and Bolts in NetBackup for VMware series continues here:
Comments
I am looking for your feedback!
As I mentioned in the blog, I am writing this series per community request. Please do give your feedback so that I can update this blog as well as the other ones coming in this series. Once we know what you guys are looking for, we can also easily accommodate those in our documentation. Thank you for your time and feedback!
Warm regards,
Abdul "Rasheed" Rasheed
Tweet me @AbdulRasheed127
Thank You
Hi Adbul,
The reason i requested for VMware backup backup & restore flowchart is to equip ourself in troubleshooting any Vmware related backup or restore issues. We would all appreciate if symantec can provide us details process flow, like the what process are involved during the backp/restore operation and on which port they call query for.
Recently we had restore problem for VMware, where the cause was unknown after checking the VxMS (dont know what VxMS does) logs that Backup host needs access on port 902 in ESX host. There are many such things which has to be explained.
I really appreciate your effort in helping us with this request.
Best Regards,
Rizwan
Restore process flow coming soon.
Hi Rizwan,
I did write a rather long blog on vStorage backup process flow. I didn't go in depth in ports there as the needs are different for different transport methods. I shall write about it when I am back in office.
Restore process flow coming soon.
I appreciate the feedback.
Warm regards,
Abdul "Rasheed" Rasheed
Tweet me @AbdulRasheed127
Maybe publish a separate TN
Maybe publish a separate TN for port usage with VMware backups? Or update the existing ones? e.g. http://www.symantec.com/docs/TECH136090
The ONLY reference to port(s) in the NBU fo VMware guide is the following:
Connect using port number
If the default port number has not been changed on the VMware server, no
port specification is required. In that case, make sure that the Connect using
port number box is not checked.
If the VMware server has been configured to use a different port, click the
Connect using port number box and specify that port number.
This is hardly helpful in a secure environment with multiple DMZ's.............
Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links
Certainly Marianne!
I shall work on a blog explaining the transport methods which is the right place for me to explain the port requirements.
Warm regards,
Abdul "Rasheed" Rasheed
Tweet me @AbdulRasheed127
VM query from cli
Abdul,
Is there a way to invoke nbcs from the cli on the hotadd proxy client to remove the master/media server connections as a troubleshooting mechanism? I have often wanted to run the VIP query straight from the hotadd.
Thanks for your time.
Darrel
Can backup host be a VM?
Hi Abdul,
Can my backup host be a VM?
thanks
Please have a look at this
Please have a look at this recommendation right in the beginning of the blog:
"If you are new to NetBackup for VMware, I would strongly recommend going through NetBackup for VMware System Administrator's Guide for an introduction."
See NBU for VMware Admin Guide www.symantec.com/docs/DOC3663
Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links
The answer is yes.
Hi vksing,
As Marianne had mentioned, the details are in the admin guide. A VM can indeed be a backup, recovery and discovery host. You will be able to use hotadd and NBD transports. More on transports and ports coming soon.
Warm regards,
Abdul "Rasheed" Rasheed
Tweet me @AbdulRasheed127
Manual backup from a VIP policy
Hello,
thank you for the useful blog post.
We like very much the VMWare Intelligent Policy discovery, the automatic query mechanism and the other related features.
In our environment we often need to manually start a backup of a single VM, but this is impossibile when using the VIP: trying to launch a manual backup on a VIP policy will result in the backup of every VM matching the query.
We also opened an idea/feature request on Symantec Connect suggesting a solution: https://www-secure.symantec.com/connect/ideas/automatic-vm-selection-through-query-manual-backup-selection-problem
Do you know if there is a way to launch a manual backup of a single VM from a automatic query policy?
Thank you very much for posting the idea!
Hi e-security,
You brought up something we had been looking to implement. Thank you very much for posting the idea. The more votes we get, its priority will go up in the implementation list.
You are right, if you need a onetime backup you can run it from a separate policy at this time. I have seen two customers who had created a policy dedicated for manual backups while using VIP. VIP automates protection and manual policy (normally in the deactivated state) is used for one time backups while preparing for upgrades, patching and such.
Warm regards,
Abdul "Rasheed" Rasheed
Tweet me @AbdulRasheed127
That's exactly how we're
That's exactly how we're dealing with the problem right now (so that's 3 clients now ;).
The sooner this is implemented, the better: the re-validation time when adding to the manual backup even a single VM on a some-hundreds-VMs-environment is taking forever!
Regards
In our environment "Test
In our environment "Test query" button in automatic selection of clients (in policy) takes from 5 to 20 minutes. Any chance we could lower this waiting time? As far as I understand, NBU could fetch vm machine list only once, and then "apply" filter rule(s), instead of every time making a connection to vCenter / ESX to get a long list of machines and then "apply/test" filter rule(s).
We're using 7.1.0.4 on master/media servers and backup/proxy hosts.
Thanks.
disableIPResolution
In your backup host, open the registry and add a new key "BACKUP" under "HKLM\SOFTWARE\Veritas\NetBackup\CurrentVersion\Config"
Click the key BACKUP and in the right hand pane, right click and add a new DWORD key with name "disableIPResolution" and value 0.
We have 3 vcenter domain, after doing the above change, the VM query only take 2 -5 minutes to fetch the inventory. Hope this is helpful.
Best Regards,
Rizwan
Hi puga, As Rizwaan
Hi puga,
As Rizwaan mentioned, disableIPresolution will mitigate the problem if you have a large environment where the proxy system cannot resolve all the VM hostnames.
Also, it is possible to minimize the number of times NetBackup contacts vCenter server by using the 'reuse query every xx' hours field. Becuase of this setting, the fetch does not occur for each job.
If this does not resolve the problem, please do work with technical support. Perhaps you have an environmental issue that they can help to isolate it for you.
Warm regards,
Abdul "Rasheed" Rasheed
Tweet me @AbdulRasheed127
Hi all, My vm clients on
Hi all,
My vm clients on vmx 7, no errors in vcb side sanpshot is happening fine on vcb end .Netbackup version is 7.1.0.4 .and my backup off host and media server is same .Still full backups fail with 156 and incrementals with 13.any suggestions?
Would you like to reply?
Login or Register to post your comment.