Nuts and bolts in NetBackup for VMware: Transport methods and TCP ports
It had been a while since my last blog on NetBackup for VMware. The next in line in that series was restore process flow. However, since there had been many questions on transport methods and TCP ports, let us talk about those for now. I will get to restore process flow soon!
Thus this blog is just an addendum to Understanding V-Ray vision through backup process flow. As that blog was too long already, I didn’t spend much time explaining various transport types and TCP ports but I received so many questions on those. Sorry for the delay.
Let us examine the transport types and ports usage through the view from VMware backup host.
There are three unique ways a backup host can stream data from VM data store. They are SAN, hot-add and NBD transports.
SAN Transport
The VMware backup host must be a physical system in order to use SAN transport. The data store LUNs are zoned to the backup host. The backup host can directly read VMDK objects from the datastore using vADP. The data stores can be Fibre Channel or iSCSI connected.
Pros:
True offhost backups, zero resources tapped from ESX/ESXi hosts
Cons:
Works only for SAN attached (Fibre Channel or iSCSI) data stores
Hot-add Transport
You can think of hot-add transport as a special case of SAN transport where the backup host is also a virtual machine. Plus there is a bonus, it works for non-SAN data stores as well. The backup host VM accesses the VMDK objects of other VMs directly from the data store (similar to SAN transport). It can protect all VMs on datastores to which it has access.
Pros:
No need for a physical backup host
Does provide offhost backups for those VMs not collocated with VM backup host
The most efficient backup method for NFS based data stores (e.g. NetApp or FileStore providing VM storage)
Cons:
It does use ESX server resources where the VM backup host is deployed
You cannot protect VMDK files larger than 1TB (a vADP limitation with hot-add)
NBD Transport
NBD stands for network block device. In this method, VMware Network File Copy (NFC) protocol is used such that the VMDK object looks like a block device to the backup host which can be read through network. You need one NFC connection for each VMDK file being backed up. Thus backup is streamed from ESX/ESXi system’s VMkernel port to VMware backup host. If your ESX/ESXi hosts have VMKernel ports (known as the management ports) with dedicated uplinks, your virtual machine traffic links are not affected.
Pros:
Works for all kinds of data stores, even the ones directly attached (DAS) to ESX hosts
The simplest one to setup from infrastructure perspective works with both physical and virtual backup hosts.
Cons:
Not an offhost solution. ESX resources are used for backups.
Which one is right for you? Well, it depends on your business needs. For a large enterprise data center where you already have invested heavily on Fibre Channel SAN for your vSphere infrastructure, SAN transport is ideal. Hot-add can also be used especially if you like to spread backup hosts onto to multiple multi-node ESX clusters. With the increasing popularity of 10Gb Ethernet, NBD is not inferior either. Good news is that NetBackup has the ability to automatically try various transports during backups.
TCP ports to vSphere infrastructure
What ports are needed from NetBackup to vSphere infrastructure? The answer is quite simple; you need access to TCP port 443 and 902.
Ability to connect to TCP port 443 on vCenter server is mandatory. This is the port at which NetBackup connects for all things needed from vCenter server. VM discovery requests, snapshot creation, snapshot deletion etc.
The backup host may also need the ability to connect to TCP port 902 on ESX/ESXi hosts. This is needed for specific use cases. If these are not applicable to your environment, no need for this port to be opened in firewall.
1. You are using NBD/NBDSSL transport for backups and/or restores.
2. You are doing restores through Restore ESX server bypassing vCenter
SAN and NBD Transports using a physical VMware backup host
Hot-add and NBD Transports using a physical VMware backup host
Our passion is protecting your data. Check out news and insights from the Symantec NetBackup team addressing datacenter issues like disaster recovery, de-duplication, Windows application protection and continuous data protection.
Comments
Thanx Abdul for this
Thanx Abdul for this information... This one was more informative unlike previous series of yours....
Cheers !!!
Thanks for the constructive feedback, Vaibhav.
It is only through the feedback that we would know if things like these are beneficial for the esteemed community and whether more like this need to be produced. I appreciate your time and feedback.
Warm regards,
Abdul "Rasheed" Rasheed
Ports between master and vCenter server
Thank you for this information, but which ports Master server and vCenter (or ESX) server use for communication?
margo
Normally master does not connect to vCenter server....
Hi Margo,
Technically speaking, the master server does not talk to vCenter server directly for backups or restores. Those are always managed via VMware backup host.
The exception to this rule is when you are browsing the virtual infrastructure (e.g. you are testing the query string for VIP) from an administration console connected to the master server, in this case it tries to communicate to vCenter/ESX server at port 443.
Hence the short answer to your question is….. NetBackup master uses port 443 to talk to virtual machine server (vCenter or ESX depending on what you had configured) if the specific configuration task requires a connection.
If you have a security requirement to limit connections to vCenter, you can house Windows Administration Console role on VMware backup host. In this case, the master never needs to connect to vCenter directly.
Warm regards,
Abdul "Rasheed" Rasheed
Luns do they need to be Online or Offline ?
Firstly great post.
I need some clarification if using SAN transport type on Windows Server 2008 for Vmware backup host.
NBU 7.1
Do the LUNs need to be just visible and Offline or do they need to show as Online.
This is for a backups.
I am aware of the Vmware best practices but I have seen some users stating the backups have been working with LUNs showing as offline but I contradict this from previous experience.
http://kb.vmware.com/selfservice/microsites/search...
NBD works and SAN fails with No Transport Method available for the given type.
OfflineShared is what you need for backups
Hi Craig,
You bring up an interesting point. This is an area that is controlled by VMware API and we have seen mixed results with different versions of API.
Strictly for backups, the SAN policy may be set to OfflineShared. I have seen this configuration working for many customers. However for restores, you would need to switch the SAN policy or online the LUN in question explicitly.
Warm regards,
Abdul "Rasheed" Rasheed
5220 appliance
May I know if the Hot-add mode is appliable to 5220 appliance? I was told that in order to utilize SAN transport, we must use a physical Backup host instead of a VM backup host which will only connect via LAN.
Yes, you can use hot-add with NetBackup 5220 appliance
Hi Alize,
You can use hot-add mode with 5220 appliance. In fact, this is a popular method with NetBackup 5220. Large environments use multiple hot-add hosts (the VMware backup hosts on VMs) which act as the V-Ray lenses for NetBackup 5220 appliance. This way you can scale-out streaming of backups from 1000+ VMs to a single appliance! You get to deduplicate the backup stream closer to the source or you could send the whole stream to appliance to do the deduplication. In NetBackup for VMware design, these hot-add hosts can send multiple streams concurrently to the appliance.
SAN transport is different from hot-add transport. SAN transport requires a physical backup host that is zoned to see the vSphere data stores. In those cases, NetBackup 5220 currently requires a seperate physical backup host because some of the key APIs (from VADP) are not ported to Linux (appliance OS uses Linux kernel) yet. Symantec (as usual) is taking the leadership role and working with VMware to make the port of required APIs to Linux possible.
Warm regards,
Abdul "Rasheed" Rasheed
Recently I install a
Recently I install a netbackup VMware solution to a customer using the hotadd method.
The backup systems ware in a different network subnet form ESXs and vcenter. As there is NO document from Symantec that described the firewall ports need to be opened, I folow your article and sent to the customer an email to open the ports 443 and 902.
Guess what, it did not work. I open a case to Symantec asking for ports and the answer was the following ports
SSL 443
SSH 21 & 22
http 80
vnetd 13724
veritas_pbx 1556
bpcd 13782
vopied 13783 (????)
No mentioning at all the 902 port, which I thing is needed. And he added to the email:
"Please not that we offer the VMware ports as advised but unconfirmed information and that you should always confirm the VMware ports with VMware themselves"
After that we add a second virtual network card bypassing the firewall. It is not good to do test to the client’s environment
I really do not know the ports needed, but definitely are not only 443 and 902.
Reading the veem manual, it has a page with a big range of ports need to be open for VMware backups. But every backup software, is different
stefanos.
I see your confusion here.
Hi smtp,
I see your confusion here. Note that the blog above is talking about the ports needed between VMware backup host and vSphere infrastructure. I didn’t spend anytime going through ports needed between VMware backup host (which is essentially a NetBackup client or NetBackup media server) and rest of the NetBackup environment. This part is well covered in NetBackup system administrator’s guide.
In your specific example, you are using hot-add which is a VMware backup host on a VM. The above blog only deals with communication requirement between this VM and your VMware infrastructure. As your media server(s) and master server are on different network, you also need to follow the instructions in NetBackup System Administrator’s guide on how to enable communications between master/media servers and NetBackup client.
Hope this helps. If you have difficulty in locating documentation for server to client communication for NetBackup, send me a note and I shall find it for you.
Warm regards,
Abdul "Rasheed" Rasheed
Probably you do not
Probably you do not understand what I’m writing. It maybe my fault as English is not my native language.
My problem is the communication from netbackup backup host and the VMware infrastructure. And My bigger problem is the answer from support…
Which VMkernel ports are used for NFC (NBD) traffic?
Thanks for the write up. I've been trying to find something similar produced by VMware but most of their documentation is related to the VDDK and VADP APIs.
So in your statement:
"Thus backup is streamed from ESX/ESXi system’s VMkernel port to VMware backup host. If your ESX/ESXi hosts have VMKernel ports (known as the management ports) with dedicated uplinks, your virtual machine traffic links are not affected."
Any idea which VMkernel port is used? If you have multiple VMkernel ports defined (for example, one dedicated for vMotion and another on a separate VLAN or dedicated 10GbE link for NFS datastores or dedicated for mangement traffic), what is the logic to decide which VMkernel port to use?
I know with ESXi this changes slightly as there no longer a service console port, and when configuring a new network port you only have connection types for Virtual Machine (VM port group) or VMkernel. But when adding a VMkernel port, you can specifically select the port properties to include: vMotion, Fault Tolerance logging and Management Traffic. I guess the question there is: do any of these port properties HAVE to be selected in order for the NFC traffic to use the port, or will it use any VMkernel port... and if so, which VMkernel port (hopefully it will be smart enough to try to use one on the same subnet if it exists).
Thanks
Hi bcblake, Unfortunately,
Hi bcblake,
Unfortunately, VDDK documentation does not state specific requirements as NBD inherits its root from Linux style kernel module. Technically, NBD can function so long as there is a path from backup host to any of the VMkernel port types. Having said that I would recommend not mixing vMotion traffic with backups as the former might be responding to DRS rules. As a best practice, I would suggest creating a dedicated network segment that backup host and an ESXi management port can use. You can create multiple management ports on an ESXi host, allocate one port to production segment for vCenter to manage the host, allocate another port to a completely different non-routable segment for backup traffic. This also allows to securely route backup traffic without going for processor intensive NBDSSL transport.
Hope this helps.
Warm regards,
Abdul "Rasheed" Rasheed
Thanks Abdul. I figured this
Thanks Abdul.
I figured this may be the case, and I am working with an environment where a new NIC will be added to a group of ESX servers and they will be placed onto a dedicated 10GbE network that will be non-routable. So as long as the VMware Access Host is on that same network, then I'm thinking normal routing rules should apply and the ESX server will decide to use the 10GbE NIC to route NBD traffic to the VMware Access Host. I just wanted to make sure that when we add the new NIC to make sure it is added to an appropriate vSwitch and what type of connection type should be used (VMkernel), which is what you have confirmed.
Brian
Hi Abdul you Nuts & bolts
Hi Abdul
you Nuts & bolts series had helped alot me in many cases to design and provide robust solution for VMware
however V-Ray is tighten with both virtualization market leader. Looking forward the series for Hyper-V too.
As far i know presently there are limitation from Hyper-V end , but need to know What NetBackup can benefit to this customers
Cheers !!!
Certainly! I hope to create one on Hyper-V by May.
Appreciate the feedback.
Warm regards,
Abdul "Rasheed" Rasheed
NFS recommendations or documentation
Thanks for the information. I wonder if there is any documentation that expands on the best practices or recommendations for NFS backups of VMware? In your example above you show the VMware Backup Host as a VMware guest. I cannot find any references to doing this in the NetBackup 7.1 Admin Guide for VMware.
Good catch!
Hi tD,
Good catch. I understand that we do not call that out explicitly in out documentation. I shall work with our writers to add a section on NFS data stores.
If data stores are on NFS, you can use any transport method other than SAN transport. That gives us hot-add, NBD and NBDSSL. The choice between hot-add and NBD may be determined as follows.
1. If you do not want ESXi network stack to be used for backups at all, hot-add is recommended. Place the hot-add systems on strategic ESXi hosts such that it has access to all the data stores in your environment. If you have VMware HA, let it be monitored by it so that it is restarted on a different ESX host in case the one currently hosting fails. I would also recommend setting the resource pools such that you have enough reservations on CPU and memory for the hot-add host. With multiple hot-add hosts, you can scale-out backup processing.
2. If you are using an enterprise grade NFS storage with dedicated network where VMkernel port is attached, you can simply use NBD or NBDSSL. This works great and also eliminates the operational complexity in managing hot-add VMs. The downside with NBD/NBDSSL is that it is not a 100% offhost solution as you do use the network stack from ESXi hosts. With modern day hardware, it may not be a concern.
Warm regards,
Abdul "Rasheed" Rasheed
Hi Abdul, Can you also
Hi Abdul,
Can you also share some light on Hyper-V integration too
Cheers !!!
Certainly.
I am travelling a lot next month. Long flights are my faviourite time to write technical articles! I shall keep this in mind. Thank you for the suggestion.
Warm regards,
Abdul "Rasheed" Rasheed
Would you like to reply?
Login or Register to post your comment.