Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.
Netting Out NetBackup

Frequently Asked Questions on NetBackup Accelerator Part II

Network Attached Storage and Network Data Management Protocol
Created: 28 Mar 2014 • Updated: 03 Apr 2014 • 17 comments
AbdulRasheed's picture
+5 5 Votes
Login to vote

Thanks to everyone who supported my earlier blog on Frequently Asked Questions on NetBackup Accelerator. I have received a number of follow up questions (as comments) in that blog and I had been answering them as time permits. Two recent questions needed some elaboration. Thus, I thought it is better to put them as a Part II in a new blog. 

Can I use NetBackup Accelerator to backup Network Attached Storage (NAS) devices?

Yes, you can use NetBackup Accelerator to protect any NAS device that supports NFS and/or CIFS protocols. You would need a mount host that supports NFS/CIFS where the NAS volume can be mounted.  Note that a given volume from NAS device must be mounted on the same host and must use the same mount point for all the backup runs to take advantage of NetBackup Accelerator.

This mount host can be a NetBackup client or NetBackup media server. NetBackup 5200 series appliances can also function as the mount host.

You also have the ability to scale-out NAS protection using multiple mount hosts. If you have a large NAS device with multiple volumes, you can distribute the volumes across multiple mount hosts and perform backups concurrently to scale-out backup jobs.

Is NFS mount host or CIFS mount host better to make use of NetBackup Accelerator?

From NetBackup Accelerator’s perspective, it does not matter which protocol you use. The question really is whether the implementation server (on NAS device) and client (on mount host) are superior for NFS or CIFS. If your environment supports both protocols, I recommend experimenting both and decide what works better for you. Furthermore, there could be other site specific or environmental factors that may decide your choice on NFS or CIFS.

Is NetBackup for NDMP or NetBackup for Accelerator using a mount host better for protecting a NAS device?

To answer this question, we need to look at the fundamentals of NDMP. As you already know, backing up NAS devices by mounting NFS volumes on a backup server in early days was quite painful. Contrary to popular myth, tape drives are much faster than disk drives and a file system backup through NFS used to cause shoe shining.

NDMP protocol was originally developed so that control and data path from primary (NAS) to secondary (backup) storage could be separated and the NAS vendors can build their own data-streaming servers (NDMP data servers) within the device. For example, NetApp by default uses ‘dump’ format while Celerra uses ‘tar’. Each has its own pluses and minuses. Many UNIX/Linux admins would agree that dump is in general faster than tar. Tape drives can directly be attached to the NAS device (or attached to other systems where NDMP tape server function is implemented, e.g. NetBackup media server) while backup server owns the control and cataloging function.

  The point here is that although NDMP is standard protocol used in NAS devices, its implementation varies from vendor to vendor. Hence it is difficult to make a blanket statement on performance stats for backing up NAS devices using NetBackup for NDMP vs. NetBackup Accelerator using NFS/CIFS.

  Now let us talk about a specific benchmark we had done in house. Here we are comparing NDMP implementation in NetApp with NetBackup Accelerator based backup using a Linux NFS mount host. Take a look at graphical representation of the results below. The vertical axis is the number of minutes taken to finish the backup.

Untitled.png

 

The Data set:

Average size of files = 529kB

Number of files = 1.7 million

Total workload ~= 900GB

The blue cones represent NetBackup for NDMP backups to a tape drive directly attached to the filer and red cones represent NetBackup for NDMP backups sent to a deduplication pool attached to a media server. These tests are given as baselines.

The green cones represent NetBackup Accelerator backups using an NFS mount.

Standard Disclaimer: This is an internal benchmark in Symantec labs. Your mileage may vary depending on your environment.

Now let us interpret these results.

Direct NDMP to tape

The first thing that you will notice is that NDMP backups to tape is the fastest when it comes to shipping entire workload onto backup storage (traditional full backup). This is not surprising because NetApp NDMP data server uses dump which does not have any overhead while moving data from its volumes, and tape is faster than SATA disks attached to deduplication pool.

Since we are on this topic, let us also remember that deduplication appliances vendors are not telling the truth if they claim that a VTL (virtual tape library) implementation will provide faster NDMP backups than those to physical tape drives. The NAS devices feature volumes created on top of high performance disks striped across multiple spindles. The read throughput from such a volume is not something matched by write performance of lower RPM SATA disks typically used for deduplication appliances.

Remote NDMP to NetBackup Deduplication Pool

You would notice that the first backup using remote NDMP to NetBackup Deduplication Pool takes longer to complete when compared to direct NDMP backup to tape. This is expected because of two reasons.

  • The pipe between NetApp and media server is 1Gb Ethernet whereas the tape drive was served by a 4Gb FC connection
  • As I mentioned earlier, low RPM SATA disks typically used for backup storage cannot match the high RPM production disks on a NAS device

There is improvement for remote NDMP to disk for incremental backups. This is because of the intelligent stream handler in NetBackup deduplication engine for NDMP, which reduces the amount of data to be written onto storage.

NetBackup Accelerator using NFS

For apple-to-apple comparison with remote NDMP, the media server acts as the NFS client and uses the same 1Gb Ethernet connection. The initial full backup using NFS does not get any ‘acceleration’ because there is no track log for the mount point yet. Even after a decade and half since NAS devices had gone mainstream, NFS by itself has the same challenges. A traditional backup using NFS takes much longer to complete. This further justifies the use of NDMP protocol even for modern data centers.

But NetBackup Accelerator changes the game and makes NFS usable. As you see in the 2nd and 3rd runs of NetBackup Accelerator based full backups, it takes much less time to finish a full backup. For 2% data change, it is possible to create a full backup 21x faster than traditional backup using NFS, 10x faster than remote NDMP backups and 6x faster than direct NDMP backups to tape. As NAS workloads typically involve large number files with low change rate, NetBackup Accelerator is a great solution if you implement this for the right workloads. 

Of course, there are exceptions. The most important one is when a NAS device is serving a VMware vSphere datastore where virtual machine disk files are continually changing. The good news is that we have NetBackup Accelerator for VMware vSphere to meet such a workload. 

NetBackup Accelerator using NFS/CIFS also solves for another problem normally present in NDMP backups; platform independence. As data streams from NDMP is vendor dependent, you cannot recover across platforms. For example, you cannot easily restore NDMP backups from EMC Celerra to NetApp Data ONTAP or vice versa. Thus, for agile data centers that prefer heterogeneity to avoid vendor lock-in, NetBackup Accelerator using NFS/CIFS can be a powerful migration tool. 

Comments 17 CommentsJump to latest comment

vzappula's picture

Abdul,

Great benchmark, thanks for running and sharing these test.  I'd like to see the restore numbers as well for NDMP vs Accelerator if you run those as well.  

0
Login to vote
AbdulRasheed's picture

Thanks vzappula. NetBackup Accelerator (today) accelerates backups. The backup images from traditional and accelerated backups look the same in backup storage. There is no difference between restores from a traditional backup vs. accelerated backups. 

I understand that you may be asking about the performance differences between restores over NFS vs. restores through NDMP. As this was not really about 'Accelerator', we had not done a benchmark in the context of that feature. 

Having said that here are my thoughts on this. 

  • If you are talking about individual file recovery, I don't see much difference for performance. But there is something to remember about ACLs. I shall add this consideration to the blog above. 
  • If the goal is to restore the entire volume, I expect NDMP to outperform restores over NFS as the former has less overhead. 

 

Warm regards,

Abdul "Rasheed" Rasheed

Tweet me @AbdulRasheed127

+1
Login to vote
Chris Weis's picture

Abdul, I assume you meant 529KB average file size, not 529MB, correct?

 

 

0
Login to vote
AbdulRasheed's picture

You are absolutely right! I went back and double checked. I messed up the unit while generating the graph. I have corrected it now. 

Warm regards,

Abdul "Rasheed" Rasheed

Tweet me @AbdulRasheed127

0
Login to vote
josemiguel_casado's picture

Hi Abdul,

first of all, thank you for your posts about Accelerator, they are great.

Now, my question for you. I have a customer who is deciding wether to use Accelerator or NDMP on their backups to an MSDP. This is the environment:

- Media Server: Windows 2008 R2 with NBU 7.6.0.1

- Hitachi HNAS Device directly connected to the Media Server via 10G dedicated LAN.

- CIFS shared volume copied via NDMP or Accelerator.

In order to get deduplication we need to go through the Media Server, and that's why we are using the 10G dedicated link (for remote NDMP or Accelerator).

NDMP backups work great, and the first Accelerator backup has been performed with acceptable results (I know the first one is always slow). The problem comes with the next "Accelerator enabled" backups, accelerator is not used and the detailed status of the job states the following:

not using change journal data for <%%s>: not supported for non-local volumes / file systems

How can we avoid this? Is there any way to make the Media Server "think" the CIFS share is local?

Thanks in advance.

Regards.

0
Login to vote
josemiguel_casado's picture

Hi AbdulRasheed,

 

could you please answer my questions.

 

Thanks.

0
Login to vote
AbdulRasheed's picture

Hi Josemiguel, 

Sorry, I had been travelling. 

The Windows Change Journal is applicable only for NTFS file systems. You need to disable the use of change journal to protect NFS/CIFS file systems. 

Warm regards,

Abdul "Rasheed" Rasheed

Tweet me @AbdulRasheed127

0
Login to vote
Enrique Pereira Calvo's picture

Hi Abdul,

Thanks for the post. I have used it several times to understand Accelerator. I have successfuly implemented it at several customers, some with cifs and some with nfs and it works great.

But at this moment, I am ugprading a 7.0 environment and switching from NDMP to CIFS/accelerator and it is not working. Hope it is still supported on 7.6.0.2. 

I can see the following on the job log:

not using change journal data for <\\server\share>: not supported for non-local volumes / file systems

and later:

not using change journal data for <\\server\share>: forcing rescan, each file will be read in order to validate checksums

The proxy server (the media server) does not have selected "use change journal" selected.

Also, it ended working, as it get 98.9% of optimization, but it re-read all the source files :-(

accelerator sent 47310848 bytes out of 4226926592 bytes to server, optimization 98.9% 

 

Regards,

0
Login to vote
Enrique Pereira Calvo's picture

Hi Abdul,

It seems that starting with 7.5.0.7, there is a requirement to have another schedule with "Force rescan" checked. If you have ONLY one policy with no "force rescan" checked, it always do a force rescan.

But if you have a second policy with FR checked, even if it does not have an open schedule the first policy works as expected.

Regards,

+1
Login to vote
Scott_123's picture

Enrique, the following describes what you're seeing and what changed to cause this.

http://www.symantec.com/docs/TECH215608

+1
Login to vote
Enrique Pereira Calvo's picture

@Scott_123,

 

That is exactly what it is happening. Thanks a lot :-)

 

Regards,

0
Login to vote
SJ Hollist's picture

Doesn't Accelerator require using a compatible disk target, so the use of Tape is no longer available if you switch from direct NDMP to Accelerator? This seems like another barrier to switching to Accelerator over NFS/CIFS. Getting to tape would then require a duplication process, correct?

- SJ Hollist

0
Login to vote
Enrique Pereira Calvo's picture

Yes, you need to use accelerator with compatible disk devices and then duplicate to tape. But, depending on the ndmp speed, it will be faster to stage first to disk.

0
Login to vote
SJ Hollist's picture

Also, is there any 3rd party OST support for Accelerator?

- SJ Hollist

0
Login to vote
Enrique Pereira Calvo's picture

You need to see the compatibility of the ost device. Data domain supports accelerator but no vmware accelerator for example.

Quantum DXi does not support any accelerator.

0
Login to vote
wilsonchong's picture

Recently we are implementing CIFS with Accelerator for NBU 7.6.0.2, the NAS is NetApp. We already followed the technote to create a schedule with "Force Rescan" option and run the first full backup.

But the subsequent incremental or full backup still take almost the same amount of time to do the backup. We don't see any improvement for the backup time. Is there any area I need to look into. I thought for the accelerator is just enable in the policy.

We also tried to disable the Antivirus scanning during the backup, but it not help alsosad

Hope can get some advise from here. A technical case already open but also no good news.

0
Login to vote
Enrique Pereira Calvo's picture

It is faster when you use NFS. I have seen poor performance doing backups with cifs. If you can you should test it and see the difference.

The only customer that I have it running with cifs is a customer with a VNX5300 and 100 disks and SSD cache.

You can see if accelerator is working reviewing the job log.

0
Login to vote