Video Screencast Help

NetBackup 7.1 Features - Auto Image Replication

Created: 15 Mar 2011 • Updated: 22 Jan 2013 • 23 comments
Dave High's picture
+7 7 Votes
Login to vote

Welcome to NetBackup 7.1! This series of blogs will provide additional information into the key features of NetBackup 7.1 continuing today with Auto Image Replication (AIR).

The growth in the use of disk storage (particularly deduplicating disk storage) for backup has highlighted a limitation of disk storage when it comes to disaster recovery. Tapes can easily be sent offsite to be used for restores after a site outage. Disk doesn’t offer this flexibility. Some OpenStorage technologies, including the PureDisk Deduplication Option, support “out of band” replication where the contents of the disk storage is replicated between devices – however NetBackup has no knowledge of the data which makes recovery in a NetBackup protected environment more difficult. Using this approach the replicated data can only be accessed after recreating the NetBackup catalog – which is done by importing the entire content of the disk storage at the remote site using the bpimport command. The nbcatsync utility, introduced in NetBackup 6.5.6 and 7.0.1 can address this challenge it relies on being able to restore the catalog from a catalog backup and then “post-process” it to reconcile the disk device mappings. While faster than importing the contents of the storage, this is still a time consuming process.

This has all changed with the introduction of AIR. The use of Storage Lifecycle Policies (SLP) and optimized duplication between disk pools allows disk based backups to be replicated between devices under NetBackup control. Prior to NetBackup 7.1 this process has been limited to the NetBackup master server from which they originated. With NetBackup 7.1 site to site disaster recovery – whether the site is two rooms in the same data center or separate data centers across the country - can now be done using two separate Masters and the process is automatic once the SLP has been configured. Obviously the greater the separation between the sites the greater the type and severity of disaster that can be protected against.

The Auto Image Replication (AIR) feature introduced in NetBackup 7.1 addresses the site to site replication challenge by allowing Storage Lifecycle Policies to duplicate selected images between NetBackup Master domains. Backups are first written to a Media Server Deduplication Pool (MSDP) using a suitably configured SLP and then duplicated to a remote target NetBackup domain. An SLP in the target domain logs the duplicated copy in the catalog of the target domain and it is held there on MSDP disk based on a retention policy set in the source SLP.  Remember, this is a “duplication” process since the data is already “deduplicated” by using the MSDP.

The retention of the data at the target site is configured from the source site. This SLP can also be configured to duplicate imported data to other types of storage at the remote site, and then expire it from the disk it was duplicated to. For example, once the data is duplicated to the remote site, it can then be copied to a tape drive at the remote site for long term storage that NetBackup is “aware of”. This process could then free up space for additional replication from the source domain. At this time, the destination retention at the Target site is a fixed time period, however support for the “expire after duplication” and “capacity managed” retention options will be included in a future release.

The primary purpose of Auto Image Replication is to create off-site copies of mission critical backups to protect against site loss. It is not intended to extend the storage capacity of a backup domain by allowing backups to be stored in a separate domain, nor is the intention to provide for day to day restores of data. Due to WAN bandwidth restrictions between sites, typically only the most critical data should be chosen for duplication using AIR. Electronic off-siting in this manner allows the backup set to be duplicated to an off-site location as soon as the backup has completed at the primary site without the need for user intervention based on the configuration of the SLP. It also means that the duplicate copy is available at the disaster recovery site as soon as the duplication has completed. This offers a significant advantage over traditional tape based off-siting in which backups must be duplicated to tapes that are then removed from the tape library and shipped to the disaster recovery site, a time consuming and costly activity with a risk that the tapes could be lost or stolen in transit.

Underlying Principles

In order to use Auto Image Replication suitable disk storage devices must be configured in the source and target domains. In the NetBackup 7.1 release the Auto Image Replication feature only supports the Media Server Deduplication Option (MSDP) to duplicate backups between domains. Support for OpenStorage devices and PureDisk Deduplication Option are expected to follow in a future release.

How does it work? Auto Image Replication works by duplicating backups from a disk pool in the source domain to a disk pool in the target domain. The replication operation requires two SLPs, one in the source domain and one in the target domain, both of which must have the same name. The SLP in the source domain is associated with the backup policy and controls the writing of backup and the subsequent duplication to the target domain. The SLP in the target domain is not associated with a backup policy but is invoked by an alerting mechanism when a new image (duplicated from the source domain) is detected. This SLP runs the process to add the information about the backup to the target domain and can also be configured to duplicate the backup to other storage locations in the target domain.

An Auto Image Replication backup image is different to a normal NetBackup backup image because once the backup has completed the image database information associated with the backup (the part of the NetBackup catalog that lists what the backup actually contains) is appended to the end of the backup image before it is duplicated to the target domain. When a new backup is detected in the target domain this information is read from the backup and used to populate the target domain’s NetBackup catalog. This information only exists in the source domain copies and the initial copy in the target domain and is not included in any subsequent duplicate copies created in the target domain.

To help understand the process, refer to the following diagram and the numbered notes following the diagram.

The figure shows the sequence of events in an Auto Image Replication backup and duplication operation broken down into 4 discrete steps:

  • Step 1 – The backup is written to disk storage in the source domain using a backup policy with an SLP configured for Auto Image Replication. When the backup completes the catalog data it generates is appended to the end of the backup.
  • Step 2 – the backup is duplicated to the target domain across the WAN (or LAN)
  • Step 3 – the storage device in the target domain alerts the target master server to the fact that a backup has been duplicated to it. This triggers the receiving SLP to run a “fast import” operation in which the catalog data transferred from the source domain is added to the target domain’s catalog.
  • Step 4 – the receiving SLP in the target domain can now duplicate the received backup to any desired location for storage – such as creating a tape for long term retention.

Hopefully this information on Auto Image Replication has been helpful. Join us tomorrow for a look at another major NetBackup 7.1 feature – OpsCenter Reporting Enhancements including Deployment Analyzer for calculating Front End Terabyte (FETB) in existing NetBackup environments. 

Comments 23 CommentsJump to latest comment

wrxguy's picture

 

Hi Dave, excellent article. Very helpful.

I just would like to clarify something.

In your article you are saying

[guote]In the NetBackup 7.1 release the Auto Image Replication feature only supports the Media Server Deduplication Option (MSDP) to duplicate backups between domains. Support for OpenStorage devices and PureDisk Deduplication Option are expected to follow in a future release.[/quote]

then I read the following in another official Symantec document

[quote]The NetBackup OpenStorage Feature now includes AIR, which automatically manages 

the replication of a deduplicated backup image from one NetBackup master server 

domain to another. [/quote]

The key word in both quote is "OpenStorage". Is there any discrepancy, or it's just my misunderstanding of the license. 

Could you clarify what license do I need in order to use this feature?

My guess is I need two "Enterprise Disk" licenses (one for each site). Is that correct?

Thanks

+6
Login to vote
AbdulRasheed's picture

OpenStorage is an API feature where partners can integrate their storage solutions (hardware or software solution) to Symantec's data protection product like NetBackup. This API now has support for AIR. That is the reason for the statement 2 you had quoted.

The partners need to develop or update their plugins to make use of AIR specific API calls. This is the reason Dave was stating about future. At the time of 7.1 release, no partner had released AIR capable plugins for their storage solutions.

Hope this clarifies the confusion.

AIR itself does not require a separate license; it is something you will be able to use with Enterprise Disk or NetBackup deduplication option license. Yes, you are right, both sites need its own license to implement AIR.

Warm regards,

Rasheed

Warm regards,

Abdul "Rasheed" Rasheed

Tweet me @AbdulRasheed127

+2
Login to vote
Mr. Topaji's picture

Hello guys,

Can we consider AIR a replacement for RealTime? If not what are the differences here?

 

Thanks,

Topaji

0
Login to vote
Dave High's picture

Hi Topaji,

AIR is not really a RealTime replacement. RTP is used to create duplicate writes of data to two destinations - so it is "Real Time" copy. AIR is used with a Storage Lifecycle Policy to replicate data that has been written to primary storage and it works "after" the data is written - so there can be a lag time (backlog) of hours or even days depending on the enviornment and bandwidth.

So in the end, you get two copies of the data, but RTP is done immedately, AIR is done when there are cycles in the infrastructure.

Hope this helps.

./dave

+1
Login to vote
Mr. Topaji's picture

It's helpful Dave, thank you, and it would be appreciated if you can post a new blog about RTP (with Diagram) for better understanding

 

Br,

Topaji

+4
Login to vote
vksingh's picture

Do we have any examples of impact on network when AIR is used? As it is duplication of deduplication data, I would assume that only the new blocks transferred across sites along with meta data for catalog and deduplication database

Some performance / load figures will really help explaining to Mgmt

+2
Login to vote
vksingh's picture

Consultants are suggesting following options to us

Virtual master - replicating thru SAN replication (EMC recover point) to second site for DR. Optimized replication of Images in deduplication pool to second site as well. Both sites are active sites instead of one being solely DR. One consultant has suggested Data Domains for deduplication and replication. Another consultant has suggest Symantec appliance (or commodity hardware DIY) for deduplication and replication

 

First I do not understand what is the difference between

"writing backups to local deduplication device via the local master servers with optimized deduplicated traffic to the remote applicance for DR and vice versa"

and AIR

Above has been suggested with Symantec appliance or DIY appliance. To me it sounds like AIR but the consultant says it is different. Would be great to have thoughts on this

 

Also would be great to get some benefits and cons of using AIR over a

1) Virtual master

2) SAN based deduplication and replication (we are considering Data domains):

As per my understanding following:

BENEFITS:

  • Symantec V  Ray vision for virtual machine backups
  • Active - Active setup: With AIR restores are immediatly available from DR site

CONS

  • Two master server / Two domains management
  • 2 master server licenses

Consultants suggested following cons as well which I do not think are true. Other thoughts would be greatly appriciated on these

  • With AIR, during DR configuration changes will be required on clients
  • With AIR, when in DR, server cannot continue to backup and lots of work will be required in configuration when recovering after DR

 

Thoughts on above would be really appriciated. Thanks

+4
Login to vote
ilovemywillow's picture

Took Advanced Unix Class for 7.0 in Dallas last week.  Instructor said we could use AIR to copy our catalogs to an alternate master for DR purposes.  We do not utilize disk for backups nor do we do deduplication or life cycle policies.   It looks like AIR isnt what we want.  Could she have been talking about a different product?  Is there anything that will duplicate our catalogs to an alternate master for DR?

+4
Login to vote
Satkay Satish's picture

Hi,

I don't think AIR is meant for replicating the entire catalog. Its just to replicate important images across different sites and then have them imported in destination NBU domain.

There are number of replication products like VVR ( Veritas Volume Replicator) that you can use for replicating data

Thanks!

Satkay Satish

-2
Login to vote
yossi.katz@simplementia.com's picture

Hi,

which data is send when using AIR between Netbackup domain?

in another articale i have read that optimized duplication is used only for duplication between MSDP servers in the same domain.

So when using AIR between different domain does the duplication is like regular duplicate image from MSDP to tape (rehydration will run)

hope some one can clarify it for me

thx in advanced

Yossi

+4
Login to vote
falti.manullang's picture

How about the speed comparison between AIR and other replication method?

Falti Manullang

 

 "If this post has helped you, please vote or mark as solution"

-2
Login to vote
Amarnath Sathishkumar's picture

Good One

Amarnath Sathishkumar

If this comment is helpfull, Don't forget to give a "Thumbs Up" or mark as "Solution"

+2
Login to vote
FujSOL's picture

Question - On an import of a backup from 1 MSDP pool to another MSDP pool, it the backup image being imported deduped?  When i look at the logs it states 0%.

I thought the whole point of MSDP was to dedupe all incoming data?

+2
Login to vote
bregmi's picture

Has anybody implemented Auto Image Replication using Data Domains? We are duplicating/replication backup images from primary NetBackup domain to DR domain and thinking of replicating catalog backup using SLP instead of VVR. What approach would be better from disaster recovery point of view?

Thanks

 

+6
Login to vote
johnnygto's picture

I really want to use AIR to replicate our backup images to DR, but we use Data Domain for storage, and EMC has not yet opened up the AIR API in OST.

Therefore, so that I don't have to buy alternate disk storage we are considering replicating our NBU catalog servers with Double-Take.

And to the point, if I did have the option of using AIR, should I switch or would I get more value in replicating the NBU catalog servers?

Thanks!

+4
Login to vote
falti.manullang's picture

If you want to replicated netbackup catalog, used Netbackup Realtime.

Falti Manullang

 

 "If this post has helped you, please vote or mark as solution"

+2
Login to vote
mohanl's picture

Hi,

I have a follow-on question.

Can the AIR target be a site with MSDP only? With mater server being wheeled and catelogue 'fast import' done as a part of the recovery time activity? Since  the appended relevant portion of first-time catalogue is already there as part of the first image with the latest subsequent image updates.

The reason for thinking on these lines are driven by financial/audit complaince factors time-line currently applicable.

Regards

+2
Login to vote
Brook Humphrey's picture

AIR is only used to send images to a remote master server. The setup alows to a media server but AIR requires a master so that the images can be imported. 

 

In your case you would just run an SLP job and backup local and then do a duplication to your remote media server. Then in a dr situation just standup a master at the remote site and import your catalog and then select the copies for the remote site for your restores and start doing your restores. 

 

Thanks

Brook Humphrey
Managed Backup Service

Principal Backup Administration Specialist

0
Login to vote
mansoor.sheik's picture

Hi. Nice Article

0
Login to vote
rizwan84tx's picture

Can i implement AIR when my PROD and DR site running with NBU 7.1 and 7.5 respectively. Does AIR support with PRD and DR domains at different NBU version or patch level?

Best Regards,

Rizwan

0
Login to vote
Brook Humphrey's picture

It is recommended to have both sites at the same patch level.

Brook Humphrey
Managed Backup Service

Principal Backup Administration Specialist

0
Login to vote
rizwan84tx's picture

Do we have this recommendation documented?

Best Regards,

Rizwan

0
Login to vote
Ajeesh's picture

Hi,

With Netbackup 7.5 or 7.6, are there any partners released AIR capable plugins for their storage solutions? Or still Auto Image Replication feature only supports the Media Server Deduplication Option (MSDP) to duplicate backups between domains.

 

Regards,

Ajeesh B

0
Login to vote