An Open Letter to IBM TSM Customers from Symantec NetBackup

This open letter has come at the exact time i am looking at options for migration of a clients backup infrastructure from TSM to NetBackup.  There are 900+ Servers, with a large amount of data being backed up to around 2600 sdlt-s4 tapes.  I am curious as to the migration path from TSM and the possibilities of a slick way of performing media migration, other than restore from tsm and then backup that data via netbackup.

There isn't a direct TSM catalog to NBU catalog conversion.  I know companies have tried that in the past but with data formats changing in each new TSM release it would be hard to keep up.

A couple of other options (rather than restore and re-backup all that data):

1. What is the retention/expiration date for those 2600 tapes? Do all of them need to be converted?  If not, stand up a NBU environment and after a weekly or monthly cycle expire the tapes not needed and then just convert the monthly/quarterly/yearly backups that TSM has.

2. Consider something like Enterprise Vault if you can archive a portion of those 900 servers to reduce the backup footprint

3. Dedupe.  If there is a large amount of redundancy across those 900 servers you may be able to get away with a smaller footprint in a new NBU environment by doing client or sever side backups in NBU 7.  Doesn't help with the conversion but combined with #1 you might be able to run TSM and NBU concurrently until the retention is up on the TSM tapes.

Adding to what Joe said, here's some notes to consider..

Typically a customer would keep a TSM server around for legacy restores, but there are companies that offer to do a conversion where they restore the data and re-back it up with NetBackup. If you contact your Account Manager here at Symantec they will be able to talk through the options with you.

Thanks!

Dear Swathi.

I have been a TSM Customer for over 15 years and I think there some sort of misunderstanding going around here.

You said:

TSM customers who need to improve recovery performance

NetBackup can restore files, emails and other granular items from applications and virtual machines from a single-pass backup. TSM requires the need for two backups passes, which can take additional time and storage, and TSM’s progressive incremental backup strategy can cause data to be dispersed over a large number of tapes, which can lead to performance problems during restores.

TSM can restore files, emails, and other granular items from applications from a single-pass backup through the use of TSM for Fastback  for instance.

Secondly the primary cause for performance problems during tape file-restores is the technological limitations of the tape technology and the application using the tapes has nothing to do with this problem. Having files dispersed over large number of tapes can cause for performance problems but only in environments with not enough mount points, with sufficient mount points having fragmented data can outperform a single full tape backup on one stream. Obviously if you do not have sufficient mount points (tape drives) you should try to avoid having fragmented data on tapes if you have a high RTO objective on the system in question.

You Said:

TSM customers wanting to implement deduplication

NetBackup offers built-in source and target deduplication, and deep integration with deduplication appliances. TSM recommends you make a non-deduplicated copy of the data before enabling deduplication, and requires additional “hydrated storage” to support its reclamation process.² This can significantly increase the required storage footprint instead of delivering the benefits of deduplication technology.

There is a reason for why TSM recommends you make a non-deduplicated copy of the data. All deduplication technologies use some sort of hashing algorithm, and the use of those algorithms have a change of making a false positive chunk that could result in the loss of data and making data unrecoverable. TSM uses the SHA-1 which causes the 50% Change of one collision in a 40PB disk storage using 4kb chunks 0.5*10-28. The chances are slim but regardless TSM informs you about the chance and allows its users to make the decision themselves.  A sales person who claims that his backup product is 100% secure deduplication technology is probably someone who you should not trust. And I think all backup providers who have embedded deduplication in their product should inform their customers about what kind of fingerprinting and hashing algorithm is being used and the chances of collision.  Better to be safe than sorry if you ask me.

The point of better recovery performance in NetBackup is not that TSM can’t do recovery just that it requires more infrastructure (storage, tape drives, add-on products etc) to do so.  Tape does inherently have challenges like you pointed out and they can be solved by more infrastructure but features in NBU like single pass backup are created to prevent people from needing to buy more infrastructure.  And single-pass backups are just a checkbox option in the same policy UI used for all other NBU backups.  Fastback is a different backup process and a different product, not just a single checkbox.  And when it comes to Vmware single-pass NBU uses FlashBackup to map the blocks in a VMDK to the files without using an agent inside the guest.  Symantec has actually filed for patents on that technology so while other products can claim a single-pass it usually results in an agent in the guest or some sort of post-backup mount of the VM to index the files.  The goal for NBU in general is to do the quickest recovery with the least amount of infrastructure.  TSM is a good product and many admins love it but when it comes to the amount of infrastructure it takes to run it properly most finance people get tired of writing checks for the hardware.

For deduplication we’re striving to give users a large range of choices for where they can dedupe but still have global deduplication across all those locations.  NetBackup 7 can now dedupe files across client-side, server-side and if you don’t want to use Symantec for dedupe we have integration through OpenStorage which supports DataDomain, FalconStor etc without treating them like tape (VTL).  The amount of choices and flexibility for dedupe in NBU is more than anything TSM has.  The hydration point is not that TSM is less secure in its dedupe algorithm (we use 256-bit blow fish which has the same risk of collisions like SHA) just that the TSM architecture is fundamentally different with the incremental-forever approach.  Like you mentioned, if the infrastructure is in place for hydration and reclamation it can work pretty well.  But hydration and reclamation are things that NBU simple doesn’t have to do.  In fact, with dedupe integrated into NBU 7 we’re finding that users are eliminating their weekly/monthly fulls and just creating synthetics by moving dedupe pointers around and creating a new full from the incrementals before going to tape.  It’s like reclamation but without any data movement. The data is rehydrated directly in the storage pool without needing a dedicated hydration space.  It's for these reasons (and others) that we’ve been seeing a lot more migrations from TSM to NBU than we ever have in the past.

I fail to see how you can reach the conclusion that TSM requires more infrastructure than NBU, It is true like you said that if we have a tape based solution, having fragmented data is going to either make for a slow restore or require more hardware. That is going to be a fact regardless of what kind of backup application you are going to use (for fragmented data).  Making a single-pass tape restore can be done just as easy in TSM by changing one attribute in the storage pool configuration (Collocation).  But using that will require more tapes for your solution.

 "There is no such thing as a free lunch“ when you want a better RTO time one of two things are going to happen, your solution is going to be more costly or it´s going to become less secure.  If you want a lower TCO it is going to bite you in the RTO time or reliability.

TSM architecture is fundamentally different like Joe pointed out, The Main objective of TSM´s Progressive Incremental forever is to store only data that you potentially are going to be needed for restore, not piling up data in multiple full backups. The objective is not to keep multiple tape copies of weekly/monthly backups of your entire environment when only a small fracture of that data is going to have any chance of ever being restored.  

But that being said, this debate is an dying argument, as customers move towards different backup architecture of a disk-based solution with tape only being used as a back end storage or large sequential data that actually suite the Tape technology very  well.

Fastback was a different product like Joe points out but it is now embedded within the TSM portfolio, and integrates very well with the classical TSM servers, it is no different than any other value add-on that customers can use.

Regarding the deduplication quotes:

"[...]TSM recommends you make a non-deduplicated copy of the data before enabling deduplication, and requires additional “hydrated storage” to support its reclamation process.² This can significantly increase the required storage footprint instead of delivering the benefits of deduplication technology.“

Like I have mentioned earlier the non-deduplication option is a redundancy issue and of course it will add to the infrastructure cost if you decide to go that way. Talking about this as some sort of shortcoming when comparing against NBU is not a fair comparison, especially when you don’t require the same level of redundancy from both applications.

The "Hydrated storage“ is a minor because TSM only uses the empty space it has assigned for the deduplicated storage to reclaim itself, if you have no hydrated storage you have no space left to put new data in it, regardless of what kind of deduplication storage technology you decide on you are always going to need some empty space for new data, TSM simply uses that space for reclamation.

Sometimes when a backup provider is comparing it´s solution against his competitors and want to emphasize how there solution has a better RTO objective they compare a high level, high cost solution with great RTO against a cheap solution with horrible RTO from the competitor. When they want to show how costly the competitor other solution is they show a high level RTO competitors solution against a cheap one with low RTO.  I have seen this done multiple times from all sides.

I beleive that TSM customers who DO need to improve recovery performance will definitely switch to NetBackup. And not only because this article says so - just naturally selecting one of the best backup and recovery product on the market.