Video Screencast Help

Compress some but not all copies of a backup

Created: 05 Dec 2012 | 6 comments

ENVIRONMENT: Netbackup 7.1.04 (going to 7.5 soon). All windows environment. 1 Master server and 3 media servers (Server 2003 and 2008).

STORAGE: 40 tape library and 3 LTO3 drives.  Recently added an Exagrid Open Storage appliance (Ex7000).

PROBLEM: Prior to adding the disk based storage, we used compression on pretty much every policy. Since adding the Exagrid, we turned comrpession off, per their recommendation (I think it's related to deduplication, as well as speed, but dedup is a big factor for us). I'm using Storage Life Cycle Policies (SLP) to duplicate to tape, and as a result I'm using alot more tapes; so many, that I'm inclined to try turning it back on and see what effect it actually has on the Exagrid.

QUESTION: Is there a practical way to compression the copies on tape, while copies to Disk remain uncompressed? Or is there some setting that I'm missing?

The driver properties of the HP Ultrium LTO4 drives say that Compression is enabled already.

The only options of which I can think are (none of which I like):

  • Create different Policies for the same backups: one to Disk, another for tape with compression enabled
  • enable compression and see what effect it has on the Exagrid

Comments 6 CommentsJump to latest comment

mph999's picture

I would image, that if you try and software compress and backup to tape, the image will be bigger than if you just compress via the tape.

(Try zipping a zipped file, it will get bigger - same theory),

The drives compress at a hardware level, this is much more efficient, and quicker, than software compression.


Regards,  Martin
Setting Logs in NetBackup:
smckelvey's picture

To reiterate, the only configuration change(s) was the addition of the Exagrid applicance, and subsequently disabling compression in the policies. The setting on the Tape drives remains the same.

The effect that we are seeing is a significant increase in tape consumption, which I can only attribute to the fact that we disabled compression on the Policies.

So, my questions are:

  • Is my reasoning correct that this is the cause?
  • If it is, is there anything else that we can do? Am I missing something: another setting to enable hardware compression at the tape drive (from what I've read, that setting in the driver properties is the only factor)? Perhaps there's a setting in netbackup that will leave the copies on disk uncompressed, and compress the duplicated copies on tape?

I already mentioned the only two alternatives of which I could think in my first post:

  • enable compression in the Policy, and see what the consequences are on the Disk appliance.
  • create separate Policies for backups to Disk and backups to Tape.

And for the record, the Disk appliance is configured to retain about 1months worth of backups, which is basically all the space that we have. All backups are copied to tapes for offsite storage. And anything that outside of 1 month's retention is also duplicated to tape for onsite storage.

RonCaplinger's picture

Can you compare how much data (on average) were you able to store on a tape when using compression before?  And how much is going on a tape now? 

I've seen where SLPs run multiple duplication jobs to multiple similar tapes at once, and that seems to spread the data across more tapes at once.  Maybe they are not completely full before the tapes are suspended/ejected.

smckelvey's picture

I was going to ask the same thing (RE quantitative comparison between size of backup on tape w/ and w/o compression)....... b/c I wasn't sure a good way to quantitatively compare.

It's not a case of partially full tapes. One reason I know this is because I only remove the tapes from the library once they are "full", and that is how I can tell that we are using alot more tapes because I'm removing and replacing 50 to 100% more tapes per week in my estimation.

I'm not sure if the values in bpimagelist are based on actual size on tape, but if they are you can see exactly what I'm talking about. Compare any of the later backups to the one on 9/21, which you can see is the only one not compressed, as noted by the "Y" in the 6th column. This backup is pretty static as far as size, and it's 64% larger since we disabled compression.

I talked to Exagrid support and they said the Dedup would be none or nearly none if I enable compression, b/c the appliance would treat each backup image as a file.

I really don't think we can sustain this kind of tape consumption, at least not without buying alot more tapes. So, outside of buying some kind of new hardware for compression, the only possibility of which I can think is:

  • make duplicates of every policy: one going to Tape w/ compression enabled, one going to Disk without compression

anyone have any other ideas?

Dyneshia's picture

One more thing to mention, compression will affect restore performance.

smckelvey's picture

Duly noted, but that's not a concern as that's how we were operating before the addition of Disk, so it won't be any slower than it was. Plus our restores times are drastically improved anyways since most fall w/in the timeframe retained on disk.