Video Screencast Help
Symantec Appoints Michael A. Brown CEO. Learn more.

The effect of -compress flag in bpplinfo command

Created: 10 Dec 2010 • Updated: 15 Dec 2010 | 4 comments
This issue has been solved. See solution.

Hi All,

When we are taking backup of some data we can specify the status of -compress flag in bpplinfo command , so that backup data can be stored after compression.

Now suppose if we are using some third party smart storage solutions ( which provide their OST plugin to work with NetBackup) then does compression happens at Netbackup side or does compression happens at actual storage side ( i.e the client software).

I think Netbackup do not compresses data by itself , rather if -compress flag is set to 1 , then  it specifies to OST plugin that data compression is being requested by user and storage software has to perform compression.

But , I am a bit confused. please help me!!!

Comments 4 CommentsJump to latest comment

Murray Butler's picture

Hello,

When using OST to do compressed backups, you can compress the data with NetBackup, we will use a compression algorithm for the data as it flows to the storage device. The interesting facet to OST is that nearly all the OpenStorage partners that create an OpenStorage plugin have deduplicaiton enabled int heir devices. If you are interested in compression, the main reason would be to avoid the use of deduplicaiton on the image.
Deduplication will reduce your storage disk usage and allow for much greater logical capacity of the storage, dependent upon the type of data and it's dedup-ability.
When you supply compressed (or encrypted) backup images to a deduplicating device, the deduplication is actually hindered somewhat and is not as effective at finding duplcate areas and reducing your storage usage.
There are cases when this is desireable: when you do not wish to use deduplication and are wary of keeping only one copy of a given data segment, when you NEED an encrypted backup such that you do not want any question about the data within the backup being visible or re-constructable to others with rights to that storage volume.
I am not aware of a vendor that is compressing data in the OpenStorage plugin at this point.
In summary, NBU will compress for OST, just as it will for tape, some of the OST partners have compression methodolgies for their devices and disk volumes, but NBU does not manage those for them, there is not a vendor (that I am aware of) that is compressing in the plugin at this point, and with deduplication, compression is actually a weight on the system, more than a relief.

Thanks,

MB

SOLUTION
pravar_12's picture

Hi Murry,

Thanks for your quick response. I have one additional query -

Is there any way through which compressed data size ( actually written data to OST plugin ) can be determinded at NBU side, so that we can measure how much capacity is saved using NBU's compression only?

Murray Butler's picture

Hi Pravar,

I don't believe that we have that number available to users, it may be calculated at some point in NBU, but I don't believe that we total it. Our capacity numbers are typically populated on a once per minute poll of the storage servers disk volumes. This then calculates physical and logical space used.
If you were developing a plugin, this info is (obviously) available to you, but if you are outside of that part of the development, the plugin appears as a loaded library to NBU and it would be very difficult to intercept the data stream and find statistics on it.
You could, if you wanted to, find out by writing the backup using compression to a non-deduplicating media (like Basic or Advanced Disk), and then writing the same backup to a deduplicating device and calculating the space utilization difference. That would give you a feel for the amount of data written and what dedup buys you with compression.

Thanks,

MB

pravar_12's picture

Hi Murry,

Thanks a lot for explaining in such a detail.

Now , If NBU does compression of data , it would certainly be visible to us on storage side ( i mean on actual hardware) , so , i  can see the effect of compression by NBU on storage side.

Am i Correct??