Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Netbackup-compression of backup on tape (h/w level)

Created: 03 Sep 2012 | 31 comments

hello all

We regularly take backup for SAP systems, recently i faced some abnormal behaviour..

The policies configured 1 year ago are able to compress backup h/w level(tape drive level), but when i configure new policy with all parameters same... it is not able to compress

old policies can write 1.5 Tb data on single tape (IBM LTO 4 tape 800/1600Gb) , new policy is barely able to write 800Gb.

what is wrong with me ... even if i copy old policy to create new ..it wont compress.

what is the issue and the solution if any one have solved this issue....

on old policies software level compression is not enabled and still it compesses perfectly but new r not..???

Please Help!!!!!

Comments 31 CommentsJump to latest comment

Nicolai's picture

A policy can't control compression on the tape drive level. But there are options that will prevent the tape drive from compressing data - that's the client side compression and encryption options. Please check those policy settings.

Please also check that in fact the tapes are LTO4. If LTO3 are used in LTO4 tape drives, they will be written in LTO3 format with maximum capacity of 400 (800 compressed).

Assumption is the mother of all mess ups.

If this post answered your'e qustion -  Please mark as a soloution.

Yasuhisa Ishikawa's picture

Are there any difference between data to be backed up?
Compression ratio will become very low with some type of data (e.g. already compressed files like zip, gz, jpg, mpeg,....)

Authorized Symantec Consultant(ASC) Data Protection in Tokyo, Japan

paroksha's picture

@ Nicolai

What are those client side settings ?? can u share any doc if possible..

and I am sure that I am using LTO4 tapes same as used with old policy there is no difference in tapes...

@ Yasuhisa

It is SAP application data archive which is same everywhere in my DC.

Is there any other option ????

 

 

Nicolai's picture

Open the policy in qustion - check if the "Compression" or "Encryption" box is checkmarked.

Is the write speed of the tape drives affected as well ?

Assumption is the mother of all mess ups.

If this post answered your'e qustion -  Please mark as a soloution.

Mark_Solutions's picture

As the others have said - H/W compression is out of the hands of NetBackup and will be done automatically at the drive firmware level.

If you enable NBU encryption or compression then this prevents the drive compressing it at the firmware level

So any form of encryption or compression implemented prior to the data arriving at the tape drive will block H/W compresion

The only places this is set is on the Policy Attributes tab

Just one other thought ... is the NetBackup Key Management Service runnign on your Master?

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

paroksha's picture

Hello all

thanx for quick reply

There is no encryption or compression implemented in any policy

but how come some policies are able to write data in h/w level compression and other not

As i said i cross verified all the options used to create policies and also copied old policies to create new one so there is no chance that i have missed any settings..

can netbackup support will help me???? and if i want to try how and where???

let me know the contact info n mail ID to raise case for the same..

and also if u have any solution i can try that toooo

 

 

Mark_Solutions's picture

Go to http://www.symantec.com/business/support/index?page=home - click on the Contact Support on the right - get help by phone - choose your country

You may find that some systems have compressed file systems or encrypted file systems (forget the name of it now but Windows has new features for that) which slows thinsg down and reduces compression

Either that or you have a bad tape drive?

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

RonCaplinger's picture

Is this only happening on SAP backups?  If so, check the settings on your database and ensure encryption and compression have not ben enabled there.

Are all of the tape drives LTO4?  If they are, be sure you have disabled compression option in the NetBackup policy.  The LTO4 drives should (at least in our case) handle the compression automatically.  If Netbackup compresses the data first, the drive cannot compress further.

 

paroksha's picture

I have tried with new and several tapes also still facing same issue..

all are LTO4 in SUN SL48 2-drives ...

HERE WHAT HAPPENING

I tried same data with old and new policies to write on same tape

with old policy it is compressing aroud 1.5 Tb but with new created policy it is not 800Gb this is the BIG JOKE... same data same tape-drive only diff is policy creation.

 

even i tried and created multiple policies for diff data it is not compressing

do I need to apply any patch ar update here ...

if u want any cmd uptput please post the full command

mph999's picture

Can you post up ...

/usr/openv/netbackup/db/class/<working policy>/info

/usr/openv/netbackup/db/class/<non working policy>/info

Also, what happens if you 'copy to new the working policy to create a new one.  Doe the compression work then ?

Martin

 

Regards,  Martin
 
Setting Logs in NetBackup:
http://www.symantec.com/docs/TECH75805
 
Mark_Solutions's picture

Sounds very much like a corrupted policy or one that has had encryption or compression enabled and then disabled but the change has not been updated somewhere leaving it as it was, even though you cannot see that in the GUI

You could run "nbpemreq -updatepolicies" to get the system to read-read all polices or just create a new one and discard the old one.

As Martin says - the info files may also hold the key.

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

paroksha's picture

Working policy: monday

 

root@BackupServer #
root@BackupServer # more /usr/openv/netbackup/db/class/monday/info
CLIENT_TYPE 0
FOLLOW_NFS_MOUNTS 0
CROSS_MOUNT_POINTS 0
CLIENT_COMPRESS 0
PRIORITY 1
CLIENT_ENCRYPT 0
DISASTER_RECOVERY 0
MAX_JOBS_PER_CLASS 2147483647
MAX_FRAG_SIZE 0
ACTIVE 0
EFFECTIVE_TIME 1291635000
COLLECT_TIR_INFO 0
EXTENDED_SECURITY_INFO 0
IND_FILE_RESTORE_FROM_RAW 0
BLOCK_INCREMENTAL 0
STREAMING 0
FROZEN_IMAGE 0
BACKUP_COPY 0
NUMBER_OF_COPIES 1
FAIL_ON_ERROR 0
CHECKPOINT 0
CHKPT_INTERVAL 0
ENABLE_PFI 0
OFFHOST_BACKUP 0
USE_ALT_CLIENT 0
USE_DATA_MOVER 0
DATA_MOVER_TYPE 2
COLLECT_BMR_INFO 0
RES_IS_SS 0
GRANULAR_RESTORE_INFO 0
JOB_SUBTYPE 0
USE_VIRTUAL_MACHINE 0
GENERATION 52
CLASS_ID 5EF877A61DD211B2B3E400144F8CBB44
RESIDENCE BackupServer-hcart-robot-tld-0 *NULL* *NULL* *NULL* *NULL* *NULL* *NULL* *NULL* *NULL* *NULL*
POOL backup_STG_monday NetBackup NetBackup NetBackup NetBackup NetBackup NetBackup NetBackup NetBackup NetBackup
SHARE_GROUP *ANY*
DATA_CLASS *NULL*
#VMD5_DIGEST=11 ec 01 21 88 6c 46 87 4d 8a 80 48 5f c5 aa 03
root@BackupServer #
____ NON WORKING policy:backup2_QER_QBI
root@BackupServer #
root@BackupServer # more /usr/openv/netbackup/db/class/backup2_QER_QBI/info
CLIENT_TYPE 0
FOLLOW_NFS_MOUNTS 0
CROSS_MOUNT_POINTS 0
CLIENT_COMPRESS 0
PRIORITY 1
CLIENT_ENCRYPT 0
DISASTER_RECOVERY 0
MAX_JOBS_PER_CLASS 2147483647
MAX_FRAG_SIZE 0
ACTIVE 0
EFFECTIVE_TIME 1314264342
COLLECT_TIR_INFO 2
EXTENDED_SECURITY_INFO 0
IND_FILE_RESTORE_FROM_RAW 0
BLOCK_INCREMENTAL 0
STREAMING 0
FROZEN_IMAGE 0
BACKUP_COPY 0
NUMBER_OF_COPIES 1
FAIL_ON_ERROR 0
CHECKPOINT 0
CHKPT_INTERVAL 0
ENABLE_PFI 0
OFFHOST_BACKUP 0
USE_ALT_CLIENT 0
USE_DATA_MOVER 0
DATA_MOVER_TYPE 2
COLLECT_BMR_INFO 1
RES_IS_SS 0
GRANULAR_RESTORE_INFO 0
JOB_SUBTYPE 0
USE_VIRTUAL_MACHINE 0
GENERATION 66
CLASS_ID 343849061DD211B29A3900144F8CBB44
RESIDENCE BackupServer-hcart-robot-tld-0 *NULL* *NULL* *NULL* *NULL* *NULL* *NULL* *NULL* *NULL* *NULL*
POOL backup2_QER_QBI NetBackup NetBackup NetBackup NetBackup NetBackup NetBackup NetBackup NetBackup NetBackup
SHARE_GROUP *ANY*
DATA_CLASS *NULL*
#VMD5_DIGEST=0a f7 0c fe 92 6d 1e 6c 48 b8 a4 20 9a 12 e4 09
root@BackupServer #
 

as I already said copying working policy to create new also not able to compress(not working).....

Mark_Solutions's picture

dont copy it - the corruption will follow - make a new one or try the nbpemreq command

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

paroksha's picture

how do u come to know that there is corruption and imp how to avoid it from happening.

what nbpemreq command does .. any option with this cmd???

any precaution should be taken before execution of this cmd.. cause it is PROD and required policies are compressing properly, i dont wanna disturb that ...

surely i m looking for solution here but with all precaution...

 

Nicolai's picture

From media management can you take a picture showing tapes that compress / do not compress. It important we get as much details as possible.

How do you determine that the tape only can hold 800GB ?

Assumption is the mother of all mess ups.

If this post answered your'e qustion -  Please mark as a soloution.

paroksha's picture

as u can see backup_STG_sunday / monday...... can write upto 1.5 Tb but while backup1-development has taken 2 tapes first tape filled .....

u can see the amount of data written is different in each policy while the data type is same for all policies..

backup_STG_sunday / monday..... are old policies others are new .

some can write upto 1 Tb also but lot policies are around 800 Gb only...

Nicolai's picture

Are tape "backup1-development" status full ?. If the status is not full, there is still free space on them.

Are backup1-development and backup_STG* the same type of data ?

Have you tried to write backup_STG* to  backup1-development ?

Could you also (physical) verify that both tape and drive really are LTO4. 

  • 1.2TB on LTO3 is very often seen while backing up databases.
  • Around 1TB per tape using unstructured data (file,mail, etc etc)

 The amount stored on the tape really look like LTO3. 

Assumption is the mother of all mess ups.

If this post answered your'e qustion -  Please mark as a soloution.

paroksha's picture

As u can see from above 3 images backup1-development policy CI0081 L4 n 012349 L4 used in same write... please see 2nd image...

Yes All the tapes are LTO4 I am the one who handles all tapes and we have recently purchased IBM LTO4 tapes...

if u want me to try some workaround /options please let me know(same as backup_STG * using backup-dev policy...

I will try that this weekend..)

 

Mark_Solutions's picture

Please give us the full screen shot of the media so that we can see the Status of the tapes involved to see that they are all Full (or active or suspended)

Thanks

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

Mark_Solutions's picture

I still cannot see the column titled "Media Status" in that screen shot.

Right click the top row and select Columns - Layout

Select Media Status and set it to "show" then move it up so that it is next to the Kilobytes column and screen shot it again

Thanks

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

Mark_Solutions's picture

Sorry - that picture is so shrunk i cannot make out the figures

I see that there are 4 tape shown as Full - what are the kb figures for those?

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

Mark_Solutions's picture

OK - so the full tapes range from 1.1TB down to 850GB - so not great.

As this is simply a difference between identical policies then that is where you need to look as we have said before

the nbpemreq --updatepolicies just make NBU re-read then all to make sure they and all settings are up to date - so no worries running this any time.

I would still be inclined to start from scratch with a new policy and try that one though.

I am curious though - why do you have 2 identical policies anyway?

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

paroksha's picture

hi 

i dont want identical policies, rather i want compression of data.

i want my all policies should compress data upto 1.5/1.6 Tb on single tape(H/W level), my old policies are able to do that but new policies are not able to compress even tapes are similar , data is similar ...

 

Will Restore's picture

noticed the "non-working" policy contains

COLLECT_TIR_INFO 2
COLLECT_BMR_INFO 1
 
where the "working" policy has
 
COLLECT_TIR_INFO 0
COLLECT_BMR_INFO 0

 

Will Restore -- where there is a Will there is a way

paroksha's picture

the other working policies has thos settings 

************************************************************

 

root@BackupServer # more info
CLIENT_TYPE 0
FOLLOW_NFS_MOUNTS 0
CROSS_MOUNT_POINTS 0
CLIENT_COMPRESS 0
PRIORITY 1
CLIENT_ENCRYPT 0
DISASTER_RECOVERY 0
MAX_JOBS_PER_CLASS 2147483647
MAX_FRAG_SIZE 0
ACTIVE 0
EFFECTIVE_TIME 1291721400
COLLECT_TIR_INFO 2
EXTENDED_SECURITY_INFO 0
IND_FILE_RESTORE_FROM_RAW 0
BLOCK_INCREMENTAL 0
STREAMING 0
FROZEN_IMAGE 0
BACKUP_COPY 0
NUMBER_OF_COPIES 1
FAIL_ON_ERROR 0
CHECKPOINT 0
CHKPT_INTERVAL 0
ENABLE_PFI 0
OFFHOST_BACKUP 0
USE_ALT_CLIENT 0
USE_DATA_MOVER 0
DATA_MOVER_TYPE 2
COLLECT_BMR_INFO 1
RES_IS_SS 0
GRANULAR_RESTORE_INFO 0
JOB_SUBTYPE 0
USE_VIRTUAL_MACHINE 0
GENERATION 47
CLASS_ID 00B713261DD211B2BB4900144F8CBB44
RESIDENCE BackupServer-hcart-robot-tld-0 *NULL* *NULL* *NULL* *NULL* *NULL*
 *NULL* *NULL* *NULL* *NULL*
POOL backup_STG_tuesday NetBackup NetBackup NetBackup NetBackup NetBackup N
etBackup NetBackup NetBackup NetBackup
SHARE_GROUP *ANY*
DATA_CLASS *NULL*
#VMD5_DIGEST=a8 22 ce a8 ba 74 5b 75 aa ac bf db dd d7 07 e5
root@BackupServer # pwd
/usr/openv/netbackup/db/class/tuesday
root@BackupServer #
 
************************************************************
please see the media it is still in active state and finished writing 1.2 Tb
backup_STG_tuesday is working working same as monday..
my original question is how older policies are able to write 1.5Tb and not new created..
now pl ans following :
1.. does NFS mount point makes any difference
2.. does netbackup anyhow can control drive level compression: (as same drive is been used and compressing properly) 
3... nbpemreq --updatepolicies command will make any changes in older policies, these are PROD backup policies.. other i m trying are DEV and Quality .
 
 

 

Mark_Solutions's picture

You say your old policies and you new policies - this is what is confusing me here ...

Why, if you have working old policies are you creating new ones ....

You say they are identical but if you are making new policies it suggests that you are backing up different clients or different data - so this is where the differences may come from .....

We need to know what a "good" policy backs up - client name, O/S, backup and data type and then the same for the "bad" policy.

I hope you see my point .... if you have a working policy then why do you need to make a new one? ... i suspect that the new one is a different client / data location and so that is where you should be looking for data difference

 

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

paroksha's picture

Hmm I Know what you want to know....

see my old policies are backing up PROD SAP data..

i created new policies to backup development & quality servers SAP data..

the data type is same and yes the clients are different offcourse......

other things are same like OS ... type etc all are SOlaris OS .

 

Mark_Solutions's picture

If that is the case then you need to look at the SAP data on the dev servers - i think the script used was mentioned earlier as well - the cause will be on the client, not the policy

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.