Video Screencast Help

Buffer Settings for Duplication to LTO 5 Tape

Created: 31 Jan 2014 • Updated: 04 Feb 2014 | 13 comments
This issue has been solved. See solution.

We're using a Netbackup 5230 Appliance connected via 8Gb fibre to an HP MSL 8096 LTO 5 tape library with 4 drives.  I've read many posts recommending leaving the buffer size as default:

SIZE_DATA_BUFFERS                                 : 262144 (Default)
SIZE_DATA_BUFFERS_DISK                        : 262144 (Default)
SIZE_DATA_BUFFERS_FT                            : 262144 (Default)
 
This is how they are currently set, and when monitoring backups to disk, I see the following in the log:
 
01/29/2014 19:49:09 - Info bptm (pid=3752) using 262144 data buffer size
01/29/2014 19:49:09 - Info bptm (pid=3752) using 30 data buffers
 
Which is expected, however, when duplicating to tape, the logs indicate the following:
 
01/31/2014 17:00:15 - Info bptm (pid=30607) using 65536 data buffer size
01/31/2014 17:00:15 - Info bptm (pid=30607) using 30 data buffers
 
Can someone clarify why it would be set this way and is this as expected?
 
Thanks

 

Operating Systems:

Comments 13 CommentsJump to latest comment

Nicolai's picture

If the backup where made with a 64K blok size they will duplicate as 64K blocksize. New backup made with 256K will duplicate as 256K.

Can this be the case ?

By the way: Set NUMBER_DATA_BUFFERS & NUMBER_DATA_BUFFERS_RESTORE to 256 as well.

Assumption is the mother of all mess ups.

If this post answered your'e qustion -  Please mark as a soloution.

Mark_Solutions's picture

Are you using tapes that have been used in another system that did not use the 256kb block size? (or this system before you changed the DATA_BUFFER settings)?

When a tape is written using a particular block size that gets written to its tape header and whenever that tape gets used it will use the original block size.

To clear this setting you need to label the tape.

To test if this is the case just select the unassigned tapes that are ready to use for the next duplication - select to label them but uncheck the Verify box - they will then be relabelled using your current 256kb block size and should then use that from then onwards

It does mean labelling all of you tapes before they get re-used (once they are unassigned  - everything on them expired)

Worth a test

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

arctics2's picture

The buffer settings on the appliances have not been modified.  The original backup images are using the default buffer settings:

01/30/2014 18:06:41 - Info bptm (pid=27084) using 262144 data buffer size
01/30/2014 18:06:41 - Info bptm (pid=27084) using 30 data buffers
 
And when duplicated they show as:
 
01/31/2014 17:00:15 - Info bptm (pid=30607) using 65536 data buffer size
01/31/2014 17:00:15 - Info bptm (pid=30607) using 30 data buffers

All tapes being used are brand new, never been used before.  However, I will try taking the next scratch tape and manually labelling it with the verify unchecked.

 

Thanks,
Aly

 

Mark_Solutions's picture

If you have not modified the default buffer sizes you will need to do so - but as the one can affect the other make sure you do both

From the CLISH of the appliance follow this process:

Go to the Settings menu and then type:

NetBackup DataBuffers Number Show
NetBackup DataBuffers Size Show

to see what they currently are. To change them use:

NetBackup DataBuffers Number Disk 64
NetBackup DataBuffers Number Tape 64
NetBackup DataBuffers Size  Tape 262144
NetBackup DataBuffers Size  Disk 1048576

The disk setting is up to you but generally works well - as do the numbers used above

Hope this helps

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

SOLUTION
arctics2's picture

Thanks for your help.  I configured the tape size using the touch files, which seemed to work.  

The settings currently show as follows:

MASTER.Settings> NetBackup DataBuffers Number show
NUMBER_DATA_BUFFERS                               : 30 (Default)
NUMBER_DATA_BUFFERS_DISK                          : 30 (Default)
NUMBER_DATA_BUFFERS_FT                            : 16 (Default)
NUMBER_DATA_BUFFERS_RESTORE                       : 30 (Default)
 
MASTER.Settings> NetBackup DataBuffers Size Show
SIZE_DATA_BUFFERS                                 : 262144
SIZE_DATA_BUFFERS_DISK                            : 262144 (Default)
SIZE_DATA_BUFFERS_FT                              : 262144 (Default)
 
Odd that it doesn't show Tape.  
 
MASTER.Settings> NetBackup DataBuffers Number Tape 64
- [Info] Tuning parameter NUMBER_DATA_BUFFERS changed successfully
MASTER.Settings> NetBackup DataBuffers Number Disk 64
- [Info] Tuning parameter NUMBER_DATA_BUFFERS_DISK changed successfully
MASTER.Settings> NetBackup DataBuffers Size Disk 1048576
- [Info] Tuning parameter SIZE_DATA_BUFFERS_DISK changed successfully
 
New settings show as follows:
 
NUMBER_DATA_BUFFERS                               : 64
NUMBER_DATA_BUFFERS_DISK                          : 64
NUMBER_DATA_BUFFERS_FT                            : 16 (Default)
NUMBER_DATA_BUFFERS_RESTORE                       : 30 (Default)
 
 
SIZE_DATA_BUFFERS                                 : 262144
SIZE_DATA_BUFFERS_DISK                            : 1048576
SIZE_DATA_BUFFERS_FT                              : 262144 (Default)
 
 
Please confirm that this all looks ok.  
 

 

arctics2's picture

Also, we are using AIR with a second identical appliance.  Do these settings need to be configured there as well, or will AIR be unaffected?

Mark_Solutions's picture

That looks fine - the one that is just buffers refers to tape (SIZE_DATA_BUFFERS)

AIR will not be an issue in relation to these but it is nice to have both appliances the same.

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

arctics2's picture

Great, thanks for your help.  One last question.  To verify that this is getting better performance, do I just watch the "02/04/2014 10:56:36 - Info bptm (pid=23368) waited for full buffer 5460 times, delayed 17187 times" line in the logs to ensure it drops?

Mark_Solutions's picture

The bptm logs are good for this plus one other location that is really useful for duplications...

Just run an All Log Entries report from the Admin Console for a period before you changed them and a period afterwards

You will see entries something like successfully wrote (duplicate) copy 2 at xxxx MB/s

It is about the only place you see duplication throughputs in clear text - you can export it to a text file, import into excel - sort by message and then have a look at the rates you are getting when duplicating to tape

This will let you do a comparision

 

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

arctics2's picture

before:

Fri Jan 31 14:37:37 EST 2014 Info 11875 Backup successfully  wrote backup id client_1390261365, copy 2, fragment 1, 70248368 Kbytes at 33861.979 Kbytes/sec

After:

Wed Feb 05 11:39:27 EST 2014 Info 13458 Backup successfully  wrote backup id client_1390609931, copy 2, fragment 1, 70282749 Kbytes at 62421.065 Kbytes/sec

So looks much better, thanks.  

However, although the FT buffer settings were set as above, this morning, I see that a SAN client backup failed with the following in the logs:

02/04/2014 18:00:55 - Info bptm (pid=31988) using 1048576 data buffer size
02/04/2014 18:00:55 - Error bptm (pid=31988) number of buffers specified for FT, 64, must be >= 4 and <= 32, modify or use LAN
02/04/2014 18:00:55 - Info bptm (pid=31988) EXITING with status 20 <----------
 
02/05/2014 09:56:05 - Error bptm (pid=10853) buffer size (1048576) for FT must be a multiple of 65536 and be >= 65536 and <= 524288, modify or use LAN
02/05/2014 09:56:05 - Info bptm (pid=10853) EXITING with status 20 <----------
 
So I was able to get it to succeed by manually specifying 65536 data buffer size and 16 buffers.  Are these the highest recommended settings for Fibre Transport backups?  
Mark_Solutions's picture

Looks like some of it is down to the O/S : http://www.symantec.com/docs/TECH62007

http://www.symantec.com/docs/TECH206660

http://www.symantec.com/docs/TECH181285

The last 2 links relate to appliances but as usual there are lots of variables - I would try 32 buffers of 256k (262144) first and see how that goes

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.

arctics2's picture

Thanks Mark.  

256k seems to work well, but as per one of the docs you sent, plus the Netbackup Tuning guide, which says:

Note: It usually does not improve performance to increase memory buffers to a
number that is significantly more than the SAN Client Fibre Transport default
(16). Such an increase usually causes the majority of the buffers on either the
client or server side to be empty.
 
So I've left it at 16 buffers.  Am I missunderstanding the document?
Mark_Solutions's picture

No - that does sounds OK to me if that is Symantec view - if it is working OK with 32 then try 16 to see if it makes any difference

But which ever works for you best is the one to go for.

Authorised Symantec Consultant

Don't forget to "Mark as Solution" if someones advice has solved your issue - and please bring back the Thumbs Up!!.