Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

memory

Created: 08 Feb 2013 • Updated: 11 Feb 2013 | 12 comments
This issue has been solved. See solution.

Data are received by a bptm process and written into memory. The bptm process works on a block basic. Once a block is full, it's written to tape by another bptm process. Since the memory block size is equal to the SCSI block size on tape this, we need to take care !!. By using bigger block size we can improve tape writes by reducing system overhead.

http://www.mass.dk/netbackup/guides/49-netbackup-b...

the memory block size is equal to the SCSI block size on tape

What above  means?

Comments 12 CommentsJump to latest comment

captain jack sparrow's picture

Very simple

Lower block size, more time to fill it complete MEMORY BLOCK and wait till next block fills up with same small blocks

Larger block size . lesser time to wait

Though quote itself is self explanatory , 

 Cheers !!!

CJS

NIKHIL2346565959's picture

the memory block size is equal to the SCSI block size on tape

What  is this memory block size and SCSI block size on tape?

mph999's picture

If I am going to be picky, this is not always true.

"Data are received by a bptm process and written into memory."

If the backup is local, (media server backing itself up) the bpbkar process sends the data directly to te memory buffer, bptm process then reads it and sends it to the OS, which writes it to tape.

If the backup is remote (eg. client over network) bpbkar on the client sends the data to a TCP socket.  The child bptm process then reads ir from the socket and puts it in the memory buffer, the parent bptm process then reads it from the memory buffer and sends it to the OS.  At no time ever, does NBU write to tape.

Anyway, I got distracted ...

The memory buffer = a bucket

Data = water

The bucket has a size eg, 256k 128k etc ...  this is specified in the SIZE_DATA_BUFFERS file.

The rule is that the bucket has to be full of water before it can be tipped out ...

So if the bucket is size 256k (= 1024 x 256 - 262144 in SIZE_DATA_BUFFER file) the 256k worth of water is sent at a time to the drives (via os of course ...).

In real life, it is data, so 256k of data is sent, we call this a block of data, so the size of the block = the size of the buffer.

So if my data size is 512k and my block size is changed to be 128k 

512/128 = 4 so a total of 4 blocks would be sent.

So a block is simply a 'chunk' of data.

Before anyone asks, If you had total data of size of 520k, you would write 5 blocks of data :

128k , 128k , 128k , 128k , 8k = 520k

When before I said the bucket (buffer) has to be full, there is an exception if the last bit of data isn't enough to fill it (like above example).  Here we write a 'short block' which simply means the block sent is less than the specified size.

This is why NBU requires the drives to be in variable block mode, the block size can change during a backup .

Martin

Regards,  Martin
 
Setting Logs in NetBackup:
http://www.symantec.com/docs/TECH75805
 
CRZ's picture

Wouldn't you have been better off PM'ing Nicolai about his blog entry than starting a thread here?

(Maybe Nicolai will show up and have more patience than I have, though...)


bit.ly/76LBN | APPLBN | 75LBN

NIKHIL2346565959's picture

Thanks Martin.

What is this SCSI block size on tape ?

mph999's picture

It is what I have explained it as, just an amount of data of a specific size.

I don't really know how else to explain it ...

If I have data which is represented by x and the 'block is represented by and each x = 64k of data I have made the SIZE_DATA_BUFFERS file with 256k  (would be 262144)

So data on disk xxxx xxxx xxxx xxxx xx

I then send the data to tape 

|xxxx|

|xxxx|

|xxxx|

|xxxx|

|xx|

So the data fills the buffer, where the buffer is 256k, so therefore 256k is sent to tape, and simply, we call this a block.  Therefore my data files in 5 blocks, 4 blocks are 256k in size and the last block is 128k , as there wasn't enough data to fill it completely.

So the data on the tape looks like this :

(MH)(BH)(TM)|xxxx||xxxx||xxxx||xxxx||xx|(TM)(EH)(TM)(TM)

MH = Tape label, or media header

BH = Backup header

TM = Tape mark

So the data can be seen to be written to the tape in 'blocks' which are the same as what was sent from NBU.

Martin

Regards,  Martin
 
Setting Logs in NetBackup:
http://www.symantec.com/docs/TECH75805
 
NIKHIL2346565959's picture

So we defined this SCSI block size on the tape?

mph999's picture

Yes

Regards,  Martin
 
Setting Logs in NetBackup:
http://www.symantec.com/docs/TECH75805
 
NIKHIL2346565959's picture

Thaks Martin.

One last query:

If we dont have  SIZE_DATA_BUFFERS and NUMBER_DATA_BUFFERS file on our media servers,Then what will happen in that case?

/usr/openv/netbackup/db/config/SIZE_DATA_BUFFERS and 

/usr/openv/netbackup/db/config/NUMBER_DATA_BUFFERS

Marianne's picture

Nikhil, you are really something else...
With all your activity on Connect and (apparent) reading of documentation, have you NEVER heard of the Planning and Performance Tuning Guide? And that this guide tells you what the defaults are? And how to use bptm logs on media servers to what actual buffer sizes and numbers are used as well as the net effect on backups?

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

mph999's picture

As Marianne pointed out - it uses default values.  Something like 64k size and 16 buffers I think.

It does not really matter what the  defaults are, as in my experience the tuning must be applied to give acceptable performance when using modern tape drives (that is drives that perform at similar speeds to LTO drives.  A good starting point is 262144 (size) and 32 or 64 (number) but it is a case of 'try it and see'.

Regards,  Martin
 
Setting Logs in NetBackup:
http://www.symantec.com/docs/TECH75805
 
SOLUTION
Marianne's picture

Defaults are different for different NBU versions. All documented in the Tuning Guides for the different versions. Also very clear in bptm log when job starts.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links