Video Screencast Help

SAN media servers backups are runnig slow

Created: 29 Dec 2012 | 8 comments
Rajesh_s's picture

Hi,

We are experiencing a huge throughput issue post upgrade of NetBackup 7.5.0.4 from 7.1. Please let me know if any changes or recommendation we need to perform.

 

Regards,

Rajesh

 

 

 

Discussion Filed Under:

Comments 8 CommentsJump to latest comment

Marianne's picture

There are no known performance issues with 7.5.x.

Please post bptm logs from before and after upgrade as File attachments.

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

Rajesh_s's picture

Attaching the logs of before 7.5 upgrade

AttachmentSize
Before_7.5 upgrade.zip 17.6 KB
 Rajesh 
Rajesh_s's picture

Attaching the logs of after 7.5 upgrade

AttachmentSize
After_7.5 upgrade.zip 39.08 KB
 Rajesh 
Rajesh_s's picture

Hello Marrine,

can you please help me if any tuning guide is available or any tuning we need to do, i wont think so its released for 7.5.

We are using windows 2008-R2 OS and all the SAN media servers are upgraded to 7.5.0.4.

Regards,
Rajes

 Rajesh 
revaroo's picture

Logs are for bpbrm, would need the bptm log from the media server and bpbkar from the client (if this is a SAN media server logs would be from the SAN Media server for both media and client as they are the same machine).

 

Logs for just bpbrm do not help with troubleshooting performance issues.

Rajesh_s's picture

Hi Revaroo, I am talking about SAN media server performance issue and i have uploaded BPTM logs . Ours is an SSO environment and the SAN media servers backups are going to tape through its own.

Previous week we have done upgrade from Netbackup 7.1 to 7.5.0.4 and on the same time we have configured Symantec End point protection (AV) earlier we were using CA , post to this changes we are experiencing this performance issue.

 

Please let me know if anything required. Rgds, rajesh

 Rajesh 
revaroo's picture

Yes SAN Media server, a server that is only allowed to backup itself. 

> Please let me know if anything required.

I already mentioned, bpbkar and bptm logs.

 

Marianne's picture

I can see big differences in transfer rates in the 2 log files, but we are looking at 2 different policies as well - do they have diferent backup selections as well?

Oh, and a matter of interest - seems SAN media server was upgraded from 6.5.1.

This is what I see in the logs :

 

Before:
 
2:35:00.048 [12048.16964] <2> bptm: INITIATING (VERBOSE = 0): -w -pid 13908 -c MSDB5 -den 6 -rt 8 -rn 1 -stunit MSDB5-hcart-robot-tld-1 -cl MSDB5 -bt 1355000648 -b MSDB5_1355000648 -st 0 -cj 2 -p Weekly -reqid -1354219791 -jm -brm -hostname MSDB5 -ru root -rclnt MSDB5 -rclnthostname MSDB5 -rl 2 -rp 1814400 -sl Sec_Weekly_Full -ct 13 -maxfrag 1048576 -tir -eari 0 -mediasvr MSDB5 -no_callback -connect_options 0x01010100 -jobid 364485 -jobgrpid 364477 -masterversion 750000 -bpbrm_shm_id 
 
02:35:03.915 [12048.16964] <2> io_init: using 65536 data buffer size
02:35:03.915 [12048.16964] <2> io_init: CINDEX 0, sched Kbytes for monitoring = 60000
02:35:03.915 [12048.16964] <2> io_set_recvbuf: setting receive network buffer to 263168 bytes
02:35:03.916 [12048.16964] <2> io_init: using 30 data buffers
 
02:36:03.041 [12048.16964] <2> manage_drive_attributes: Host Attributes: Vendor [SYMANTEC], Name [NetBackup BPTM], Version [6.5.1]
 
02:50:04.989 [12048.16964] <4> write_backup: successfully wrote backup id MSDB5_1355000648, copy 1, fragment 1, 14469440 Kbytes at 20537.662 Kbytes/sec
 
03:06:46.718 [12048.16964] <4> write_backup: successfully wrote backup id MSDB5_1355000648, copy 1, fragment 2, 47914112 Kbytes at 47893.326 Kbytes/sec
 
03:23:48.491 [12048.16964] <4> write_backup: successfully wrote backup id MSDB5_1355000648, copy 1, fragment 3, 66235392 Kbytes at 64892.189 Kbytes/sec
....
....
04:45:35.085 [12048.16964] <2> write_data: waited for full buffer 165339 times, delayed 327544 times
04:45:35.085 [12048.16964] <2> write_data: Total Kbytes transferred 336418887
 
After:  (backup starting during working hours??)
 
08:44:55.229 [7344.7956] <2> bptm: INITIATING (VERBOSE = 0): -w -pid 7504 -c MSDB5 -den 6 -rt 8 -rn 1 -stunit MSDB5-hcart-robot-tld-1 -cl MSDB5 -bt 1356664459 -b MSDB5_1356664459 -st 0 -cj 2 -p Daily -reqid -1354311913 -jm -brm -hostname MSDB5 -ru root -rclnt MSDB5 -rclnthostname MSDB5 -rl 1 -rp 1209600 -sl Full -ct 13 -maxfrag 1048576 -tir -eari 0 -mediasvr MSDB5 ...
 
08:45:00.761 [7344.7956] <2> manage_drive_attributes: Host Attributes: Vendor [SYMANTEC], Name [NetBackup BPTM], Version [7.5.0.4]
 
08:44:58.923 [7344.7956] <2> io_init: using 65536 data buffer size
08:44:58.923 [7344.7956] <2> io_init: CINDEX 0, sched Kbytes for monitoring = 600000
08:44:58.923 [7344.7956] <2> io_set_recvbuf: setting receive network buffer to 263168 bytes
08:44:58.923 [7344.7956] <2> io_init: using 30 data buffers
 
08:59:58.751 [7344.7956] <4> write_backup: successfully wrote backup id MSDB5_1356664459, copy 1, fragment 1, 8596864 Kbytes at 9631.693 Kbytes/sec
 
09:14:58.636 [7344.7956] <4> write_backup: successfully wrote backup id MSDB5_1356664459, copy 1, fragment 2, 8271424 Kbytes at 9222.915 Kbytes/sec
 
09:29:58.403 [7344.7956] <4> write_backup: successfully wrote backup id MSDB5_1356664459, copy 1, fragment 3, 10112640 Kbytes at 11277.102 Kbytes/sec
....
23:48:03.258 [7344.7956] <4> write_backup: successfully wrote backup id MSDB5_1356664459, copy 1, fragment 46, 7390208 Kbytes at 10594.156 Kbytes/sec
 
We don't see the end result as backup ran into next day with rest of info in next day's bptm log.
 
II would blame SEP for slow throughput rather than NBU upgrade. You may want to add NBU executables to SEP exclude lists. See these previous discussions: 
 
 

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links