Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Netbackup 7.5.0.5 on Windows 2008 r2 - slow performance from DSSU to tape

Created: 01 Mar 2013 | 7 comments

Hello,

I have a new Windows 2008 r2 server running Master&Media server v7.5.0.5.

I have a Arcvault 12 tape library - which Windows 2008 r2 sees correctly in Device Manager and I have installed the latest HP drivers (v3.7.0.0) for the Ultrium LTO 4 that is in that library.

Backing up clients to a DSSU (basic disk).  That appears to work really well.

I can backup the DSSU to tape and everything works, except the tape drive backup speed is rock bottom - around 2MB/sec.

Running HP's tape tool on the system to test outside of NB, I get tape write speeds around 70MB/sec.

I've tried different SIZE_DATA_THIS and NUMBER_DATA_THAT settings with no tape write speed changes.

 

Any thoughts?

 

Thanks,

-Bill

Operating Systems:

Comments 7 CommentsJump to latest comment

Yasuhisa Ishikawa's picture

Please tell us more about your enviroment like:

  • exact value you wrote into SIZE_DATA_BUFFERS and NUMBER_DATA_BUFFERS
  • 2MB/s on each drives(24MB/s total), or 2MB/s total with 12 drives
  • disk idle % of DSSU disk while duplication
  • performance of direct backup to tape
  • output of "bpstulist -label stu_name -L" for DSSU and Final Destinaion Storage Unit

Authorized Symantec Consultant(ASC) Data Protection in Tokyo, Japan

Nicolai's picture

Ensure NUMBER_DATA_BUFFERS and SIZE_DATA_BUFFERS don't have ant extensions like .txt or .rtf

The files MUST be named exactly NUMBE_DATA_BUFFERS & SIZE_DATA_BUFFERS

Look in the activity monitor. Both the number and size of the buffers should be in the job status detailed view.

Assumption is the mother of all mess ups.

If this post answered your'e qustion -  Please mark as a soloution.

BillHutchison's picture

It appears something inside of NetBackup's disk staging process is throttling DSSU reads to 2MB/s.

My DSSU existd on it's own volume under Windows 2008 R2.  This is a brand new install, fresh, no upgrades etc.  And I can read off that volume at very high speeds at the OS level.

But if I watch the Disk Read byte/sec for that volume while the DSSU process of NetBackup's is moving those files in the DSSU out to tape, it's reading right up to 2MB/sec, tops.

The DSSU's disk/volume is between 80-90% idle during the DSSU to tape.

And while that is running, I can copy a 100GB+ file off that same volume to another location on my network at ~70MB/sec (I can see the Disk Read byte/sec climb up to 70MB/sec while both the DSSU to tape and the network copy are happening at the same time).  Then when the network copy finishes, the volume's Disk Read Byte/Sec drops right back to 2MB/sec while the DSSU pokes along.

So my tape drive throughput is being held back by NB's DSSU read throughput, but that's not being held back by any of the under lying system....

What's the magic to getting 7.5.0.5 on Windows 2008 R2 DSSU's to tape process to read from the DSSU faster?

 

Thanks,

-Bill

BillHutchison's picture

Attached is a screen shot of a performance test run using HP's LTT on this very same server to the exact same SCSI tape drive (an HP LTO 4 in a Arcvault 12 Library).

HP's LTT performance test writes at an average rate of 47MB/s (using 16GB total of random data and using a 64K block size).

If I then fire up Netbackup 7.5.0.5 services (master & media) on this same server, the best I can get bptm to send to the very same tape drive is 2MB/s.

There are no special drivers from Symantec to install for tape drives now with 7.5+, I'm using the current HP Windows driver 1.0.7.0.

There is only this one tape drive directly attached to this system via a SCSI HBA.

And if I try to send data to that tape drive from a networked 7.5.0.5 NBU client I get the same rate as I do sending from that's server's hosted DSSU to the same tape drive - 2MB/s.  Always.

 

bptm's NUMBER_DATA_BUFFERS = 256 and SIZE_DATA_BUFFERS = 65536.  Changing these values has NO effect on the 2MB/s rate - not for better or for worse.

Is everyone with Windows 2008 R2 and Netbackup 7.5+ services that uses SCSI tape drives suffering this same terrible performance?

Everything on this server is current, everything on these devices are current.  All firmware, drivers, software, etc.

 

-Bill

HPLTT_perf_results.jpg
Marianne's picture

Please see this TN: http://www.symantec.com/docs/TECH145791

Netbackup 7 running on Windows 2008 R2 platform may encounter slow performance

 

Performance Tuning Article: https://www-secure.symantec.com/connect/articles/tuning-windows-2003-and-2008-symantec-netbackup

 

Supporting Storage Foundation and VCS on Unix and Windows as well as NetBackup on Unix and Windows
Handy NBU Links

BillHutchison's picture

Hello Marianne,

Thank you - I have seen TN 145791 and did not have any of those issues (I'm running Service Pack 1 on this Windows Server 2008 R2 server, the power settings were set to 'high performance', and no chiminey offload on the NICs of this system and what TCP offload tweaks they have, they are enabled).

The second one, the performance tuning article, is pretty vast - I'll have to dig through it - it's difficult to know which variables to try first.  But a lot of them say they are obsolete with 2008R2.

Again thank you for this.

I'm still waiting on Symantec support - they have been chewing on this for a few days now without any word back.

 

-Bill

BillHutchison's picture

Switched my HBA from LSI to a new Adaptec 29320LPE and the throughput to tape is around 70MB/sec.

 

Thanks for any and all suggestions.

-Billl