Have you checked/verified the network path between the machines?

Have to tried to ftp a nice big chunk of data from the client(s) to the Linux server?

Hi Marianne,

All paths have been checked and I ahve compared my routing information on the Linux server with that of the old HP server and they are identical, I am unable to run a full traceroute due to firewall issues, all the clients are behind a firewall.

The new Linux master has retained both the FQDN and the IP of the old HP server so the ports should not be being blocked, unfortunately I am unable to test ftp as that is blocked by our firewalls, it certainly seems like a network issue but our Network Team cannot find any issues, connections to and from the Master/Client are working as normal it is just the backup speed that has sunk to an all time low.

It just seems strange that this was Ok on the HP master server but has caused issues on the Linux one, it is only a small subset of clients and all others are running as normal.

Have you check the media server speed at OS level with dd command? This will help to isolate the issue at media server level. If the issue is only with solaris clients why you dont upgrade just one box and see how it dors.

regards

Is there any type of non-NBU network transfer that the firewall admins will allow? scp? sftp?

It is important to get NBU 'out of the way' and test network transfer in other ways.

If you have bptm logs on the media server, you will merely see 'waited for full buffers ### times, delayed #### times'.
This is just saying that bptm did not get the data fast enough...
Combine this with bpbkar log on the clients.

If you feel that the different versions might be causing the problem, why not upgrade one client as a test?
I have not seen different NBU versions causing a significant slow-down.

I have just upgraded one of the clients to NBU6.5.5 (Solaris_x86_10_64) but am still only getting about 3Kb/s rather than approx 10Mb/s before moving from the HP master to the Linux one, I have had the network guys check their end and they claim there are no issues, all other backups are working apart from approx 5 Solaris clients which have almost ran to a stop since migrating the master server.

I have tried playing around with the Buffers but don't want to cahnge too much as the other client are Ok

Not sure what else to look at now to be honest so any pointers in the right direction would be appreciated

"Not sure what else to look at now..."

I have tried.... See this: https://www-secure.symantec.com/connect/forums/slo...

Have seen same, like very slow backups of RH5x using NBU6.x.

Supported but sloooooooow. Did upgrade of master and media servers to 7.x  - and then the RH clients to 7.x,  then backup ran fast.

Check bptm log as Marianne pointed out earlier.

If network remains the same, it's worth to run a local backup test on the client itself - see if disk I/O is a problem.

Command to use, as in technote http://www.symantec.com/docs/TECH17541:

Note it's for Windows, but can easily figure it out to be:

/usr/openv/netbackup/bin/bpbkar -nocont <path_with_10GB_data>  1> /dev/null  2> /dev/null

Run one on Linux, another on a HP-UX client (or whatever is faster), check their differences from bpbkar log.

This test would at least eliminate the disk I/O cause.

Hi,

Apologies Marianne, I was not ignoring your advice, I have checked the bptm logs on the Master/Media server and for the clients that are having the issue I cannot see any error, below is an example from one of the clients:

00:00:14.519 [23061] <2> bptm: INITIATING (VERBOSE = 0): -w -c devhost-5.ipcmedia.com -den 6 -rt 8 -rn 1 -stunit bfbackup-hcart-robot-tld-1 -cl VLS-DEVHOST -bt 1322006412 -b devhost-5.ipcmedia.com_1322006412 -st 1 -cj 12 -p HPUX_VLS -reqid -1321970945 -jm -brm -hostname devhost-5.ipcmedia.com -ru root -rclnt devhost-5.ipcmedia.com -rclnthostname devhost-5.ipcmedia.com -rl 0 -rp 604800 -sl Inc_Daily -ct 0 -maxfrag 1048576 -mediasvr bfbackup -connect_options 0x01010000 -jobid 2925 -jobgrpid 2925 -masterversion 650000 -bpbrm_shm_id 72417293 -blks_per_buffer512 00:00:16.575 [23061] <4> report_client: VBRC 2 23061 1 devhost5.ipcmedia.com_1322006412 0 VLS-DEVHOST 1 Inc_Daily 0 1 1 00:00:25.455 [23061] <4> write_backup: begin writing backup id devhost-5.ipcmedia.com_1322006412, copy 1, fragment 1, to media id B0450A on drive BFBACKUP-VLS-DRIVE2 (index 8)00:00:25.455 [23061] <2> io_write_back_header: drive index 8, devhost-5.ipcmedia.com_1322006412, file num = 7, mpx_headers = 0, copy 1

As you can see I do not get any wait messages at all.

I did not have the bpbkar logs set up on the client so have now implemented this and started a backup about an hour ago but am not seeing anything in there:

[root@devhost-5.ipcmedia.com]$ tail -f log.112311

08:11:48.992 [23513] <4> bpbkar: INF - setenv RESTARTED=0
08:11:48.992 [23513] <4> bpbkar: INF - setenv BACKUPID=devhost-5.ipcmedia.com_1322035597
08:11:48.992 [23513] <4> bpbkar: INF - setenv UNIXBACKUPTIME=1322035597
08:11:48.992 [23513] <4> bpbkar: INF - setenv BACKUPTIME=Wed Nov 23 08:06:37 2011
08:11:48.992 [23513] <4> bpbkar: INF - BACKUP START
08:11:48.992 [23513] <4> bpbkar: INF - Estimate:-1 -1
08:11:48.993 [23513] <2> bpbkar add_to_filelist: starting sizeof(filelistrec) <112>
08:11:48.993 [23513] <4> bpbkar: INF - Processing /export/zones/dev5/root/export/trac
08:11:49.014 [23513] <4> bpbkar: INF - VxFS filesystem is /export/zones/dev5/root/export/trac for /export/zones/dev5/root/export/trac

Below is the output of the local backup using the bpbkar -nocont command on a 1.77Gb area:

09:09:56.921 [23597] <2> logparams: /usr/openv/netbackup/bin/bpbkar -nocont /export/home/software
09:09:56.925 [23597] <4> bpbkar: INF - setenv KEYWORD=NONE
09:09:56.925 [23597] <4> bpbkar: INF - setenv STREAM_PID=23597
09:09:56.925 [23597] <4> bpbkar: INF - setenv STREAM_NUMBER=0
09:09:56.925 [23597] <4> bpbkar: INF - setenv STREAM_COUNT=0
09:09:56.925 [23597] <4> bpbkar: INF - setenv STREAMS=0
09:09:56.925 [23597] <4> bpbkar: INF - setenv BPSTART_TIMEOUT=0
09:09:56.925 [23597] <4> bpbkar: INF - setenv BPEND_TIMEOUT=0
09:09:56.925 [23597] <4> bpbkar: INF - setenv RESTARTED=0
09:09:56.925 [23597] <4> bpbkar: INF - Estimate:-1 -1
09:09:56.925 [23597] <2> bpbkar add_to_filelist: starting sizeof(filelistrec) <112>
09:09:56.925 [23597] <4> bpbkar: INF - Processing /export/home/software
09:10:47.459 [23597] <4> bpbkar: INF - Client completed sending data for backup
09:10:47.529 [23597] <4> bpbkar: INF - bpbkar exit normal
09:10:47.541 [23597] <4> bpbkar: INF - EXIT STATUS 0: the requested operation was successfully completed
09:10:47.541 [23597] <4> bpbkar: INF - setenv FINISHED=1
[root@devhost-5.ipcmedia.com]$

Looking at all the logs I am not getting any errors at all showing up.

I have just ran a Master server initiated backup of the client using the same area (/export/home/software) as I did with the client initiated one and this is still running over 10 minutes later at 361Kps and has written only 180Mb

Looking at the difference between the time of the clinet initiated backup and the master initiated I would think that this is now pointing to some form of network issue rather than an NBU problem, the worrying thing is that this has only started to happen since moving from the HP-UX Master onto a RHEL Master, and as I have previously stated all the Network configurations are identical between the two different servers, does RHEL do something different than HP??