Backup ends in Status Code 24
I have upgraded a number of my NetBackup clients from version 6.0 MP7 to version 7.0.1. Since doing this, all of the upgraded clients report status code 24 socket write failed. Sometimes this error appears 8 times during a backup, other times it might only appear once.
I did raise a support ticket with Symantec who advised me that this is a networking issue and that I need to get my network engineer to look at it, I was also advised to disable the TCP Chimney offload as per one of the tecnotes.
Unfortunately, both the Master server and some of the clients are connected to the network using LACP port channels, if you disable the TCP offload, then the LACP bond does not get created, so I cannot disable the TCP offload
I also asked my network engineer to take a look at it, he went on the LAN switch and noticed a high number of output packet drops were being recorded on the network interfaces for both the Master server and clients.
What is curious is that this output packet drop does not appear for servers that have the NetBackup 6.0 client installed - only the ones upgraded to version 7.0
According to Cisco's documentation, output packet drop happens when the NIC is overloaded so the switch cannot process the packets fast enough. Cisco recommend reducing the bandwidth used to resolve this.
Has anyone else seen this issue? If so, how did you resolve it?
Finaly cracked it
The problem was being caused by there not being enough memory on the Master Server to receive the incomming packets. We could see that by doing a netstat -i which showed a high receive packet drop. This corresponds to the output packed drop seen on the cisco switches.
The problem was resolved by setting the /usr/openv/netbackup/NET_BUFFER_SZ to 262144 now it all works properly