GENERAL ERROR: The bpbkar process hangs when attempting to back up a NFS file system.

Article:TECH29475  |  Created: 2003-01-06  |  Updated: 2013-10-23  |  Article URL http://www.symantec.com/docs/TECH29475
Article Type
Technical Solution


Environment

Issue



GENERAL ERROR: The bpbkar process hangs when attempting to back up a NFS file system.

Solution



Overview:
It is possible for the bpbkar daemon to hang when backing up a client that has a Network File System (NFS) directory mounted.  If this occurs, there are different troubleshooting steps to determine what file or directory is causing the backup to hang.  Typically, any hangs relating to NFS file systems are due to some underlying network problem or problem accessing files on the NFS server.  

NetBackup has to be configured to follow NFS mounts within a given backup policy.  However, each backup builds a mount list at the start of the backup.  It is possible for a hang to occur trying to get the mount information for the NFS file system.  This step will occur regardless of what attributes have been set in the policy or what exclude_list entries have been added to the client.  If the NetBackup policy has been configured with the Follow NFS option to backup NFS file systems, then it is possible a file or directory cannot be read properly.  In either case, it will be necessary to enable bpbkar logging to determine what file or directory is being read when the backup hangs.

Troubleshooting:
To troubleshoot NFS hangs, it is best to enable bpbkar logging on the client.  This will capture the creation of the mount list at the start of the backup, as well as any files that are being backed up.

To display what file is being read, it is necessary to enable bpbkar_path_tr logging on the client.  This will create additional SelectFile: log entries in the bpbkar log that show the current file being read during the backup.
First, run the following commands on the client:
# cd /usr/openv/netbackup
# touch bpbkar_path_tr
# mkdir /usr/openv/netbackup/logs/bpbkar

Finally, edit the /usr/openv/netbackup/bp.conf file on the client and add the entry:
VERBOSE = 5
It will not be necessary to restart the NetBackup daemons.  These changes will take effect the next time a backup is run.

Log Files:
The /usr/openv/netbackup/logs/bpbkar/log.<date> file on the client will show the following entries if the backup is failing to build the mount list for the client.  The last entry recorded in the log for the process id listed shows the backup was trying to process an NFS mounted directory.  A normal log file should continue past this step and display the backup options and a list of files that were backed up.
07:08:38.444 [28849] <2> mount build_mount_list: INF - Processing (autofs) -hosts on /net
07:08:38.444 [28849] <2> mount build_mount_list: INF - Processing (autofs) -xfn on /xfn
07:08:38.444 [28849] <2> mount build_mount_list: INF - Processing (NFS) server123:vold(pid27240) on /test

In this case, attempt to cd to the NFS mounted file system to see if it is accessible.
# cd /test

If the bpbkar log gets past the build_mount_list section then check for any SelectFile: entries that indicate what file was being read.  The following example shows a different backup where the backup hung reading a file.
14:47:27.696 [23703] <4> bpbkar PrintFile: /test/dir1/file2
14:47:27.696 [23703] <2> bpbkar SelectFile: INF - cwd = /test/dir1
14:47:27.696 [23703] <2> bpbkar SelectFile: INF - path = file3

In this case, verify the system can read the file name listed in the log.
# ls -l /test/dir1/file3
If operating system commands such as ls and cp hang as well, this indicates the system cannot properly access the NFS directory.

Resolution:
After determining which NFS file or directory that is causing bpbkar to hang, it will be necessary to determine the cause.  Below are some possible causes for NFS hangs during a client backup.  If the operating system is unable to access the files to be backed up, then the backup will fail as well.

If basic system commands such as ls and cp also hang, then please contact the operating system vendor for assistance with troubleshooting the issue.

In some cases if NetLink 1.2 is installed on a NetBackup 4.5 client, the backup will hang.  Disabling or removing these application may allow the bpbkar process to complete successfully.

Check the mount options for the NFS directory by running the mount command on the client.  The NFS option intr should be enabled to allow file operations to be interrupted.  This would be a default mount option on Solaris systems and would allow clients to interrupt applications that may be waiting for a remote mount.  Some Linux clients do not use the intr option by default which may cause the client to hang indefinitely waiting for NFS operation to complete.


Legacy ID



264524


Article URL http://www.symantec.com/docs/TECH29475


Terms of use for this information are found in Legal Notices