Error Codes/Messages:
The Problems report on the master server will list the following errors.  

3/29/2005 12:49:11 AM <master> <client> Error 4832
Backup from client <client>: ERR - Cannot lstat <filename>. Errno = 5: I/O error

or

3/29/2005 12:49:11 AM <master> <client> Error 4832
Backup from client <client>: ERR - Cannot lstat64 <filename>. Errno = 5: I/O error

The /usr/openv/netbackup/logs/bpbkar/log.<date> file on the client will list the filename that generated the error.
03:15:29 [17734] <16> bpbkar: ERR - Cannot lstat64 <filename>. Errno = 5: I/O error

Overview:
When the bpbkar daemon on the system is unable to read a file on the file system, it can fail with lstat or lstat64 errors. These errors will occur in the Problems report.  The client backup will fail with a status 1 (the requested operation was partially successful). This can also occur during catalog backups on the master server, which will fail with a status 124 (NetBackup database backup failed, a path was not found or is inaccessible).

The operating system returns an "Errno = 5: I/O error" from the lstat or lstat64 system call. Whether the error comes from lstat or lstat64 will depend on whether the lstat system call is used or whether the 64-bit lstat64 is used for files larger than 2GB in size.

Troubleshooting:
Check the NetBackup (tm) Problems report for errors, or enable bpbkar logging on the client. These will list the specific file(s) that are triggering the error message.

To get the error report for the specific job ID that failed with the status 1, run the following command on the master:
# /usr/openv/netbackup/bin/admincmd/bperror -jobid <jobid>

This will show the specifics for the individual job ID as opposed to searching the whole Problems report.

Resolution:
Typically, the lstat or lstat64 message followed by an "Errno = 5: I/O error" indicates a file system or hardware error. Another potential cause is when a file is removed during the backup.

Check the log for the file(s) causing the error. Then check the following to see why lstat errors are generated when reading the file:
I. What file system are the files on?

Run ls -l from within the directory where the filename resides.

Run ls -l <filename> on the filename itself.

This will perform a simple stat on the file and verify that the filename can be seen in the directory.

 

II. What type of files do they appear to be?

If this is a text or application file, try editing the filename or opening it in an application to verify the file can be read properly.

Try copying the file to another file system to verify the operating system can read the file properly.  

 

III. Review the system logs (syslog, messages, etc) for SCSI errors or anything related to the disk where the files are reported to exist.  

Use the timeframe reported in the Problems report or bpbkar log to check for corresponding errors in the system logs.

 

IV. Unmount the file system where the files are reported and run a file system check to uncover any underlying file system issues.

Contact the operating system vendor for instructions on how to unmount and check the file system.

 

V. Set up logging and run a bpbkar interactive test on the directory where the files are purported to exist: