VERITAS File System reports inodes 'marked bad incore' or 'marked bad ondisk' in the /var/adm/messages file.

Article:TECH32495  |  Created: 2004-01-04  |  Updated: 2010-01-09  |  Article URL http://www.symantec.com/docs/TECH32495
Article Type
Technical Solution


Environment

Issue



VERITAS File System reports inodes 'marked bad incore' or 'marked bad ondisk' in the /var/adm/messages file.

Error




vxfs: mesg 017: V-2-17: vx_iread_1 - /cfstest file system inode 29 marked bad incore

or

vxfs: mesg 079: V-2-79: vx_tranuninode - /cfstest file system inode 5 marked bad ondisk

Solution



This problem occurs only if all the following conditions exist:

1. VRTSvxfs 4.0ga is installed

2. A full fsck [without the -n option] is run on a file system

3. The file system is then mounted

4. And while the file system is actively being updated either:
         a. for local file system - the system crashes
         b. for cluster file system - a change of primaryship takes place

5. a. For local file system, the system will boot and log replay fsck run and the file system is then mounted
   b. For cluster file system, log replay fsck will occur on the assuming primary node

6. Further file system activity might result in the following type of errors being seen in the /var/adm/messages file:
  vxfsckd: [ID 801593 daemon.notice] /dev/vx/rdsk/testdg/testvol: replay complete - marking super-block as CLEAN
  vxfs: [ID 702911 kern.warning] WARNING: msgcnt 1 mesg 000: V-2-95: vx_setfsflags - /dev/vx/dsk/testdg/testvol filesystem fullfsck flag set - vx_ierror
  vxfs: [ID 702911 kern.warning] WARNING: msgcnt 2 mesg 017: V-2-17: vx_iread_1 - /cfstest file system inode 29 marked bad incore  <=========
  vxfs: [ID 885974 kern.info] vxfs msgcnt 2 offset 0x00000000     81a0        1      3e7       c7
  vxfs: [ID 885974 kern.info] vxfs msgcnt 2 offset 0x00000010        0 a3a02000 40b238df     2e5e
  vxfs: [ID 885974 kern.info] vxfs msgcnt 2 offset 0x00000020 40b238dc    9fdff 40b238dc    9fdff
  vxfs: [ID 885974 kern.info] vxfs msgcnt 2 offset 0x00000030    30020     162e        0        7
  vxfs: [ID 702911 kern.warning] WARNING: msgcnt 3 mesg 016: V-2-16: vx_ilisterr: vx_iread_3 - /cfstest file system error reading inode 29



For example,  in a cluster file system environment:
clus1 # fsck -F vxfs -o full -y /dev/vx/rdsk/testdg/testvol
log replay in progress
pass0 - checking structural files
pass1 - checking inode sanity and blocks
pass2 - checking directory linkage
pass3 - checking reference counts
pass4 - checking resource maps
OK to clear log? (ynq)y
flush fileset headers? (ynq)y
set state to CLEAN? (ynq)y
clus1 #

clus2 # mount -F vxfs -o cluster /dev/vx/dsk/testdg/testvol /cfstest
clus1 # mount -F vxfs -o cluster /dev/vx/dsk/testdg/testvol /cfstest

clus2 # fsclustadm -v showprimary /cfstest
clus2

clus1 # fsclustadm setprimary /cfstest
vxfs: WARNING: msgcnt 3 mesg 000: V-2-95: vx_setfsflags - /dev/vx/dsk/testdg/testvol filesystem fullfsck flag set - vx_ierror
vxfs: WARNING: msgcnt 4 mesg 017: V-2-17: vx_iread_1 - /cfstest file system inode 6 marked bad incore
vxfs: WARNING: msgcnt 5 mesg 016: V-2-16: vx_ilisterr: vx_iread_3 - /cfstest file system error reading inode 6
vxfs: WARNING: msgcnt 6 mesg 017: V-2-17: vx_iread_1 - /cfstest file system inode 5 marked bad incore
vxfs: WARNING: msgcnt 7 mesg 016: V-2-16: vx_ilisterr: vx_iread_3 - /cfstest file system error reading inode 5


This issue is fixed in Rolling Patch2 (RP2) for VERITAS File System 4.0.


As this is a potentially serious state for any file system, it is imperative that RP2 for File System 4.0 is installed whether or not these symptoms have been seen.


To clear down or avoid the issue, the following steps need to be done:

1. Apply RP2 to all nodes in the cluster
2. Reboot all nodes in the cluster
3.Unmount the affected file systems from all nodes
4. Run a full fsck -y on the affected file system
5. Mount the file system

All traces of this issue will have been removed and the file system will be in a completely clean state.



However,
should errors be seen which report inodes "marked bad ondisk" as in the /var/adm/messages example below:

Jun  1 10:33:37 clus1 vxfs: WARNING: msgcnt 3 mesg 000: V-2-95: vx_setfsflags - /dev/vx/dsk/testdg/testvol filesystem fullfsck flag set - vx_ierror
Jun  1 10:33:37 clus1 vxfs: WARNING: msgcnt 4 mesg 017: V-2-17: vx_iread_1 - /cfstest file system inode 6 marked bad incore
Jun  1 10:33:37 clus1 vxfs: WARNING: msgcnt 5 mesg 016: V-2-16: vx_ilisterr: vx_iread_3 - /cfstest file system error reading inode 6
Jun  1 10:33:37 clus1 vxfs: WARNING: msgcnt 6 mesg 017: V-2-17: vx_iread_1 - /cfstest file system inode 5 marked bad incore
Jun  1 10:33:37 clus1 vxfs: WARNING: msgcnt 7 mesg 016: V-2-16: vx_ilisterr: vx_iread_3 - /cfstest file system error reading inode 5
Jun  1 10:33:37 clus1 vxfs: WARNING: msgcnt 8 mesg 079: V-2-79: vx_tranuninode - /cfstest file system inode 6 marked bad ondisk     <=========
Jun  1 10:33:37 clus1 vxfs: WARNING: msgcnt 9 mesg 079: V-2-79: vx_tranuninode - /cfstest file system inode 5 marked bad ondisk     <=========

it is very important that there is no attempt to run a full fsck on the affected file system as these inodes will be removed during a 'full fsck -y' operation.

In this situation, you must contact Symantec Enterprise Technical Support








Supplemental Materials

SourceiTools
Value143127
Descriptionfcl can cause skipping of extop processing on mount


Legacy ID



268523


Article URL http://www.symantec.com/docs/TECH32495


Terms of use for this information are found in Legal Notices