Puredisk content router process (spoold) failing to startup.

Article:TECH169920  |  Created: 2011-09-20  |  Updated: 2012-03-08  |  Article URL http://www.symantec.com/docs/TECH169920
Article Type
Technical Solution


Issue



The spoold process is not starting. After running a PureDisk services startup "/etc/init.d/puredisk start" the spoold process shows as "failed".

Starting PureDisk ContentRouter: spoold                              failed

Error



Checking the /Storage/log/spoold/spoold.log on the content router we can see this error message

August 23 12:47:03 INFO [46992284444128]: Database Manager: initializing
August 23 12:47:03 INFO [46992284444128]: Database Manager: initialization complete
August 23 12:47:03 INFO [46992284444128]: Storage Cache Manager: Initialized in 0.44 seconds.
August 23 12:47:03 INFO [46992284444128]: Storage Cache Manager: initialization complete
August 23 12:47:03 INFO [46992284444128]: Spooler Cache Manager: initializing
August 23 12:47:03 INFO [46992284444128]: Spooler Cache Manager: initialization complete
August 23 12:47:03 INFO [46992284444128]: DataStore Manager: initializing
August 23 12:47:03 ERR [46992284444128]: -1: ds_checkIdentity: identity file /Storage/data/.identity does not exist
August 23 12:47:03 ERR [46992284444128]: 25004: _storeInit: the identity of data store is not valid, spoold cannot con                      tinue.
August 23 12:47:03 ERR [46992284444128]: 26016: Store Manager: Activate failure.

Cause



Checked the /Storage/data directory to see if this .identity file existed. The .identity file should contain the storagepool ID and the contentrouter ID

# ls -al
total 40
drwxr-x---  3 root root 8192 Aug 23 13:03 .
drwxr-xr-x 16 root root 8192 Jan 26  2011 ..
-rwxr-x---  1 root root    8 Jul  1  2010 .cnf
-rwxr-x---  1 root root   56 Jul  1  2010 .cnt
-rwxr-x---  1 root root   56 Jul  1  2010 .path.cnt
drwxrwxrwx  2 root root   96 Aug 23 13:03 deref

As the file does not exist, it can be generated by running the command

/opt/pdcr/bin/spoold --genidentity      

 

We then see that a .identity file exists

# ls -dl .id*
-rw-r--r-- 1 root root 45 Aug 23 14:16 .identity

And the file contains the required information

# cat .identity
[Main]
StoragePoolID = 1
ContentRouterID = 2

Before starting the contentrouter process it should be investigated why this file was missing and why there are not other files in /Storage/data.

Checked the /etc/fstab and /Storage/data is a separate file system to /Storage.

/dev/vx/dsk/PDDG/PDCRVOL /Storage/data          vxfs    suid,_netdev    0 0  

 This file system is not mounted so that is why the .identity file does not exist and why there is no other files in /Storage/data.

Trying to manually mount /Storage/data results in an error

 # mount /Storage/data UX:vxfs mount.vxfs:                                                                             ERROR: V-3-21268: /dev/vx/dsk/PDDG/PDCRVOL is corrupted. needs checking

Checking the /var/log/messages on the content router we can see errors

Aug 22 19:00:55 server1 kernel: vxfs: msgcnt 6 mesg 039: V-2-39: vx_writesuper - /dev/vx/dsk/PDDG/PDCRVOL file system super-block write error
Aug 22 19:00:56 server1 kernel: vxfs: msgcnt 7 mesg 037: V-2-37: vx_metaioerr - vx_logbuf_write - /dev/vx/dsk/PDDG/PDCRVOL file system meta data write error in dev/block 0/752965
Aug 22 19:00:56 server1 kernel: vxfs: msgcnt 8 mesg 031: V-2-31: vx_disable - /dev/vx/dsk/PDDG/PDCRVOL file system disabled
 

 

 


Solution



After making sure the backend storage has no issues an FSCK of the file system should resolve the problem with the file system

/opt/VRTS/bin/fsck -t vxfs  /dev/vx/dsk/PDDG/PDCRVOL

Once complete you should be able to mount the file system

#mount /Storage/data

Then start the Puredisk services

#/etc/init.d/puredisk start

NOTE:- Once the system is back up and running, it may be worth doing a consitenct check in case any data has been lost due to the file system issues.

 




Article URL http://www.symantec.com/docs/TECH169920


Terms of use for this information are found in Legal Notices