Veritas Operations Manager "xprtld" doesn't startup

Article:TECH140387  |  Created: 2010-09-20  |  Updated: 2012-10-10  |  Article URL http://www.symantec.com/docs/TECH140387
Article Type
Technical Solution


Issue



After installing VRTSsfmh, the xprtld process will not start on AIX.

A lslpp confirms VRTSsfmh is installed and committed

# lslpp -l | grep VRTSsf
 VRTSsfmh                 3.0.357.0  COMMITTED  Veritas Storage Foundation
 VRTSsfmh                 3.0.357.0  COMMITTED  Veritas Storage Foundation

BUT there is no xprtld as SHOULD be seen here from a "ps" output

# ps -ef | grep xprtld
    root 1015956       1   0 06:58:50      -  0:00 /opt/VRTSsfmh/bin/xprtld -X 1 /etc/opt/VRTSsfmh/xprtld.conf
    root 1147066  229490   0 06:59:41  pts/0  0:00 grep xprtld


Error



# /opt/VRTSsfmh/bin/xprtld /etc/opt/VRTSsfmh/xprtld.conf
Cannot get credential for service

A truss from the above command shows:

688378: 1527845: 0.3753:        statx("/var/VRTSat/.VRTSat/profile/certstore/keystore/KeyStore.lock", 0x2FF1FEF0, 76, 0) = 0
688378: 1527845: 0.3754:        kfcntl(12, F_SETLK, 0x2FF1FFB0) = 0
688378: 1527845: kwrite(2, " C a n n o t   g e t   c".., 33) = 33
688378: 1527845: kwrite(2, "\n", 1)                  = 1


Environment



VCS 5.1RP1

AIX6.1

VOM 3.0

NOTE: Do not use this tech note when /opt/VRTSsfmh/bin/xprtlc  -l https://localhost/world/getvitals  return correct information, Only proceede further when /opt/VRTSsfmh/bin/xprtlc  -l https://fqdn.com/world/getvitals is not working


Cause



 

The VRTSat Authentication debugging showed us that there is a problem with the credentials:

Sep 16 12:49:43 2010:50826,18,56,778488,1,debug,EAT2,1:      Error data: Expecting: RSA PUBLIC KEY
Sep 16 12:49:43 2010:50826,18,65,778488,1,debug,EAT2,1:      Error data: Unable to read pub key
Sep 16 12:49:43 2010:50826,18,83,778488,1,debug,EAT2,1:      Error data: Expecting: ANY PRIVATE KEY
Sep 16 12:49:43 2010:50826,18,92,778488,1,debug,EAT2,1:      Error data: Unable to read priv key
 


Solution



Enabling VRTSat Authentication debugging

First ensure xprtld is not running using ps

# ps –ef | grep xprtld

Once it is confirmed that xprtld is NOT running:

# export VRTSat_API_DEBUG_LEVEL=10
# export AtClientDebugLog=4:/tmp/at_debug.log
# /opt/VRTSsfmh/bin/xprtld /etc/opt/VRTSsfmh/xprtld.conf > /tmp/xprtld_start.log 2>&1


Wait 30 seconds


<ctrl-c>

Collect files, and send to SYMC :
/tmp/at_debug.log
/tmp/xprtld_start.log
 

Solution

1. Delete files under /var/VRTSat_lhc/
# rm /var/VRTSat_lhc/*

2. Move creds under /var/VRTSat/.VRTSat/profile/certstore/
# mkdir /tmp/backupcreds
# mv /var/VRTSat/.VRTSat/profile/certstore/*.0 /tmp/backupcreds/.

3. Delete pem keys
# rm /var/VRTSat/.VRTSat/profile/certstore/keystore/PubKeyFile.pem
# rm /var/VRTSat/.VRTSat/profile/certstore/keystore/PrivKeyFile.pem

4. Restart any shared broker process if running (/opt/VRTSat/bin/vxatd) (not running..so not required)

5. Do a localhost authentication to recreate the pem files we just deleted in step 4.
# export EAT_HOME_DIR=/opt/VRTSsfmh
# export EAT_DATA_DIR=/var/opt/VRTSsfmh/sec/root
# cd $EAT_HOME_DIR/bin
# ./vssat authenticate -d localhost

6. Start xprtld
# /opt/VRTSsfmh/adm/xprtldctrl start
 

 


Supplemental Materials

SourceETrack
Valuee2130590, e2145925


Article URL http://www.symantec.com/docs/TECH140387


Terms of use for this information are found in Legal Notices