Video Screencast Help

How To Configure VCS To Noify (or Take Action) When A DB Conifugration Has Changed?

Created: 23 Dec 2013 • Updated: 30 Dec 2013 | 5 comments
This issue has been solved. See solution.

An Oracle DB is part of a resource group along with all the filesystems it needs to start. Someone logs in to a db and sets an initiization parameter to point to a filesystem that is not part of the RG, for example, to address a log destination full. When a failover happens, db fails to come up because its log destination is pointing to a non-existent path on failover node.

I want to develope a script that checks these parameter settings and verifies. if one exists, that it is identical on all nodes e.g. the directory path is vallid on all.

Having skimmed through the VCS user guide, it appeas it can be one either using VCS events or notifier framework..

1) I couldn't find an event to capture this e.g. postonline, post offline. Ideally, I want this check to run regularly, may be, once a day.

2) If db runs on node 1, my script will retrieve these parameter settings e.g. directory path, Can it, however, check presence of this directory on all nodes of the cluster?

 

Thanks in advance.

Operating Systems:
Discussion Filed Under:

Comments 5 CommentsJump to latest comment

mikebounds's picture

A few things that may help:

  1. If you want to run something periodically using VCS then you would need to create a resource - this could be a dummy resource that just creates and monitors a file using the Application agent (see Application agent in bundled agent guide) and the monitor could check your Oracle parameters and issue an error message (using halog) if it finds an incorrect parameter.
    Use can use the MonitorInterval to determine how often to run the monitor which you may need to override (using hares -override) if you have other Application resources
     
  2. If you configure a notifier resource to send messages that are ERROR, then this will pick up the ERROR message and send to email/SNMP server depending on what you configure in the notifier resource
     
  3. If you want to run a command on another server then you could use ssh or you can use the VCS "hacli" command.
     
  4. The Virtual firedrill (sometimes called HA firedrill) is setup for what you are trying to acheive - i.e check your failover node is capable of running your service group.  You can find details on running a virtual firedrill in VCS admin guide ("Testing resource failover by using HA fire drills" section) and the Oracle agent guide lists what it currently checks:

 

getid  (Oracle agent)
Verifies that the Oracle Owner exists on the node.
 
home.vfd (Oracle agent)
Verifies the following:
  •  ORACLE_HOME is mounted on the node and corresponding entry is in the fstab.
    If the ORACLE_HOME is not mounted, the action entry point checks if any other resource has already mounted ORACLE_HOME.
  • Pfile is provided and it exists on the node.
  • Password file from $ORACLE_HOME/dbs/orapw[SID] is present.
owner.vfd (Oracle agent)
Verifies the uid and gid of the Oracle Owner attribute.
Checks if uid and gid of Owner attribute is the same on
the node where the Oracle resource is currently ONLINE.
 
pfile.vfd (Oracle agent)
Checks for the presence of pfile or spfile on the local disk.
If both pfile and spfile are not present, the agent function
exits. If the Oracle resource is online in the cluster, the
agent function logs a message that the spfile must be on
the shared storage because the Oracle resource is online.
 
tnsadmin.vfd (Netlsnr agent)
Checks if listener.ora file is present. If the listener.ora file
is not present, it checks if ORACLE_HOME is mounted
and displays appropriate messages.
So you could add your own virtual filedrill actions by adding a file with extension vfd to /opt/VRTSagents/ha/bin/Oracle/agents and then use hatype to modify SupportedActions attribute for Oracle agent.  Then run virtual firedrill periodically in cron or use a VCS resource (as in 1).  The virtual firedrill will also checks other things like mount points exists on both nodes.
 
Mike
 
 

 

UK Symantec Consultant in VCS, GCO, SF, VVR, VxAT on Solaris, AIX, HP-ux, Linux & Windows

If this post has answered your question then please click on "Mark as solution" link below

SOLUTION
Gaurav Sangamnerkar's picture

Hello,

I echo Mike on the first comment that if there is some check which needs to be periodic, you should go with an approach to create a resource itself either an actual resource (for change that has been made, in your e.g, a new FS being introduced to RG)

Ideally when someone is changing a parameter on DB, it should be in collaboration with platform or cluster team to ensure that new filesystem or any other relevant change is done in VCS configuration as well. This should part of change control board to ensure all relevant teams are engaged.

Secondly, I was thinking to put a preonline trigger for use. This option won't make daily checks however during an actual event of failure, it will ensure preonline scripts are executed before a resource group is brought online. This script can be a user defined script where you can check the parameters you wish. This will ensure that necessary checks are done before onlining a group. Refer below for preonline trigger:

https://sort.symantec.com/public/documents/sf/5.1/...

As Mike indicated you on using hacli commands, I believe you can put that in the script to check the parameters from other nodes of cluster & this script can either be used with preonline trigger OR you can put this script in cron to generate regular notifications.

Lastly, you can do some predictive analysis by using VCS Simulator as well, this is an offline method where you simply need to pull of configuration files & work with simulator to see if failover is successful or not. Refer below link for more details

https://sort.symantec.com/public/documents/sf/5.1/...

 

G

 

 

 

PS: If you are happy with the answer provided, please mark the post as solution. You can do so by clicking link "Mark as Solution" below the answer provided.
 

SOLUTION
cnewtonne's picture

This is great support. Thank you both.
Do you have to be root to run hacli. When using it as non-root, it prompts for a password. Can it be configured to run silently?

mikebounds's picture

 

From VCS 5.0, you have to be root to be able to use hacli to run commands.  In VCS. 4.1 you could use hacli to run commands for Cluster admins by setting HacliUserLevel to CLUSTERADMIN, so I guess the prompt for a password is a reminent which has not been cleaned up from 4.1 as you can't set HacliUserLevel to CLUSTERADMIN anymore.

If you need to run commands as non-root, then you will need to use ssh, or you could use VCS actions - i.e setup commands you want to run as an action (this can be a firedrill action with vfd extension or a normal action without vfd extension), then you can use hares -action to run action containing commands on any node in the cluster.

Mike

 

UK Symantec Consultant in VCS, GCO, SF, VVR, VxAT on Solaris, AIX, HP-ux, Linux & Windows

If this post has answered your question then please click on "Mark as solution" link below

Gaurav Sangamnerkar's picture

From VCS Admin guide

To run a command on any system in a cluster, Type the following command:

hacli -cmd command [-sys | -server system(s)]

Issues a command to be executed on the specified system(s). VCS must be running on the systems.
 

The use of the hacli command requires setting HacliUserLevel to at least COMMANDROOT. By default, the HacliUserLevel setting is NONE.
If the users do not want the root user on system A to enjoy root privileges on another system B, HacliUserLevel should remain set to NONE (the default) on system B.

You can specify multiple systems separated by a single space as arguments to the option -sys. If no system is specified, command runs on all systems in cluster with VCS in a RUNNING state. The command argument must be entered within double quotes if command includes any delimiters or options.

From the attributes section:

For HacliUserLevel cluster attribute, following attributes possible

This attribute has two, case-sensitive values:
NONE–hacli is disabled for all users regardless of role.
COMMANDROOThacli is enabled for root only.
Note: The command haclus -modify HacliUserLevel can be executed by root
only.
■ Type and dimension: string-scalar
■ Default: NONE

 

G

PS: If you are happy with the answer provided, please mark the post as solution. You can do so by clicking link "Mark as Solution" below the answer provided.