Hi Ireyes,

Thank you for the info you provided! I will indeed increase logging as you have suggested, hoping that if it occurs again, I am able to provide more info...

I've also opened a case with Symantec ( Case 414-760-213 )

They suggested the following (After receiving VxExplorer Logging). Maybe it can help others reading this as well:

The below message indicates that the service being monitored by the cluster is offline. This was not initiated by Veritas Cluster Server (VCS) and does not indicate a problem with VCS itself. VCS monitors the status of the service and triggers a fault if the items being monitored go offline unexpectedly. This is by design.

2011/05/22 05:51:21 VCS ERROR V-16-2-13067 (SRV0401) Agent is calling clean for resource(NetBackup_Server-Lanman) because the resource became OFFLINE unexpectedly, on its own.

2011/05/22 05:51:21 VCS INFO V-16-1-10307 Resource NetBackup_Server-Lanman (Owner: unknown, Group: NetBackup_Server) is offline on SRV0401 (Not initiated by VCS)

From the Lanman circular logs, that has debugging messages show the following;

2011/05/22 05:51:21 VCS DBG_21 V-16-50-0 Lanman:NetBackup_Server-Lanman:monitor:VLibNetIP::IsNetBTEnabled() returned 2, 0x0000006F

            LibVirtualName.cpp:VLibVirtualName::Check[601]

2011/05/22 05:51:21 VCS DBG_21 V-16-50-0 Lanman:NetBackup_Server-Lanman:monitor:vname.Check() returned 2, 0x0000006F

            LanmanAgent.cpp:CLanmanAgent::Monitor[412]

There is a windows API call (GetAdapterAddresses()) that’s returning that error. We call that routinely to get the network adapters information.

This error would only happen if there’s any change to the network adapters when that API call is made.

Were any changes made to the network settings at the times when these events occurred?

 Examples of changes are add/removing of IP addresses, changes to network settings, etc.

 I have also seen a change of the GPO which it may cause this particular windows API to return this error.

5/21/2011        10:33:18 PM   INFORMATION           1704(0x400006a8)  SceCli  srv0401.scrambled.somewhat     

Security policy in the Group policy objects has been applied successfully.

 Possible cause:

 A network config changes that might be occurring causing this particular windows API to return an error.

 Workaround:

 Set Lanman ToleranceLimit to 2

 The time it takes to detect a resource fault or failure depends on the MonitorInterval attribute for the resource type. When a resource faults, the next monitor detects it. The agent may not declare the resource as faulted if the ToleranceLimit attribute is set to non-zero. If the monitor entry point reports offline more often than the number set in ToleranceLimit, the resource is declared faulted. However, if the resource remains online for the interval designated in the ConfInterval attribute, previous reports of offline are not counted against ToleranceLimit.

When the agent determines that the resource is faulted, it calls the clean entry point (if implemented) to verify that the resource is completely offline. The monitor following clean verifies the offline. The agent then tries to restart the resource according to the number set in the RestartLimit attribute (if the value of the attribute is non-zero) before it gives up and informs HAD that the resource is faulted. However, if the resource remains online for the interval designated in ConfInterval, earlier attempts to restart are not counted against RestartLimit.  Chapter 22, VCS Performance Considerations Detecting System Failure 633

In most cases, ToleranceLimit is 0. The time it takes to detect a resource failure is the time it takes the agent monitor to detect failure, plus the time to clean up the resource if the clean entry point is implemented. Therefore, the time it takes to detect failure depends on the MonitorInterval, the efficiency of the monitor and clean (if implemented) entry points, and the ToleranceLimit (if set).

Fred