With a base understanding of how to subscribe and validate out-of-band alerts, the next step is to designate the response. The Alert Rule Settings provides an interface to designate one of four core possibilities:
- Discard the alert
- Forward the alert to another management station
- Perform a designated TaskServer job based on the alert
- Start a Workflow process based on the alert (Note: This was added in 7.1)
In this article, the third option (TaskServer response) will be used in 2 separate examples to demonstrate automated responses to out-of-band alerts.
Alert Rule Settings are accessed via the Setting > All Settings menu selection at the top of the Symantec Management Console. The left window will look similar to the image below, once the Monitoring and Alerting menu is expanded.
Example 1: Notify the Administrator of a Password Attack
If an incorrect Intel AMT password is used multiple times to access the management engine of the client, an email notification can be sent to notify the administrator. This scenario can help to identify an improperly entered password used by an application, a possible incorrect configuration on the client, or other situation where incorrect passwords are preventing access to Intel AMT.
Using the Task Rules, a simple rule is defined as follows:
- Check if an SNMP alert
- Check if the SNMP alert has a severity of Critical
- Check if a predefined alert message of “Password Attack”
As shown in the example below, if the basic rule filter conditions are met a task will be performed. Responses can be selected based on available TaskServer jobs\tasks – whether the base template tasks or existing administer defined. This specific example will send an email notification.
Once the desired Task is defined, click Save. Next, ensure the desired filter is changed to “On” in the left column. The Notification Server is now ready to receive and respond to the Password Attack.
The next screenshot shows several windows within a single view. The Event Console is in the background and has been filtered to list all systems alerting on a Password Attack. The window on the left shows a specific Password Attack alert for the target test system. The window on the right, which is also the top or active window, shows the email template used to respond to the alert.
Note that the email message includes tokens or variables specific to the alert. After searching through various manuals and internet pages, the following list of available tokens was found:
- ALERTCATEGORYGUID
- ALERTDEFINITIONGUID
- ALERTGUID
- ALERTHOSTNAME
- ALERTMESSAGE
- ALERTPRODUCTGUID
- ALERTPROTOCOLGUID
- ALERTRESOURCEGUID
- ALERTSEVERITYLEVEL
- ALERTTIMESTAMP
- ALERTVARIABLE:variable_name
The final bullet item listed allows any alert variable to be used. For example, in the received email message shown below, the current IP address of the alerting client was obtained by used the token ALERTVARIABLE:SNMP::SendingHost. (Note: “Variable_Name” is exactly as stated in the screen. In this example, note that two colons - :: - between SNMP and SendingHost)
Knowing the exact IP address of the alerting client can prove very helpful in situations where Name-to-IP resolution is incorrect within the environment. This can commonly happen if the client was recently issued an IP address due to change of LAN segments or moving to wireless LAN. If the resulting IP address change has not been updated within the name resolution solution (i.e. DNS, WINS, etc), the received alert directly specifies the correct IP address.
In the example email below, the alert specific values in the email include: date, time, hostname, IP address, and alert message.
Example 2: If System Stopped Responding, Reboot and Notify Administrator
In this second example a scenario will be shown to notify the administrator of a system that has stopped responding and automatically reboot the system. The intent of the scenario is to show that multiple tasks can be encapsulated within a job. If any of the tasks utilize Intel AMT or other authenticated communication to the target client, the appropriate credentials must be specified.
The alert rule settings are shown below. The system hang event in this example was generated via the Sysinternals utility “NotMyFault” which causes the system to bluescreen. (see Part 1 for link and information). When this occurs, the management engine detects that the system is powered on yet a heartbeat check to the operating system fails to respond. To avoid a false positive, the alert rule waits until 3 missed heartbeat checks within a 2 minute time interval before triggering the automated response.
A custom job was defined to send a message to the administrator and then force a reboot of the client. This job was selected via the “Add Existing” option. The job is shown below with the Task Input details for “Reboot via AMT” selected to show that a Connection Credential Setting for Intel AMT protocol must be used.
Note: It is common to have the Windows Recovery option set to automatically reboot after a system failure. For the purposes of this article and demonstration, that option was unselected.
A custom email notification is sent to the administrator as shown in the example below:
The administrator and helpdesk may perform additional diagnostic tests to determine the underlying cause of the Operating System Hung event. The key part is that they are aware of the event within a few minutes of it occurring.
In a production environment, an alert subscription for this event might apply only to critical client systems within the environment. Testing and experimentation within individual environment is recommended.
In the next part of this series – an intriguing setup and response will be shown. The user of the target system will be able to notify the administrator out-of-band and request assistance. This notification can be sent via a pre-boot key sequence or via the host operating system (even if the network driver is not responding).
Return to Part 1
Continue to Part 3
The opinions expressed on this site are mine alone and do not necessarily reflect the opinions or strategies of Intel Corporation or its worldwide subsidiaries.