Incidents with large comments cause slow performance, timeouts or out of memory errors in Helpdesk

Article:TECH145515  |  Created: 2010-12-03  |  Updated: 2011-08-04  |  Article URL http://www.symantec.com/docs/TECH145515
Article Type
Technical Solution

Issue



When using Helpdesk, slow performance, timeouts, or out of memory errors occur. These symptoms can be seen when using any area in Helpdesk, including creating or editing incidents or viewing reports and queries, such as the Worker Report. These symptoms may occur while low or high production is occurring or when using or not using incidents that specifically have large comments.


Error



Various. Error messages can include the following, which can be found in the Helpdesk console and/or in the Notification Server logs:

  • Exception - Altiris.Helpdesk.Services.ListException: An unexpected error has occurred. Exception of type System.OutOfMemoryException was thrown. ---> System.OutOfMemoryException: Exception of type System.OutOfMemoryException was thrown.
  • Request timed out.
  • Thread was being aborted. <SQL query>
  • Timeout expired. Timeout period elapsed prior to completion of the operation or the server is not responding. <stored procedure name>  

Cause



Helpdesk incidents that have comments exceeding 4,000 characters in even one incident can cause .NET to overflow and the Helpdesk database (Altiris_Incidents) to increase substantially in size. The more incidents that have this issue, the more frequent the slow performance, timeouts or out of memory errors will occur. This is why, initially, this issue may exist but not result in any problems, but later, after time, the issue occurs more and more frequently.

Note: Viewing the Helpdesk server's Windows Event Logs may show .NET errors, but if not, this is not necessarily an indication that this issue is not occurring or that this is not caused by large comments.

This issue will often occur when an incident is attempted to be accessed that has this issue, such as editing it or viewing it in a report, but can occur anywhere while using Helpdesk. When this issue is found, checking the Helpdesk database will usually result in hundreds of affected incidents. This is the number one cause of slow performance, timeouts and out of memory errors while using Helpdesk.

Large comments can be caused by:

  • Logs being sent in email to Helpdesk in the email body instead of as a file attachment or being saved in the Comments field while editing an incident.
  • HTML-formatted email sent to Helpdesk.
  • Email threads being sent to Helpdesk in the email body or being saved in the Comments field while editing an incident.
  • Incident or automation rules that use HDQUERY or otherwise directly access the Helpdesk workitem table to create, modify or delete incidents.
  • Incident or automation rules that are configured incorrectly and loop back in on themselves by repeatedly adding the same comments over and over again.
  • Notify rules that loop back in on themselves by using the Helpdesk email address as a worker or contact, thereby repeatedly re-sending the same email over and over again.

Solution



Truncate large comments by using a SQL query and then add Helpdesk business rules to prevent these from re-occurring.

  1. In SQL Sever Management Studio, run the attached SQL query "Find Helpdesk Incidents With Large Comments.sql" to determine if large comments are present or not. If large comments are found, continue following these steps. If large comments are not found, refer to the following related article for additional troubleshooting:

    Slow performance, timeouts, out of memory errors or high CPU/w3wp.exe/IIS usage occurs when using Helpdesk
    http://www.symantec.com/business/support/index?page=content&id=TECH146510
     
  2. In SQL Sever Management Studio, truncate existing large comments by running the attached SQL query "Truncate Large Comments in Helpdesk Incidents.sql".
  3. In Helpdesk, import the business rules and e-mail template from the attached "Prevent Large Comments Business Rules.xml" file.

    a. In the Helpdesk console, click on the Incidents tab (if using the Altiris Console 6.5, instead click on the Tools menu > Helpdesk).
    b. Click on the Admin menu > Import Admin Data.
    c. Click on the Browse button.
    d. Navigate to where the file "Prevent Large Comments Business Rules.xml" was saved to, select it, and then click on the Open button.
    f. Click on the Open button.
    g. Click on the Import button.
    h. Under Admin > Validation Rules > List Validation Rules, the new "Prevent Saving an Incident if it Contains Large Comments" validation rule is now active. Using this rule will prevent users from saving an incident that has large comments until they reduce it size.
    i. Under Admin > Incident Rules > List Incident Rules, the new "Truncate Large Comments in E-mail Sent as Helpdesk Incidents" incident rule is now active.  Using this rule will truncate large message bodies sent as e-mail to Helpdesk as incidents.
    j.  Under Admin > Notify Rules > List Notify Rules, the new "Alert a Helpdesk Administrator if an Incident Contains Large Comments" notify rule is now available. This is inactive by default. Using this instead of the validation and incident rules will send an email alert to a Helpdesk Administrator to enable them to manually correct the issue.
    k. Under Admin > E-mail Templates > List E-mail Message Templates, the new "Send an E-mail Alert to a Helpdesk Administrator if an Incident Contains Large Comments" e-mail template is now active. This is used in conjunction with the inactive (by default) "Alert a Helpdesk Administrator if an Incident Contains Large Comments" notify rule.
     
  4. If the issue still occurs after the above steps have been followed, the Helpdesk and/or SQL Server may need to be rebooted in order to release stuck resources.

Warning: If the solution documented in this article is not able to be fully implemented, such as not agreeing to truncate large comments, Symantec Technical Support may be unable to provide additional assistance as the full solution is required to resolve the issue. The only other workarounds are listed below:

  • An alternate solution is to delete a range of incidents. This can be helpful if many contain large comments or there are 500,000 or more records in the workitem table and there is no concern over deleting these incidents.
  • Archive incidents that have large comments. The following article describes how to archive Helpdesk incidents:

    How to archive incidents in Helpdesk
    http://www.symantec.com/business/support/index?page=content&id=HOWTO53151

Related Articles

Error "An unexpected error has occurred. Exception of type System.OutOfMemoryException was thrown." occurs or Helpdesk times out when trying to create, edit or view an incident
http://www.symantec.com/business/support/index?page=content&id=TECH25700

Error "Request timed out" occurs in the Helpdesk console
http://www.symantec.com/business/support/index?page=content&id=TECH166341

Error "Thread was being aborted. <SQL query>" occurs in the Notification Server logs after timeouts occur in Helpdesk
http://www.symantec.com/business/support/index?page=content&id=TECH159340

Error "Timeout expired. Timeout period elapsed prior to completion of the operation or the server is not responding. <stored procedure name>" occurs in the Notification Server logs after timeouts occur in Helpdesk
http://www.symantec.com/business/support/index?page=content&id=TECH159341




Article URL http://www.symantec.com/docs/TECH145515


Terms of use for this information are found in Legal Notices