Incidents with large comments cause slow performance, timeouts or out of memory errors in Helpdesk
|Article:TECH145515|||||Created: 2010-12-03|||||Updated: 2011-08-04|||||Article URL http://www.symantec.com/docs/TECH145515|
When using Helpdesk, slow performance, timeouts, or out of memory errors occur. These symptoms can be seen when using any area in Helpdesk, including creating or editing incidents or viewing reports and queries, such as the Worker Report. These symptoms may occur while low or high production is occurring or when using or not using incidents that specifically have large comments.
Various. Error messages can include the following, which can be found in the Helpdesk console and/or in the Notification Server logs:
Exception - Altiris.Helpdesk.Services.ListException: An unexpected error has occurred. Exception of type System.OutOfMemoryException was thrown. ---> System.OutOfMemoryException: Exception of type System.OutOfMemoryException was thrown.
Request timed out.
Thread was being aborted. <SQL query>
Timeout expired. Timeout period elapsed prior to completion of the operation or the server is not responding. <stored procedure name>
Helpdesk incidents that have comments exceeding 4,000 characters in even one incident can cause .NET to overflow and the Helpdesk database (Altiris_Incidents) to increase substantially in size. The more incidents that have this issue, the more frequent the slow performance, timeouts or out of memory errors will occur. This is why, initially, this issue may exist but not result in any problems, but later, after time, the issue occurs more and more frequently.
Note: Viewing the Helpdesk server's Windows Event Logs may show .NET errors, but if not, this is not necessarily an indication that this issue is not occurring or that this is not caused by large comments.
This issue will often occur when an incident is attempted to be accessed that has this issue, such as editing it or viewing it in a report, but can occur anywhere while using Helpdesk. When this issue is found, checking the Helpdesk database will usually result in hundreds of affected incidents. This is the number one cause of slow performance, timeouts and out of memory errors while using Helpdesk.
Large comments can be caused by:
- Logs being sent in email to Helpdesk in the email body instead of as a file attachment or being saved in the Comments field while editing an incident.
- HTML-formatted email sent to Helpdesk.
- Email threads being sent to Helpdesk in the email body or being saved in the Comments field while editing an incident.
- Incident or automation rules that use HDQUERY or otherwise directly access the Helpdesk workitem table to create, modify or delete incidents.
- Incident or automation rules that are configured incorrectly and loop back in on themselves by repeatedly adding the same comments over and over again.
- Notify rules that loop back in on themselves by using the Helpdesk email address as a worker or contact, thereby repeatedly re-sending the same email over and over again.
Truncate large comments by using a SQL query and then add Helpdesk business rules to prevent these from re-occurring.
- In SQL Sever Management Studio, run the attached SQL query "Find Helpdesk Incidents With Large Comments.sql" to determine if large comments are present or not. If large comments are found, continue following these steps. If large comments are not found, refer to the following related article for additional troubleshooting:
Slow performance, timeouts, out of memory errors or high CPU/w3wp.exe/IIS usage occurs when using Helpdesk
- In SQL Sever Management Studio, truncate existing large comments by running the attached SQL query "Truncate Large Comments in Helpdesk Incidents.sql".
- In Helpdesk, import the business rules and e-mail template from the attached "Prevent Large Comments Business Rules.xml" file.
a. In the Helpdesk console, click on the Incidents tab (if using the Altiris Console 6.5, instead click on the Tools menu > Helpdesk).
b. Click on the Admin menu > Import Admin Data.
c. Click on the Browse button.
d. Navigate to where the file "Prevent Large Comments Business Rules.xml" was saved to, select it, and then click on the Open button.
f. Click on the Open button.
g. Click on the Import button.
h. Under Admin > Validation Rules > List Validation Rules, the new "Prevent Saving an Incident if it Contains Large Comments" validation rule is now active. Using this rule will prevent users from saving an incident that has large comments until they reduce it size.
i. Under Admin > Incident Rules > List Incident Rules, the new "Truncate Large Comments in E-mail Sent as Helpdesk Incidents" incident rule is now active. Using this rule will truncate large message bodies sent as e-mail to Helpdesk as incidents.
j. Under Admin > Notify Rules > List Notify Rules, the new "Alert a Helpdesk Administrator if an Incident Contains Large Comments" notify rule is now available. This is inactive by default. Using this instead of the validation and incident rules will send an email alert to a Helpdesk Administrator to enable them to manually correct the issue.
k. Under Admin > E-mail Templates > List E-mail Message Templates, the new "Send an E-mail Alert to a Helpdesk Administrator if an Incident Contains Large Comments" e-mail template is now active. This is used in conjunction with the inactive (by default) "Alert a Helpdesk Administrator if an Incident Contains Large Comments" notify rule.
- If the issue still occurs after the above steps have been followed, the Helpdesk and/or SQL Server may need to be rebooted in order to release stuck resources.
Warning: If the solution documented in this article is not able to be fully implemented, such as not agreeing to truncate large comments, Symantec Technical Support may be unable to provide additional assistance as the full solution is required to resolve the issue. The only other workarounds are listed below:
- An alternate solution is to delete a range of incidents. This can be helpful if many contain large comments or there are 500,000 or more records in the workitem table and there is no concern over deleting these incidents.
- Archive incidents that have large comments. The following article describes how to archive Helpdesk incidents:
How to archive incidents in Helpdesk
Error "An unexpected error has occurred. Exception of type System.OutOfMemoryException was thrown." occurs or Helpdesk times out when trying to create, edit or view an incident
Error "Request timed out" occurs in the Helpdesk console
Error "Thread was being aborted. <SQL query>" occurs in the Notification Server logs after timeouts occur in Helpdesk
Error "Timeout expired. Timeout period elapsed prior to completion of the operation or the server is not responding. <stored procedure name>" occurs in the Notification Server logs after timeouts occur in Helpdesk
Article URL http://www.symantec.com/docs/TECH145515