Performance of indexing may degrade over time causing a large backlog of items waiting to be indexed

Article:TECH190302  |  Created: 2012-06-04  |  Updated: 2013-06-13  |  Article URL http://www.symantec.com/docs/TECH190302
Article Type
Technical Solution


Environment

Issue



After operating normally for some period, the performance of the indexing service may degrade and not be able to keep up with ingestion rates or respond to search requests in a timely fashion.

Indexer-service.exe processes may consume too much memory and become unresponsive.

 


Cause



The wildcard dictionary is a list of unique words in an individual index volume. This list is updated everytime an item is ingested.  If the size of this dictionary grows too large, the associated index may be unresponsive to search or ingestion requests. 


In the scenario described here, the wildcard dictionary was artificially large due to PGP encrypted attachments being converted to text and sent to the indexing engine.  This caused the indexing engine to add meaningless encrypted strings to the dictionary.  A 1 MB encrypted attachment would add an additional 1 MB of unique words to the dictionary.  This type of data should be excluded from conversion and not sent to the indexing engine.

The size of the wildcard dictionary can be checked by going to the index volume folder on disk (e.g. E:\index2\index1\1BAA95751C944234490D77ABDB567A218_858) and navigating to the live\expansions subfolder.   The wildcard dictionary will be the file without a file extension. 


It is expected this file should generally be under 2 GB.  
 


Solution



This issue has been addressed in the following releases:  


Enterprise Vault 10.0.1 Hotfix
http://www.symantec.com/docs/TECH195828

Enterprise Vault 10.0.2
http://www.symantec.com/docs/TECH147789


Supplemental Materials

SourceETrack
Value2807397
Description

Wildcard Dictionary bloated due to content from PGP Encrypted Messages




Article URL http://www.symantec.com/docs/TECH190302


Terms of use for this information are found in Legal Notices