Performance of indexing may degrade over time causing a large backlog of items waiting to be indexed
|Article:TECH190302|||||Created: 2012-06-04|||||Updated: 2013-06-13|||||Article URL http://www.symantec.com/docs/TECH190302|
After operating normally for some period, the performance of the indexing service may degrade and not be able to keep up with ingestion rates or respond to search requests in a timely fashion.
Indexer-service.exe processes may consume too much memory and become unresponsive.
The wildcard dictionary is a list of unique words in an individual index volume. This list is updated everytime an item is ingested. If the size of this dictionary grows too large, the associated index may be unresponsive to search or ingestion requests.
In the scenario described here, the wildcard dictionary was artificially large due to PGP encrypted attachments being converted to text and sent to the indexing engine. This caused the indexing engine to add meaningless encrypted strings to the dictionary. A 1 MB encrypted attachment would add an additional 1 MB of unique words to the dictionary. This type of data should be excluded from conversion and not sent to the indexing engine.
The size of the wildcard dictionary can be checked by going to the index volume folder on disk (e.g. E:\index2\index1\1BAA95751C944234490D77ABDB567A218_858) and navigating to the live\expansions subfolder. The wildcard dictionary will be the file without a file extension.
It is expected this file should generally be under 2 GB.
Wildcard Dictionary bloated due to content from PGP Encrypted Messages
Article URL http://www.symantec.com/docs/TECH190302