While searching on a large quantity of Index Volumes, the performance is extremely slow causing the User Interface to become unresponsive

Article:TECH156425  |  Created: 2011-03-24  |  Updated: 2013-07-15  |  Article URL http://www.symantec.com/docs/TECH156425
Article Type
Technical Solution

Product(s)

Environment

Issue



When the Accelerator product is performing a search on a very large quantity (typically in the tens of thousands) of Index Volumes, the Accelerator Service process consumes virtually all available memory on the Accelerator Server. The limited memory resources cause the User Interface to become unresponsive as well as volume processing to become extremely slow; in some cases, taking up to days to complete the search.

 


Error



Event Type: Error
Event Source: Accelerator Service Processor
Event ID: 93
Description:
APP AT - Customer ID: X - An error occurred while updating Vaults for Vault Store: ID=X KVS ID: 1234567890ABCDEF1234567890ABCDEF12345678test.com. System.OutOfMemoryException: The EnterpriseVault.DirectoryConnection object reported an error.
Ran out of memory

Event Type: Error
Event Source: Accelerator Service Processor
Event ID: 117
Description:
APP AT - Customer ID: X - Error adding Background SQL to the queue. System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.String.Concat(String str0, String str1, String str2)
   at Microsoft.Win32.RegistryKey.OpenSubKey(String name, Boolean writable)
   at System.Data.Common.ADP.LocalMachineRegistryValue(String subkey, String queryvalue)
   at System.Data.SqlClient.SqlConnectionString..ctor(String connectionString)
   at System.Data.SqlClient.SqlConnectionFactory.CreateConnectionOptions(String connectionString, DbConnectionOptions previous)
   at System.Data.ProviderBase.DbConnectionFactory.GetConnectionPoolGroup(String connectionString, DbConnectionPoolGroupOptions poolOptions, DbConnectionOptions& userConnectionOptions)
   at System.Data.SqlClient.SqlConnection.ConnectionString_Set(String value)
   at System.Data.SqlClient.SqlConnection.set_ConnectionString(String value)
   at KVS.Accelerator.Common.BackgroundSQL.InitializeComponent()
   at KVS.Accelerator.Common.BackgroundSQL..ctor()
   at KVS.Accelerator.Common.BackgroundSQLTaskQueue.QueueWork(ThreadSafeQueue& theQueue, Object myData).

Event Type: Error
Event Source: Accelerator Service Processor
Event ID: 137
Description:
APP AT - Customer ID: X - An error occured in ProcessingMultiQueues::QueueingThread. System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.Collections.Hashtable.ValueCollection.GetEnumerator()
   at KVS.Accelerator.Common.ProcessingMultiQueues.SignalQueuingFinishedEvent(Boolean Set)
   at KVS.Accelerator.Common.ProcessingMultiQueues.QueueingThread().

Event Type: Error
Event Source: Accelerator Service Processor
Event ID: 223
Description:
APP AT - Customer ID: X - An error occurred. System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.Data.SqlClient.TdsParser.ProcessEnvChange(Int32 tokenLength, TdsParserStateObject stateObj)
   at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
   at System.Data.SqlClient.SqlDataReader.ConsumeMetaData()
   at System.Data.SqlClient.SqlDataReader.get_MetaData()
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
   at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
   at System.Data.SqlClient.SqlCommand.ExecuteReader()
   at KVS.Accelerator.Production.ProductionQueue.DoCheckForNewProductionRun().

Event Type: Error
Event Source: Accelerator Service Processor
Event ID: 293
Description:
APP AT - Customer ID: X - Error adding Vault to Search in the queue. System.Data.SqlClient.SqlException: The query processor could not start the necessary thread resources for parallel query execution.
   at KVS.Accelerator.Search.SearchVaults.GetVaultToSearchDS(Int32 SearchID, VaultSearchStatus Status, Int32 IndexServerID)
   at KVS.Accelerator.Search.SearchVaults.GetVaultToSearchToAddToQueueDS(Int32 IndexServerID)
   at KVS.Accelerator.Search.SearchQueues.SearchVaultQueueWork(ProcessingMultiQueues TheProcessingMultiQueues, Object myData).

Event Type: Error
Event Source: Accelerator Service Processor
Event ID: 323
Description:
APP AT - Customer ID: X - An error has occured when initializing the Search Vaults. System.Data.SqlClient.SqlException: The query processor could not start the necessary thread resources for parallel query execution.
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
   at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
   at System.Data.SqlClient.SqlDataReader.HasMoreRows()
   at System.Data.SqlClient.SqlDataReader.ReadInternal(Boolean setTimeout)
   at System.Data.SqlClient.SqlDataReader.Read()
   at System.Data.Common.DataAdapter.FillLoadDataRow(SchemaMapping mapping)
   at System.Data.Common.DataAdapter.FillFromReader(DataSet dataset, DataTable datatable, String srcTable, DataReaderContainer dataReader, Int32 startRecord, Int32 maxRecords, DataColumn parentChapterColumn, Object parentChapterValue)
   at System.Data.Common.DataAdapter.Fill(DataTable[] dataTables, IDataReader dataReader, Int32 startRecord, Int32 maxRecords)
   at System.Data.Common.DbDataAdapter.FillInternal(DataSet dataset, DataTable[] datatables, Int32 startRecord, Int32 maxRecords, String srcTable, IDbCommand command, CommandBehavior behavior)
   at System.Data.Common.DbDataAdapter.Fill(DataTable[] dataTables, Int32 startRecord, Int32 maxRecords, IDbCommand command, CommandBehavior behavior)
   at System.Data.Common.DbDataAdapter.Fill(DataTable dataTable)
   at KVS.Accelerator.Search.SearchVaults.GetVaultToSearchDS(Int32 SearchID, VaultSearchStatus Status, Int32 IndexServerID).

Event Type: Error
Event Source: Enterprise Vault
Event ID: 40966
Description:
A program fault has raised an exception.
Exception: Exception of type 'System.OutOfMemoryException' was thrown.
Diagnostic:
Type: System.OutOfMemoryException
Reference:
Command Line: ".\AcceleratorService.exe" -URL:tcp://10.10.10.10:9501/81e60b38_5608_4cac_aede_e1bf0ac983b1/KVS.EnterpriseVault.ProcessManager12036987 -URL:ipc://EVAccelerator:AcceleratorManager/81e60b38_5608_4cac_aede_e1bf0ac983b1/KVS.EnterpriseVault.ProcessManager12036987 -CUSTOM:2|2|4
Application Domain: AcceleratorService.exe
Process Id: 4800
Thread Id: 9400
Stack Trace:    at System.Collections.Hashtable.HashtableEnumerator.get_Current()
   at System.Runtime.Remoting.Channels.SocketCache.TimeoutSockets(Object state, Boolean wasSignalled)
   at System.Threading._ThreadPoolWaitOrTimerCallback.PerformWaitOrTimerCallback(Object state, Boolean timedOut).

 

 


Solution



This issue has been addressed as part of the following release:


Hotfix for Symantec Enterprise Vault (EV) Discovery Accelerator (DA) 10.0, Build 10.0.0.1159
http://www.symantec.com/docs/TECH187335

Hotfix for Symantec Enterprise Vault (EV) Discovery Accelerator (DA) 10.0.1, Build 10.0.1.1069
http://www.symantec.com/docs/TECH187349


Supplemental Materials

SourceETrack
Value2273560
Description

Large quantity archive searches are performing extremely slow


SourceETrack
Value2273564
Description

The DA server is receiving Out Of Memory errors when attempting to update Volume status and running a blanket search on all archives.




Article URL http://www.symantec.com/docs/TECH156425


Terms of use for this information are found in Legal Notices