Searches remain in 'Not Started' or 'Searching' but never complete

Article:TECH53400  |  Created: 2007-01-10  |  Updated: 2012-11-08  |  Article URL http://www.symantec.com/docs/TECH53400
Article Type
Technical Solution

Product(s)

Environment

Issue



Searches remain in 'Not Started' or 'Searching' but never complete


Solution



Overview:
 
  • Searches that do not start can be caused by several factors:  Transaction logs restrictions, licensing, older searches that have never completed, or configured incorrectly.  
  • Searches that never complete are normally waiting on a previous search to release resources before the current search in question progresses.


Resolution:

Step 1: Verify there are no SQL database issues.

 
  • Tempdb database Data and Transaction logs are not full
  • Customer or Configuration database Data and Transaction logs are not full
  • Customer database indexes for the tables are not over 50% fragmented

Run the following SQL queries (if the Front Line Diagnostic script version 2 has already been run, look at the results)

DBCC SQLPERF(LOGSPACE)
DBCC SHOWCONTIG (tblIntDiscoveredItems) WITH ALL_INDEXES
DBCC SHOWCONTIG (tblProductionToDiscoveredItem) WITH ALL_INDEXES
DBCC SHOWCONTIG (tblDiscoveredItemToSearch) WITH ALL_INDEXES
DBCC SHOWCONTIG (tblSearchItems) WITH ALL_INDEXES


In the results below examine the Bold items

master                        0.4921875        68.25397           0
tempdb                      1.242188          92.20126           0
model                         0.4921875        55.55556           0
msdb                          0.7421875        56.31579           0
EVDAConfiguration      9.992188          17.26935           0
EVBADiscovery          299.9922          95.42372           0
EVMailboxVS2            79.99219          5.009034           0
EVJournalItems           79.99219          2.987963           0
EVCAConfiguration      9.992188          26.04574           0
EVBACompliance        299.9922          4.311799           0


DBCC SHOWCONTIG scanning 'tblIntDiscoveredItems' table...
Table: 'tblIntDiscoveredItems' (450100644); index ID: 1, database ID: 13
TABLE level scan performed.
- Pages Scanned...........................................: 54089
- Extents Scanned.........................................: 6802
- Extent Switches..........................................: 6994
- Avg. Pages per Extent.................................: 4.2
- Scan Density [Best Count:Actual Count].......: 38.89% [7:18]
- Logical Scan Fragmentation ....................: 72.22%
- Extent Scan Fragmentation .........................: 46.08%
- Avg. Bytes Free per Page............................: 1188.4
- Avg. Page Density (full)................................: 65.32%


The results from the first query for LOGSPACE indicates that both transaction logs space for the tempDB and the customer database EVBADiscovery are over 90%.  No more items will be added to these transaction logs and thus must be truncated or allocated additional space.

The results from the second query has a logical scan fragmentation of over 50% forcing the SQL server to use a table scan instead of the index scan.  This means a top to bottom, row by row scan to insert the new items into the table.  If the search as a large hit count this could take numerous hours to complete.  If this issue the issue, run the Symantec recommended SQL maintenance on the customer database (see TECH74666 in related articles below for details):  
  
  • Shrink Databases
  • Rebuild Indexes
  • Update Statistics

 




STEP 2:  Determine the cause of failure or delay (if not SQL related)

1. Verify there is not a licensing issue.  Examine the Event logs on the Accelerator server for Event ID: 201.  If you are receiving a 201 event, please contact your Symantec sales representative to obtain a correct license.

2. Determine the oldest search that never completed by running the following queries on the Compliance Accelerator or Discovery Accelerator Customer database.

SELECT TOP 10 c.Name AS 'Case/Dept Name', ints.Name AS 'Search Name', ints.SearchID, ints.RunDate
FROM tblIntSearches ints
INNER JOIN tblCase c ON ints.CaseID = c.CaseID
WHERE ints.StatusID NOT IN (1, 3, 4, 7, 8, 9, 10, 173)
ORDER BY ints.RunDate ASC
 
Paused Accelerator searches can prevent the release of worker threads for use by active searches, thus causing slow or no search processing.  The following SQL script can be run against each of the Accelerator customer databases to show 'Paused' searches:
 
SELECT c.Name AS 'Case/Dept Name', ints.Name AS 'Search Name', ints.SearchID, ints.RunDate
FROM tblIntSearches ints
INNER JOIN tblCase c ON ints.CaseID = c.CaseID
WHERE ints.StatusID IN (175, 176, 177, 178, 179)
ORDER BY ints.RunDate ASC
 
 
-- Status Codes
-- 175 = Paused
-- 176 = To Be Paused
-- 177 = Pausing
-- 178 = To Be Submitted
-- 179 = Resubmitting

Each paused search needs to be investigated and resolved. 

Note: The top row listed will be the oldest search.

3. Examine the oldest search using the information obtained from step 1.

4. Inspect the INFORMATION column for possible causes for the search to have never completed.
 

 
5. Resolve the issue that stopped the oldest search from completing.
 
  • if the resolution is not clear from the INFO column, then run the following queries against the Accelerator Customer database.  Replace the XX with the correct Search ID number.
SELECT SearchID, [Name], NumIndexVolsSearch, NumIndexVolsTotal
FROM tblIntSearches WHERE SearchID = XX

SELECT SearchID, SearchVaultID, NumHits, ExpectedNumHits, CurrentHitCount
FROM tblSearchVault WHERE StatusID IN (12, 18) AND SearchID = XX

 
  • The first query numbers should match.  This indicates that all volumes that are expected to be search have been searched
  • The second query set of three numbers should match.  This indicates that the expected number of hits accepted into the review set table match the current and expected hit counts.

6. In most cases restarting the Enterprise Vault Accelerator Manager will resolve the searches stuck in 'searching' status

7.  If the searches are 'Failed' or do not complete after restarting the Enterprise Vault Accelerator Service, they will need to be rejected or stopped and run again

 
a. From the Search Details from step 2, navigate to the search, click the Stop Search button or Reject button at the bottom of the webpage.
 
b. Next screen will be the Search Criteria page
 
c. To run the search again, click OK.
 
d. If the search is no longer desired,  click Close to remove the search
 

8. Run the SQL query again to verify the searches are no longer producing errors.

Step 3: Verify there isn't missing information in the configuration database

Verify that there is a configuration ID of "218" for each customer database. Run the following SQL Query on the Configuration database

SELECT * FROM tblConfiguration WHERE ConfigurationID = 218

There should be one for each customer and the Value must be 0.
If not, do the following.

1. Open the CA/DA client

2. Navigate to <Configuration> --> <Settings>

3. Hold the <CTRL> key and click on "Configuration Settings" to display hidden settings

4. Expand "Vault Directory Synchronization"

5. Look for "Upgrade Vault v5 Searches".  See Figure 1.

Figure 1
 

If the last modification column is blank, the System did not properly insert the value '0' into the configuration table.

6. Enable the option (select the Value check box), then disable the option (remove the check mark in the Value check box)

7. Select 'Save'.

8. Re-start the Enterprise Vault Accelerator Manager Service

9. Run the SQL Query again to see if there is a configuration ID for each customer, if not, the Configuration database is corrupt and a new Configuration database will need to be created.  See Creating a new Configuration database.

10. Test the search again, if it is still not working, create a new Configuration database

Creating a new Configuration database

1. Stop the Enterprise Vault Accelerator Manager Service (EVAMS).

2. In the following files located in "C:\Program Files\Enterprise Vault Business Accelerator\" edit these files..

AcceleratorManager.exe.config
AcceleratorManagerConsole.exe.config
AcceleratorService.exe.config
ADSynchroniser.exe.config

change the following line:

<add key="State" value="Installed" />

To:

<add key="State" value="Uninstalled" />

3. Start the EVAMS

4. Open the EVBAAdmin webpage and create a new configuration database.

5. Connect the existing Customer databases to the new Configuration database by 'Create new customer' and 'connect to the existing database'
 

 




Legacy ID



291019


Article URL http://www.symantec.com/docs/TECH53400


Terms of use for this information are found in Legal Notices