Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

Event id: 7288, 7284, 7180: Index Server warning and errors

Created: 21 Oct 2010 • Updated: 19 Nov 2010 | 6 comments
This issue has been solved. See solution.

We have Enterprise Vault 2007 productversion 7.5 SP6 running on a failover cluster with 4GB of RAM, archiving Exchange and several file system volumes and folders. 2 months ago, one of our index volumes entered the failed status.

At first we tried to update the index, but this wasn’t successful. After that we decided to rebuild the failed index. After a while the status didn’t report the failed status anymore, but kept on rebuilding the index until now. The eventlog “Enterprise Vault” keeps on reporting the same errors, followed by a warning.

First we have event id 7180:
Could not access indexable item: Not enough storage is available to complete this operation. [0x8007000e]
Internal reference: PFS/ce

Followed by event id 7284:
Failed to add item to the index due to failure loading the item's index metadata.
Reason: Could not access indexable item: Internal reference: [0xc0041c0c]
Index Id: 1C6DA0BED7336644C8E9599116799C3D31110000sa-kvs/Volume:903 (group$)
Saveset Id: 600766000000000~200709051744410000~0~5E45055F7DB346F099E95420C246725
Internal reference: II load

This is repeated twice and then followed bij event id 7288:
Addition of item to index abandoned.

Index: 1C6DA0BED7336644C8E9599116799C3D31110000sa-kvs/Volume:903 (group$)
Saveset Id: 600766000000000~200709051744410000~0~5E45055F7DB346F099E95420C246725
Sequence Number: 131977

All previous attempts to add the item to the index have failed. To prevent further failures the item will not be indexed.

These errors and warning are repeated every few seconds, reporting the same over and over again. It looks like rebuilding the index went into a loop and doesn’t seem to get out of it.

We looked at this article: http://www.symantec.com/business/support/index?page=content&id=TECH55404 and increased the pagefile as recommended, because it was set to small. We rebooted both nodes, Enterprise Vault reported the index as failed again and continued to rebuild. After a while the failed state disappeared and the event id’s mentioned above appeared again.

We also looked at this article:
http://www.symantec.com/business/support/index?page=content&id=TECH59638 and emptied the temp and brought the EVS services offline and online with the same result.

I’m out of known possibilities now. The only thing I wonder about, are a few files on that file store which are very big (3 GB) and happen to be plain text files (sql database dumps). They are archived, but take all the time in the world to be opened. Maybe rebuilding the index suffers from the same? I don’t know… But maybe you can help?
 

Comments 6 CommentsJump to latest comment

JesusWept3's picture

yeah its simply not going to work

check this thread which had the same issue
https://www-secure.symantec.com/connect/forums/index-failure

mickelingon's picture

Well, the problem wasn't solved really. I had to restore all the files and make new policies to see what extensions were making the index faile.
What I had to do to find the fileextensions that were srewing up the index:

In the IndexMissing.log file, locate the last series of numbers after the Z~B0D2BD499CB82C26BD607F5EF01D96C1 (example)
Then go to the SQL server and run this Query:
Notice the first WHERE parameter checks whether the saveset has an index sequence number.

SELECT SUBSTRING(dbo.SavesetProperty.Properties, CHARINDEX('', dbo.SavesetProperty.Properties) + 10, CHARINDEX('',
dbo.SavesetProperty.Properties) - (CHARINDEX('', dbo.SavesetProperty.Properties) + 10)) AS filename
FROM dbo.SavesetProperty INNER JOIN
dbo.Saveset ON dbo.SavesetProperty.SavesetIdentity = dbo.Saveset.SavesetIdentity INNER JOIN
dbo.Vault ON dbo.Saveset.VaultIdentity = dbo.Vault.VaultIdentity INNER JOIN
EnterpriseVaultDirectory.dbo.Root AS RT ON dbo.Vault.VaultID = RT.VaultEntryId INNER JOIN
EnterpriseVaultDirectory.dbo.Root AS ParentRoot ON RT.ContainerRootIdentity = ParentRoot.RootIdentity INNER JOIN
EnterpriseVaultDirectory.dbo.Archive AS ARCH ON ParentRoot.RootIdentity = ARCH.RootIdentity
WHERE (dbo.Saveset.IndexSeqNo IS NOT NULL) AND (ARCH.ArchiveName = 'Your archive name')

try it and se if it works.

Mikael

ArjenB's picture

I don't get the right output from the SQL-query:

Server: Msg 536, Level 16, State 3, Line 1
Invalid length parameter passed to the substring function.

Could there be something wrong with the query?

ArjenB's picture

I applied the following registry entries:

HKEY_LOCAL_MACHINE\SOFTWARE\KVS\Enterprise Vault\Indexing\PoisonPillCount
DWORD: 1
HKEY_LOCAL_MACHINE\SOFTWARE\KVS\Enterprise Vault\Indexing\MaxConsecutivePoisonPillItems
DWORD: 100000
HKEY_LOCAL_MACHINE\SOFTWARE\KVS\Enterprise Vault\Indexing\NumWordsToWrite
DWORD: 50000

And in the Archive properties, tab advanced, I set indexing to "Brief"

Then I started a simple update, and voila, after a few days the update finished and the index is not marked as failed anymore. However, Volume Information reports 3464 failed items of a total of 610669 items.

SOLUTION
JesusWept3's picture

Just as a suggestion to anyone reading this and thinking of applying this solution, please don't do it as there are major consequences to settings such as setting indexing levels to brief etc

ArjenB's picture

What are the major consequenses? Apart from searching the index with less possibilities? In our case this was the trigger to solve the problem. The registry settings alone were not sufficient.