Hotfix for Enterprise Vault (EV) 8.0 SP4, Build 1991, Cumulative Hotfix
|Article:TECH136842|||||Created: 2010-01-21|||||Updated: 2013-05-30|||||Article URL http://www.symantec.com/docs/TECH136842|
Hotfix for Enterprise Vault (EV) 8.0 SP4, Build 1991, Cumulative Hotfix
What issues does this hotfix resolve?
Which file(s) does this Hotfix replace?
How to install this Hotfix?
If experiencing the issue referenced in article 357007 in the Related Documents section apply the cumulative hotfix then apply the hotfix in article 357007.
NOTE: CUSTOMERS MUST TAKE A BACKUP OF THE ENTIRE ENVIRONMENT (All databases and file system data) PRIOR TO APPLYING THE HOTFIX
1. On each Enterprise Vault server, stop all Enterprise Vault Services and verify that all EV related processes have stopped by using Windows Task Manager to confirm this. Take note especially that all Storage related processes have stopped, such as StorageServer.exe and StorageFileWatch.exe
2. On each Enterprise Vault Server take a backup of the following files (if present) and deposit the backup in a separate location (not under the Enterprise Vault installation path):
On EnterpriseVaultDirectory Database and any Vaultstore Database make a copy of the following Stored Procedures:
3. On each Enterprise Vault Server copy the following files from the hotfix to the Enterprise Vault installation directory:
Files for SQL
4. Register the StorageOnline server as follows (run from the Enterprise Vault directory):
5. Register EVStgAPI.dll COM interface changes and proxy stub dlls as follows (on 32-bit systems):
C:\windows\system32\regsvr32.exe "C:\Program Files\Enterprise Vault\EVStgAPI.dll"
C:\windows\system32\regsvr32.exe "C:\Program Files\Enterprise Vault\StorageOnlineOpnsps.dll"
C:\windows\system32\regsvr32.exe "C:\Program Files\Enterprise Vault\AutoStorageOnlineOpns.dll"
(Note: on 64-bit systems, use C:\windows\SysWOW64\regsvr32.exe "...")
6. Run the Hotfix_2069692-DirectoryUpdate.sql script against the Directory database.
7. Run the Hotfix_2069692-VaultStoreUpdate.sql against each vault store database
- In order to activate folder creation, alter the call to uspcu_Vault by setting the input variable @CanCreateNewVault to 1
- This line is marked by the tag "<EDIT>"
- Deactivate folder creation once the desired folders have been created by setting the input variable @CanCreateNewVault to 0.
8. Restart all Enterprise Vault services and tasks.
9. To uninstall the fix, stop all the Enterprise Vault services, restore the original files, repeat steps 4 & 5 above.
10. Restore the following SQL Stored Procedures:
11. Restart all the services and tasks.
For EVSVR Usage:
**The following instructions represent an outline only and does not constitute a comprehensive recovery procedure that will cater for every scenario resulting from the use of EVSVR.**
1. Place the entire Vault Store Group(s) being repaired into BACKUP MODE.
** Failure to do so can affect the running of EVSVR and result in serious data loss**
If the Enterprise Vault services are not currently running then in order to place the Vault Store Group being repaired into BACKUP MODE then ONLY the Enterprise Vault Admin Service and Enterprise Vault Directory Service should be started. DO NOT START the Enterprise Vault Storage Service.
2. With the entire Vault Store Group being repaired still in BACKUP MODE, restart ONLY the following Enterprise Vault Services needed for the EVSVR repair operation:
Enterprise Vault Admin Service
Enterprise Vault Directory Service
Enterprise Vault Indexing Service (and all Enterprise Vault Indexing Services that relate to the archives in the Vault Store Group being repaired).
* typically the Enterprise Vault Storage Service is not required and should therefore not be started
3. With the entire Vault Store Group being repaired still in BACKUP MODE, perform the EVSVR operation Repair>RecreateMissingDBReferences. If you use deferred indexing (FSA), have a backlog of index operations outstanding on any archives, or any index rebuilds currently running, you may wish to deselect the "Require Index Entries" checkbox, as this checkbox controls whether database records are repaired based on the existence of index entries for the items.
4. With the entire Vault Store Group being repaired still in BACKUP MODE, after successful completion of the EVSVR operation Repair>RecreateMissingDBReferences repair operation, now perform the EVSVR operation Verify>Complete.
Any errors detected should be investigated. Depending on the nature of the errors it may be advisable to contact Support before proceeding.
5. With the entire Vault Store Group being repaired still in BACKUP MODE, consider removing some or all records from the relevant Vault Store database WatchFile and WatchSISPartFile tables depending on how old the restored copy of the Vault Store database was and whether or not Collections or Migrations are enabled. If the possibility exists that some records in the restored WatchFile table have since then been collected then it is recommended that the WatchFile and WatchSISPartFile records be removed. For further information, contact technical support.
6. All archive pending items in mailboxes will need to be cancelled and reverted to normal state.
7. Only when the state of the repaired databases have been backed up and their state is deemed to be acceptable should the remaining Enterprise Vault services be started and the system taken out of BACKUP MODE.
Known limitations of EVSVR
While every effort has been made to recover content from the converted content, certain document types do not lend themselves to recovery so well.
For example: Excel spreadsheets cannot be reliably recovered as the converted content contains only a "preview" of the spreadsheet.
EVDuplicateCleaner - Additional installation steps and Usage
Files for SQL
Files to be installed in Enterprise Vault Installation Directory
1. Close Outlook (if running) on the EV server.
2. Stop all Enterprise Vault services.
3. Backup the original versions of the files listed above if they exist.
4. Copy the supplied files listed above into the EV installation folder.
5. Duplicate Cleaner tool:
- Ideally this utility (EVDuplicateCleaner.exe) should be run from a storage server.
However it can in theory also run from any client machine as long as ECM API related and other dependent assemblies are present and that client machine has connectivity to the SQL server hosting Directory and Vault Store databases.
- Edit the EVDuplicateCleaner.exe.config setting the configurations values as detailed in the following section.
- Run the SQL script provided (VaultStoreDB_DuplicateTransaction.sql) with the hotfix ON EACH VAULTSTORE DATABASE TO BE PROCESSED.
6. Restart the Services.
- The Duplicate Cleaner utility can be run from the command prompt.
Execution/running Instructions for the Duplicate Cleaner tool
IMPORTANT: *** THIS UTILITY MUST BE RUN UNDER THE VSA ACCOUNT.***
The following are the various settings (in EVDuplicateCleaner.exe.config) that should be correctly configured before running this utility.
1. DirDBSQLServer - Mandatory, the name of the SQL Server hosting the Directory database.
2. MaxAllowedFailures - Maximum number of errors after which it will stop processing (default = unlimited) .
3. SavesetChunkSize - Number of duplicate savesets to fetch in one call to directory database (default = 10000).
4. ErrorToLogFile - Do we need to log errors to output log file (default = yes).
5. MaxDuplicateSavesetsToReport - Maximum number of duplicated savesets that will be processed in 'Report Mode' (default = 100000).
6. TryToOverrideDumspter- If dumpster is enabled then this decides if we want items to be hard deleted (i.e. not going to dumpster but deleted directly). This override will work only from V75 onwards. (default = true)
7. SkipLegacySavesets - If there are no legacy savesets then set this to True to skip even bothering looking for them.
The following are the supported commands for running this utility.
EVDuplicateCleaner.exe Execute[E,Ex,Exe,...,Executiv] <VaultStoreName> [<ArchiveEntryId>]
EVDuplicateCleaner.exe Report[R,Re,Rep,...,Repor] <VaultStoreName> [<ArchiveEntryId>]
EVDuplicateCleaner.exe Summary[S,Su,Sum,...,Summar] <VaultStoreName> [<ArchiveEntryId>]
Details of various commands
1. Execute - In this mode EVDuplicateCleaner.exe will delete all duplicate savesets. VaultStoreName is mandatory, ArchiveEntryId may also be provided in order to process one single archive.
2. Report - The output log will contain a list of duplicate savesets (by default maximum 100000) and a count of how many duplicate items each one has.
3. Summary - Will just output summary of duplicates per archive based on the supplied inputs.
Limitations of the Duplicate Cleaner tool
1. This utility uses the following attributes of a saveset to identify duplicates
Legacy (pre-Version 8.0) savesets:
ArchivePointIdentity, IdPartition, VaultIdentity (folder identity), IdChecksumHigh, IdChecksumLow, IdDateTime (creation date), IdUniqueNo
So the items which may have same binary content may not be treated as duplicate if they are not on the same partition, not in the same folder or do not have the same unique number.
Enterprise Vault Version 8.0 savesets:
ArchivePointIdentity,VaultIdentity (folder identity), IdDateTime (creation date), IdUniqueNo, SisPartCount
In addition to the above, each item's content fingerprint is checked. Items with different content fingerprints will not be treated as duplicate.
It should be noted that due to the reduced number of attributes that can be used from within the Vaultstore Database for the V8 savesets when determining the list of possible duplicate savesets there is a higher possibility for false positives to occur. Actual processing of the duplicate items will potentially result in less items being removed once the validation of the fingerprint value contained within the Index has been performed.
2. Maximum number of duplicate savesets processed in 'report mode' will be limited to value configured through 'MaxDuplicateSavesetsToReport' setting
(default = 100000).
3. This utility will not delete duplicate items that are on legal hold or on other compliance devices or if for similar reasons (retention category) delete is prohibited.
4. Running the utility and removing duplicates may have the effect of temporarily "breaking" shortcuts in user mailboxes as the saveset id that is contained in the shortcut may be incorrect if it points to a now removed duplicate.
This can be fixed by using the modified Shortcut Processing - see below.
5. It may take the index for each archive processed a while to catch up if lots of duplicates are removed - so you may still "appear" to have lots of duplicates in the web app search even after running the tool. Once the index has caught up these will disappear.
Enhancements to Shortcut Processing (Duplicate Cleaner tool)
!!BEFORE running this tool turn off the Journal mailbox task.!!
In certain conditions the duplicate removal tool may have the effect of breaking shortcuts.
After running EVDuplicateCleaner users may complain that they can no longer retrieve items from their mailbox shortcuts.
This does not affect archive explorer or the web search app, and shortcuts will still be expired according to site policy, however if users really require their shortcuts repaired then the following regkey can be used to activate the new "shortcut fixup" functionality:
Create the following registry value:
HKLM\SOFTWARE\KVS\Enterprise Vault\Agents\FixOrphanedShortcut - DWORD; set to 1 to enable new function.
* BE SURE TO REMOVE THE REGVALUE ONCE THE MAILBOXES HAVE BEEN FIXED UP - otherwise mailbox processing performance may be adversely affected *
Also: if the mailboxes contain duplicates of meeting requests, be sure to add "IPM.Schedule*" to the message classes supported by the site and
then check the same message class in the policy for the mailbox in question.
This regvalue has the effect of FORCING the new shortcut processing to use more intensive checks for orphaned shortcuts.
If a shortcut is deemed to be "orphaned" before it is finally removed a check is made to see if a duplicate saveset could exist that this shortcut can be fixed to point to. If that is the case then the shortcut will be fixed up to point to the duplicate of the original saveset. If there is any doubt about the duplicate saveset then the shortcut will not be fixed.
If no duplicate can be found then the shortcut will be deleted as orphaned.
To see the effect of the new shortcut processing BEFORE actually performing shortcut procesing run the mailbox archiving task in "Report" mode against any mailboxes you wish to fixup.
A report will be generated per-mailbox in the Reports folder beneath the EV install folder. It details either what would happen or what happened during shortcut processing.
Index addition fails due to Saveset Recombine issue
Unable to retrieve attachments from mailbox shortcut due to missing Centera Blob
Runtime error is occurring from StorageCrawler during indexing and from EVSVR causing indexing to hang.
EVSVR: A repair of the database resulted in incorrect refcounts (refcount > Totalcount)in the collection table
Large file recall fails due to StorageOnlineOpns
FP_OPTION_STREAM_STRICT_MODE can result in dataloss if disabled resulting in a Collections backlog
Requirement for an event to be generated when the number of recipients passed to indexing is less than the number of recipients on the original item when ingested using the CM API.
EVSVR reports a TotalCount < RefCount issue after having done a repair
Under some circumstances post-processing failed to convert items to shortcuts resulting in duplicates in the archive.
Mailbox archiving is failing to archive certain subfolders
Mailbox archives with very large folder hierarchies took a long time process.
When traversing the folder hierarchy an invalid pointer could point back to folder higher up in the tree resulting in endless loop.
Too many SQL records were being returned within a join of the Stored procedure causing performance issues.
Items on legalhold cause StorageDelete threads to fail during Dumpster expiry processing
When attempting to archive (manual or scheduled archiving), SMIME signed email goes to pending, then back to an unarchived item.
MAPI message items with invalid Original Message Class MAPI custom property value types are not archived correctly.
Article URL http://www.symantec.com/docs/TECH136842