File System Archiving performance degrades when a large quantity of files are archived from a single target folder

Article:TECH201776  |  Created: 2013-01-17  |  Updated: 2013-01-22  |  Article URL http://www.symantec.com/docs/TECH201776
Article Type
Technical Solution


Issue



 When File System Archiving (FSA) is configured to archive from a single folder, containing a large quantity of files, archiving performance will degrade.


Error



A dtrace log capturing the EvFsaArchivingTask will show long run times of the GetArchivedItemPropertiesInBatch  stored procedure.  Times shown in the dtrace for “Time taken to execute StoreDB Query” are in Milliseconds.

13:22:49.481 EvFsaArchivingTask)  EV-L {STOREDB.EN_US} Executing stored procedure GetArchivedItemPropertiesInBatch 
13:24:00.746 (EvFsaArchivingTask)  EV-L {FILESYSTEMITEMPROCESSOR.EN_US} Time taken to execute StoreDB Query = 71261 

Cause



The issue will reveal itself when an FSA archive folder contains a large quantity of archived files and there is a large quantity of file to be archived to it.


Explanation:
During the FSA archiving process, new files are compared to what has already been archived from the specified folder.  The comparison of the file names is done within the SQL database.  When the destination folder within the archive contains a very large number files, the SQL query run time will increase proportionally to the number of items within the folder.  As more items are archived into the single destination folder, performance will degrade.  This behavior is dependent upon the processing power of the SQL server.
 
Example Scenario:
The EV FSA task targets the \\FSA_Server_Name\File_share and archives its contents into a single folder in the FSA archive.  2 Million Files have already been archived into the single folder within the FSA Archive and 100K+ messages still remain in the \\FSA_Server_Name\File_share  folder. Archiving performance begins to degrade as more files are archived.

Solution



Create a new logical folder structure within the target volume archive point and then move the files from the folder causing the issue into the new folder structure.  By distributing the files into the appropriate folder structure long query times may be avoided.





Article URL http://www.symantec.com/docs/TECH201776


Terms of use for this information are found in Legal Notices