Video Screencast Help

FSAUTIL - when complete ? (Urgent - sorry guys)

Created: 08 Feb 2013 • Updated: 14 Feb 2013 | 13 comments
This issue has been solved. See solution.

EV7.5sp2  Win2K3

 

Trying to do a bulk recall of our archived filesystem files to a new location using fsautil.  At completion of the command prompt, the bottom portion of output says "NOTE: File restore operation has been started".  There is no entry in the EV event log for this process starting or completing.

Anyone know what I should be looking for to determine when this has completed ?  At the moment I am watching the destination size grow but this is quite impractical.

 

Also, the command prompt produces an xml log file suggesting the files have been copied.  This is not the case - the files are yet to be copied (they're part of the submitted restore operation). Anyone know of how to log the actual restore process ?

 

Thanks

(NOTE: our config was really just for compression/diskspace savings - not planned for bulk restore/searching capabilities)

Comments 13 CommentsJump to latest comment

Rob.Wilcox's picture

Well, there are a few ways, but they might not be practical now - I mean that they might be too late for you.

 

One of the options on the command line is to change the logging so that instead of logging ONLY failures, it will log successes and failures. Also at the command line when it starts (before it says complete) is that you see it counts up the number of files.

So, if you enable the extra logging (warning: large file might be built) and you know how many files were counted then you can do a quick open of the XML file in something like notepad++ (which has line numbers) .. scroll to the bottom ...  and estimate then how many are done (because they'll be one file per line) versus how many you know need to be done..  and hey presto.

 

I don't think any events are logged anywhere.. but I could be wrong.. and I'm not in front of a server at the moment to verify.

Tony Uren's picture

Rob

Thanks for your input.

I am already using the   -l 0   option. It does create the xml as expected. But there is no logging for the restore/recall process itself. 

The end result of your suggestion is similar to what I'm doing now - I have to keep running a dir size / file count to determine if the process is complete.

 

We also use EV for mail archiving. I could monitor the message queues to see if email archiving process are comepleted or stopped. Is there anything similar for file system archiving ?  And on that, do you know the process for a file to be archived/recalled (similar to the email message queue process) ?

 

Thanks again

Rob.Wilcox's picture

FSA doesn't use MSMQ's for it's processing.

 

I don't know if the XML file gets written at the END of the process, or during.  I don't have sufficient data here at the moment to test a large recall operation to be able to answer the question.  Plus it's Saturday and I'm going out for the afternoon :)

 

For the recalling/archiving process - FSA doesn't use MSMQ's it's simple process to process communication, each process keeps it's own 'queue' of work operations to do.  For example when archiving the FSA Archiving task walks the folders and builds a list of files to archive, a number of threads then operate on that queue and start to archive them.  Periodically the task will update the checkpoint file (so it can resume next time).

 

Hope that helps,

HolgerMu's picture

I sometimes look for the CPU-Load on the Servers, when they significantly drop you will know something is finished.

It's also not a very smart Option but it already helped me sometimes.

 

Cheers,

Holger

------

Holger Mundt | I.Tresor GmbH & Co. KG | Germany | http://www.i-tresor.de

Rob.Wilcox's picture

By the way I've posted a little blog on my discoveries in this area.

You can't actually see the 'real' progress _at all_.  Well not in a particularly nice, easy, intuitive way.

 

http://www.symantec.com/connect/blogs/when-using-fsautility-bulk-recall-files-how-can-progress-be-tracked

Tony Uren's picture

Holger's response above got me thinking and I found that the process in question would seem to be EVFileSvrArcMngr.exe.  Monitoring this process seems to give a good indication of what the recall process is doing.

Tony Uren's picture

CPU cycles. Seems a much better way then doing a 'properties' on my destination to see if size/filecount is still increasing.

Rob.Wilcox's picture

Oh okay I see.

Well, very soon, they'll be a little app to 'help' with this sort of situation.

Nathan Clark 2's picture

Hi Tony you can Dtrace and use the filter:

 (FSAUtility)  {BulkRecall.WaitForBulkRecallThreadsToComplete} All item processor threads have completed their processing

 

or use PowerShell to simply check the attrubutes:

 $rt = GCI "E:\NateFSA\" -recurse |where {$_.attributes -match "ReparsePoint"}

 

 $rt.count
 
But its a good point and would be easy to add an event when done I will log this, and hopefully get it added soon.
 
Thanks
 
SOLUTION
Tony Uren's picture

Nathan,

Can you confirm the DTRACE process ?

Do I;

1. Open DTRACE

2. type set fsautility v

3. type filter

4. type BulkRecall.WaitForBulkRecallThreadsToComplete

5. type q

6. type log filepath

After running fsautility

7 type log and turn logging off

8. return fsautility logging to off

 

 

Tony Uren's picture

I have also encountered another problem

The data recalled using FSAUtil does not seem to be complete.

 

Using TECH189647 and totalling the OriginalSize(bytes) and Items columns, I get values that often do not correspond with the properties of the folder once copied.  It seems to match on small folders, but larger ones can differ significantly.

The xml log produced by FSAUtil would suggest that the copies were successful. I fail to see how this can be, seeing that this log file is created well before the completion of the recall job. It would seem that the xml log file is really only an indication of the files discovered to be to be recalled. As far as I can tell, there is no logging for the actual file recall, making it nigh on impossible to determine what files may have been missed.

Am I expecting too much from FSAUtil ?

Nathan Clark 2's picture

in the filter menu:

DT Filter>c i (to clear default includes)

DT Filter>i "WaitForBulkRecallThreadsToComplete or whatever you want to include"

also I just noticed your version EV7.5sp2 thats a bit old skool, and FSAUtility has vastly (and fixed) improved in the latter 9/10 versions.  But has everything recalled ok? check with Powershell or whatever.

Also if your just recalling you could also use powershell to recall something like:

PS> ls E:\FoolsGold -rec | where{$_.attributes -match "Offline"} | gc | out-null