Video Screencast Help

Snapshot Explanation!

Created: 05 Sep 2007 • Updated: 21 May 2010 | 8 comments
Hello All,
 
I am sure that among you there is someone out there that can demistify the whole snapshot subject for me by answering my questions. Here goes!
 
1. Is the .vsp file that is created when using VSP thye virtual volume that is referenced in the documentation or is it the VSP cache file.
2. What does the vsp file hold? Is it a compressed backup of the data to be backed up?
3. How can the snapshot complete so quickly if it taking a full backup of the volume?
 
As I'm sure you can gather I am having a really difficult time understanding this and would be very grateful for any pointers and help on the subject.
 
Thanks in advance.

Comments 8 CommentsJump to latest comment

nbuw1980's picture
Here is my understanding of the process with VSP...
 
When backing up a client:
 
1. VSP takes a snapshot and creates a virtual volume constructed from the metadata it gathers on
the current state of that volume.
2. NBU uses the virtual voume to perform a backup of the actual volume.
3. The VSP cache file is used whilst the backup is being done to act as a buffer and soak up any
read or write operations to ensure that the data on the volume is in sync with the virtual
volumes metadata about that volume.
4. VSP related snapshot files and cache files are removed from the disk.
 
Hope you found this helpful.
sdo's picture
A Windows 2003 NetBackup client typically has two types/methods of snapshot available:
VSP by Veritas/Symantec (you can choose not to install this by using the silentclient.cmd to create your own custom client install script - because the installation of VSP always requires a reboot, whereas if you don't install VSP, you can use the NetBackup client right away).
VSS by Microsoft, which I hear some people seem to have better results with on Windows 2003.
 
Both VSP and VSS are not used, and cannot be used, to take backups of databases, e.g. Exchange, SQL, Domino etc...  For these you either close the app down and secure using the plain client (i.e. NTFS disk backup), or you purchase the appropriate NetBackup database client "extension/option".  Incidentally, the licencing model for v6.5 has changed, and now a "database extension" licence can be used to backup any of the supported databases.
 
VSP and VSS are not snapshot tools in the old sense of the word.  What they both actually do is to save the old/original copy of data as updates are made to disk by other programs/systems.  I.e. when other applications write to disk, then the O/S is aware that the disk is being backup and that VSP/VSS is being used, and so the O/S performs a read of what is about to be overwitten and saves a copy of the "old" data (in the snapshot) before completing the disk write for the other app.  This way the NetBackup client can still backup what the disk originally looked like because the O/S I/O disk subsystem "rebuilds" the data, i.e. the NetBackup client is none the wiser, and thus backs-up/saves a copy of the original data when it comes to reading a file that has changed since the backup started.  Also, this means that the cached data-set can simply be discarded at the end of the backup.
 
My tips/notes (for VSP mostly, and a little for VSS):
 
1) The VSP cache file for a disk is always stored on another local disk.  If you have a system with only one local drive, then VSP will never be used.  I'm not sure about VSS.  (Can anyone else comment please?)
 
2) VSP cache files will never be saved to network drives, and VSP cannot be used to secure network drives.  I assume the same is true for VSS.
 
3) If you are using ALL_LOCAL_DRIVES and no multi-streaming then you don't need to worry about controlling how many backup streams run at once... but if you specify folders or disks separately or use ALL_LOCAL_DRIVES plus multi-streaming then you may want to look at specifying the maximum amount of concurrent backup jobs that the client can run at once (by using the "Client Attributes" tab of the master server properties).  The reason I say this is more for VSP (but I suspect the same is probably true for VSS)... for example, you have a big fast client with six local drives in seperate spindles/buses etc and the client could easily support all six disks being backed-up at once - but if you use VSP then the NetBackup client/VSP will not have anywhere to store a VSP cache file, because a VSP cache file (for any given disk) cannot be stored on any other disk for which VSP is being used to cache.  Makes sense?  So, in this example I would limit the number of streams to half the number of local drives, i.e. three backups - this way all backup jobs will each have another drive (that is not being backed up), upon which to store the VSP cache file.
 
4) VSP and/or VSS is not a way to read through locked files.  If a file is really locked down by an app or O/S then the NetBackup client is still not going to be able to read the file whether VSP or VSS is used or not.
 
5) Enabling VSP on a NetBackup client does not necessarily mean that it will be used.  The O/S (i.e. the VSP driver) needs to ensure that a disk is not being written to before it can insert itself into the I/O sub system.  I mean that the VSP driver needs to see specific period of quiet (no writes).  If the VSP driver does not experience the quiet period then VSP gives up and is not used, and the NetBackup client continues its backup without the use of VSP.  You can specify how long the quiet period needs to be, and you can specify how long to wait for the sufficiently long quiet period (see next item below).
 
6) By default VSP cache files can grow up to a max of 30GB.  I always configure the "VSP" tab in the client "Host Properties".  My preferred default settings are;
   - (no included or excluded drives)
   - specify size in MB
   - initial size 500 MB
   - maximum size 3000 MB
   - Leave the "busy file wait" at five seconds.  (The name of this item "busy file wait" is not very accurate/helpful, what it actually specifies is the length of time that the VSP driver should wait for a disk to be quiet (no writes) before interjecting in the I/O sub-system, i.e. actually turning on VSP.  This is a "quiesce wait", i.e. it specifies how long a period of time must pass of no disk writes.
   - "VSP busy timeout" 120 seconds (the default is 300 seconds).  This specified how long to wait for the "5 seconds of quiet" (above).  To my mind, if a disk isn't quite for a period of 5 seconds during a two minute window, then the disk probably isn't going to be quite for 5 seconds during a five minute window.   I'd rather VSP gave up sooner and let the NetBackup client move on and take the backup without VSP. 
   - I also add the client to the master server host properties "Client Attributes" tab and and then for the client in the "Windows Open File Backup" tab, for the "Snapshot error control" select "Disable Snapshot and continue" - so that backups during which VSP fails will at least try to continue.
On one or two busy clients (out of 150+) I have set max to 5000 MB to get it to work.
 
7) Always exclude the VSP cache files from anti-virus scanners.  This is the single biggest problem most people/sites come up against.  The trouble is that the "naming" of the VSP cache file has changed with several times with the different NetBackup client versions.  Here's what I have found so far:
   v4.5                   *:\VSPCache*.VSP 
   v5.1                   *:\_vxfiVspCacheFile_*.tmp
   v5.1 MP3aS2+  *:\_vxfiVspCacheFile_*.VSP
   v6.0                   *:\NBU_VSP_Cache\_vxfiVspCacheFile_*.vsp
   v6.5                   (Can anyone else confirm naming convention for v6.5 clients, please?)
Sometimes VSP cache files get left behind, and you don't want to back them up, so I usually add the above to each clients NetBackup exclusion list as well as the anti-virus scanner exclusion list.
 
8) I used to recommend adding drive letter "C" to the list of drives to be excluded from VSP, in the client "Host Properties" - but these days I would only consider it if the C drive was very low on disk space, and/or was constantly being thrashed or updated (written to) by other apps.  I.e. these days I let the C drive be used to host a.n.other volume's VSP cache file.   If you have other disk volumes that are constantly being updated by other apps you may want to consider excluding those particular drives too - because the VSP cache files will get very big quite quickly trying to hold all the old/original data that once existed.
 
9) Sometimes VSP cache files get left behind from failed snapshots and/or failed backups.  Sometimes these files cannot be deleted because the I/O kernel is still holding on to them.  A reboot will get rid of the lock, but doesn't always cause the file to be discarded - sometimes you still need to manually delete it after a reboot.  Supossedly the O/S's lock on a VSP cache file can be broken/severed by using a Sys Internals tool (now owned by MS) - but personally I'd have to be pretty desperate before doing that on a production system - hence I always configure a VSP maximum size.
 
10) I have a DOS script that can scout around Windows clients looking for old/orphaned VSP cache files.  Let me know if you want a copy.
 
11) MS Windows VSS does have a DOS command line enquiry tool, named "vssadmin" in WinXP and Win2003.  (Does anyone know if/how VSS is configurable?)
 
HTH,
Dave.



Message Edited by sdw303 on 09-05-2007 11:21 PM

Stumpr2's picture
Thank you! Thank you! Thank you!
 
I wish we had a FAQ to stash postings like this into for future reference.
Great work!!
 
 
 

VERITAS ain't it the truth?

d_roble's picture
In response sdw303's wonderful explanation of VSS and VSP, I thought I would add by answering #11
 
11) MS Windows VSS does have a DOS command line enquiry tool, named "vssadmin" in WinXP and Win2003.  (Does anyone know if/how VSS is configurable?)
 
I did find some instruction on how to configure VSS in Windows2003.  It can be found here http://seer.entsupport.symantec.com/docs/276739.htm
 
Regards,
 
Dale
Omar Villa's picture
I just copy this to my notes, is awsome.

Omar Villa

Netbackup Expert

Twiter: @omarvillaNBU

Daniel Dilbeck's picture
I would like to get the script you have for deleting the old files.
sdo's picture
The script doesn't delete the files - it just identifies them.
I use this DOS script in an NBU ES v5.1 MP6 environment on Windows 2000 SP4.
 
I haven't coded it to find v6.x VSP cache files, as we're still v5.1.  Let me know if you need it modified for v6.x.
 
I run it on the master server usually.
 
Code:
@echo offsetlocal enabledelayedexpansionREM For DOS on Windows 2000 SP4, NetBackup ES v5.1 MP6.REM Does not currently identify v6.x VSP cache files.bpdir -— 1>NUL 2>NULif /i "%errorlevel%"=="9009" (  echo !time:~0,8!  Unable to locate BPDIR command, script exiting...  goto:eof)set z_started=!time:~0,8!set z_here=%~dp0set z_name=%~n0set z_data=!z_here!data\set z_file_cli=!z_name!.cliset z_file_csv=!z_name!.csvif exist "!z_file_cli!" del "!z_file_cli!"if exist "!z_file_csv!" del "!z_file_csv!"echo Client,Drive,DateTime,Size,File,KB,MB > "!z_file_csv!"echo !time:~0,8!echo !time:~0,8!  Started:          !z_started!echo !time:~0,8!  Folder:           !z_here!echo !time:~0,8!  Data:             !z_data!echo !time:~0,8!  Script:           !z_name!if not exist "!z_file_cli!" (  echo !time:~0,8!  echo !time:~0,8!  Listing clients...  bpplclients -allunique -noheader > !z_file_cli!)if not exist "!z_data!" (  echo !time:~0,8!  echo !time:~0,8!  Creating data folder...  mkdir "!z_data!")set z_clients_tot=0set z_clients_win=0set z_clients_checked=0set z_clients_not=0set z_drives_tot=0set z_caches_tot=0echo !time:~0,8!echo !time:~0,8!  Checking Windows clients...for /f "tokens=1,2,3" %%a in (!z_file_cli!) do (  set /a z_clients_tot+=1  set z_platform=%%a  set z_os=%%b  set z_client=%%c  set  z_os=!z_os:~0,3!  if /i "!z_os!"=="win" call :process-client)set z_finished=!time:~0,8!echo !time:~0,8!echo !time:~0,8!  Total clients:        !z_clients_tot!echo !time:~0,8!  Windows clients:      !z_clients_win!echo !time:~0,8!  Clients checked:      !z_clients_checked!echo !time:~0,8!  Clients not checked:  !z_clients_not!echo !time:~0,8!echo !time:~0,8!  Drives:               !z_drives_tot!echo !time:~0,8!  Cache Files:          !z_caches_tot!echo !time:~0,8!echo !time:~0,8!  Started:              !z_started!echo !time:~0,8!  Finished:             !z_finished!pauseexit /bREM ############################################################:process-clientset /a z_clients_win+=1set z_ignore=nREM if /i "!z_client!"=="xxxx" set z_ignore=yREM For testing...REM if /i not "!z_client!"=="xxxx" set z_ignore=yif /i "!z_ignore!"=="y" (  echo !time:~0,8!  Ignoring client:  !z_client!  set /a z_clients_not+=1  echo !z_client!,ignored >> "!z_file_csv!"  goto :eof)echo !time:~0,8!  Checking client:  !z_client!for /f "tokens=1" %%a in ('ping -n 1 !z_client!') do (  if /i "%%a"=="request" (    echo !time:~0,8!  Client: !z_client! does not ping...    set /a z_clients_not+=1    echo !z_client!,no ping >> "!z_file_csv!"    goto :eof  ))set /a z_clients_checked+=1set z_file_txt=!z_data!!z_name!-!z_client!.txtif not exist "!z_file_txt!" (  bpdir -M !z_client! > "!z_file_txt!")for /f "tokens=1,2" %%a in (!z_file_txt!) do (  set z_drive=%%b  set z_letter=!z_drive:~0,1!  if /i "!z_drive!"=="!z_letter!" (    if /i not "!z_drive!"=="A" (      if /i not "!z_drive!"=="Z" (        call :process-drive      )    )  ))goto :eofREM ############################################################:process-driveset /a z_drives_tot+=1echo !time:~0,8!    Checking drive:  !z_drive!set z_file_dri=!z_data!!z_name!-!z_client!-!z_drive!.driif not exist "!z_file_dri!" (  bpdir -M !z_client! "!z_drive!:" > "!z_file_dri!")for /f "tokens=1,2,3,*" %%a in (!z_file_dri!) do (  set z_date=%%a  set z_time=%%b  set z_size=%%c  set z_file=%%d  set z_part=!z_file:~0,5!  set z_found=n  if /i "!z_part!"=="_vxfi" set z_found=y  if /i "!z_part!"=="VSPCa" set z_found=y  if /i "!z_found!"=="y" (    set /a z_caches_tot+=1    echo !time:~0,8!    Found:  !z_file!    Call :write-csv  ))goto :eofREM ############################################################:write-csvset z_date_mm=!z_date:~0,2!set z_date_dd=!z_date:~3,2!set z_date_yyyy=!z_date:~6,4!set z_date_ddmmyyyy=!z_date_dd!/!z_date_mm!/!z_date_yyyy!set /a z_size_kb=!z_size!/1024set /a z_size_mb=!z_size_kb!/1024echo !z_client!,!z_drive!,!z_date_ddmmyyyy! !z_time!,!z_size!,!z_file!,!z_size_kb!,!z_size_mb! >> "!z_file_csv!"goto :eof


 

Casey King's picture
I am running a 6.x environment but half of my clients are still running 5.1.  Have you upgraded to 6.x yet?  If possible can you make the script work for the 6.x environment and post.  Also, thanks for the detailed information concerning VSP and VSS.
 
Casey