Does XP SP2/SP3 have built-in VSS for open file management?

NBU 7.x no longer uses/supports VSP and uses VSS.

Create bpbkar and bpfis logs on client to see if snapshots are attempted, otherwise close Outlook when backups are running to ensure PST's are not open.

NBU Dedupe was introduced in NBU 7.0. NBU 6.5 had no client-side dedupe.

The difference will be all the new features included in 7.0, 7.0.1, 7.1, etc:
http://www.symantec.com/docs/TECH75944
http://www.symantec.com/docs/TECH135812
http://www.symantec.com/docs/TECH147815

You need bpfis and bpbkar logs on client to see why files are skipped.

"If outlook is closed backup works"

So, your problem is not related to MSDP, but with open files.

Create bpfis on the client and look for errors/problems with snapshot.

There is the possibility that it is somesort of timeout issue but as Marianne says it does look VSS related so we need to see the bpfis logs although i would have expected there to have been an error in the clients event logs.

I have not tested this for your type of issue so cant say if it will help but when you are using client side de-dupe you could try making some edits:

On the client open the <install_path>\NetBackup\bin\ost-plugins\<storageservername>.cfg file

and change this value:
MaxRetryCount=5
to read:
MaxRetryCount=50
 
If still failing then add these lines to the client-side pd.conf file:
MATCH_PDRO = 0
LOCAL_SETTINGS = 1

You can also add COMPRESSION = 1 to help with client side de-dupe further

May be worth doing anyway?

Dear all

Thanx folks for sharing knowledge, but to no surprise NBU is still failing to backup .pst files where in PureDisk is winner on this head. The only limitation i found in compatibility list was XP_SP3 doesn't have support from NBU deduplication. help file contents says different story of native support from XP_SP2 onwards.