I've attempted to use the ShrinkEmbeddedDB tool for SEP and it'll run for about 90 minutes before erroring out with "Faile to unload DB."

It'll stay on "Unloading DB... 10%" for about that whole time then goes up to 55% just before it quits.

I had an additional 28GB free on the volume in question.  The original database is 27.9GB.  I looked in the dbdump folder and the file 749.dat grew to 44.2GB before it stopped running.  In the unload.log file there was the following message:

***** SQL error: Cannot access file 'D:\Symantec\dbdump/749.dat' -- Invalid argument

Below is the contents of the command window.  Thanks for your help in advance!

D:\Symantec\Tools>shrinkembeddeddb.bat
Jul 22, 2014 9:08:51 AM com.sygate.scm.server.util.ServerLogger writeHeader
SEVERE: ================== Server Environment ===================
Jul 22, 2014 9:08:51 AM com.sygate.scm.server.util.ServerLogger writeHeader
SEVERE: os.name = Windows Server 2008 R2
Jul 22, 2014 9:08:51 AM com.sygate.scm.server.util.ServerLogger writeHeader
SEVERE: os.version = 6.1
Jul 22, 2014 9:08:51 AM com.sygate.scm.server.util.ServerLogger writeHeader
SEVERE: os.arch = x64
Jul 22, 2014 9:08:51 AM com.sygate.scm.server.util.ServerLogger writeHeader
SEVERE: java.version = 1.7.0_51
Jul 22, 2014 9:08:51 AM com.sygate.scm.server.util.ServerLogger writeHeader
SEVERE: java.vendor = Oracle Corporation
Jul 22, 2014 9:08:51 AM com.sygate.scm.server.util.ServerLogger writeHeader
SEVERE: java.vm.name = Java HotSpot(TM) Client VM
Jul 22, 2014 9:08:51 AM com.sygate.scm.server.util.ServerLogger writeHeader
SEVERE: java.vm.version = 24.51-b03
Jul 22, 2014 9:08:51 AM com.sygate.scm.server.util.ServerLogger writeHeader
SEVERE: java.home = D:\Symantec\jre
Jul 22, 2014 9:08:51 AM com.sygate.scm.server.util.ServerLogger writeHeader
SEVERE: catalina.home = D:\Symantec\Tools\..\tomcat
Jul 22, 2014 9:08:51 AM com.sygate.scm.server.util.ServerLogger writeHeader
SEVERE: java.user = null
Jul 22, 2014 9:08:51 AM com.sygate.scm.server.util.ServerLogger writeHeader
SEVERE: user.language = en
Jul 22, 2014 9:08:51 AM com.sygate.scm.server.util.ServerLogger writeHeader
SEVERE: user.country = US
Jul 22, 2014 9:08:51 AM com.sygate.scm.server.util.ServerLogger writeHeader
SEVERE: scm.server.version = 12.1.4104.4130
Has valid SAV license
Info>> No SNAC license file in D:\Symantec\Tools\..\tomcat\etc\license
Jul 22, 2014 9:08:52 AM com.sygate.scm.tools.db.ShrinkEmbededDB initLogger
INFO: logger initialized ...
Jul 22, 2014 9:08:52 AM com.sygate.scm.tools.db.ShrinkEmbededDB shrinkDB
INFO: shrinkDB: Start to shrink DB.
Jul 22, 2014 9:08:53 AM com.sygate.scm.tools.db.ShrinkEmbededDB detectBackupFile

INFO: detectBackupFile: Detect bak DB file ...
Jul 22, 2014 9:08:57 AM com.sygate.scm.tools.db.ShrinkEmbededDB stopTomcat
INFO: stopTomcat: Stopping Tomcat services...
Executing command - stop semsrv
The Symantec Endpoint Protection Manager service is not started.

More help is available by typing NET HELPMSG 3521.

Waiting for service termination: semsrv
Checking whether the service is stopped: semsrv
Retrieve status for service semsrv
The status for semsrv' service is 1
Service is stopped.
Jul 22, 2014 9:08:57 AM com.sygate.scm.tools.db.ShrinkEmbededDB shrinkDB
INFO: shrinkDB: The OS is Windows 2008....
Jul 22, 2014 9:08:57 AM com.sygate.scm.tools.db.ShrinkEmbededDB shrinkDB
INFO: shrinkDB: Stopping DB service....
Jul 22, 2014 9:08:57 AM com.sygate.scm.tools.db.ShrinkEmbededDB stopDBServices
INFO: stopServices: Stopping DB services...
Executing command - stop SQLANYs_sem5
The Symantec Embedded Database service is not started.

More help is available by typing NET HELPMSG 3521.

Waiting for service termination: SQLANYs_sem5
Checking whether the service is stopped: SQLANYs_sem5
Retrieve status for service SQLANYs_sem5
The status for SQLANYs_sem5' service is 1
Service is stopped.
Jul 22, 2014 9:08:58 AM com.sygate.scm.tools.db.ShrinkEmbededDB shrinkDB
INFO: shrinkDB: The DB vendor is:Embedded
Jul 22, 2014 9:08:58 AM com.sygate.scm.tools.db.ShrinkEmbededDB unloadDB
INFO: unloadDB: [Tue Jul 22 09:08:58 EDT 2014] Start to unload the DB....
Jul 22, 2014 9:08:58 AM com.sygate.scm.tools.db.ShrinkEmbededDB unloadDB
INFO: unloadDB: The command line is:"D:\Symantec\ASA\win32\dbunload.exe" -c "UID
=dba;pwd=**********;dbf=D:\Symantec\db\sem5.db" "D:\Symantec\dbdump" -r "D:\Syma
ntec\dbdump\reload.sql" -v  -q  -y  -o "D:\Symantec\dbdump\unload.log"
Jul 22, 2014 10:42:06 AM com.sygate.scm.tools.db.ShrinkEmbededDB unloadDB
INFO: unloadDB: [Tue Jul 22 10:42:06 EDT 2014] End to unload the DB, it spends t
ime 5588 seconds.
Jul 22, 2014 10:42:06 AM com.sygate.scm.tools.db.ShrinkEmbededDB shrinkDB
INFO: shrinkDB: Faile to unload DB.

What's the exact version here?

The server is running 12.1.4104.4130.  I had tried it previously with 12.1.4013.4013 with the same results.

You need to follow these steps

dbunload utility for SEP 12.1

https://www-secure.symantec.com/connect/ideas/dbunload-utility-sep-121

These steps tell me about the fact there is no dbunload utility, has someone talking about doing a backup and restore just made their database bigger, and then a link to ShrinkEmbeddedDB.zip

can you check the size of your DB?

https://www-secure.symantec.com/connect/downloads/sql-database-schema-information-and-database-planning

when you restored the DB and run livepdate, it might have added contents to the size, may be thats why a size variation.

See Elisha Comments

Try stopping the SEP Manager and copying the sem5.db and sem5.log files to a separate folder first before running the tool

https://www-secure.symantec.com/connect/downloads/shrink-symantec-endpoint-protection-manager-121-embedded-database#comment-7292011

Thanks.  I tried this but it made no difference.