Hi,
System Requirements for File Share Encryption you can find on the page 2 of the document "symcFileShareEncr_1032_win_quickstart_en.pdf" available as an attachment under this link:
https://support.symantec.com/en_US/article.DOC7062.html
SEMS you would need to install on ESXi. VMware Workstation I would rather consider as a test environment.
Bare in mind, that there are some requirements for installing SEMS on ESXi server. Here are couple of useful articles:
Best Practices for Creating a New Virtual Machine for Symantec Encryption Management Server
https://support.symantec.com/en_US/article.TECH186060.html
Pictured Installation Guide for Symantec Encryption Management Server
https://support.symantec.com/en_US/article.TECH197003.html
Installing Native VMware ESX/ESXi/vSphere Tools on Symantec Encryption Management Server
https://support.symantec.com/en_US/article.TECH176852.html
Hope that helps.