the SEP_INST.log should be there. You can try this. Export the install for this server Os from SEPM (NOT single executable). Move the exported install folder wherever you are going to run it on the server from. Create a shortcut to the setup.exe in it's same folder. Edit the properties of the shortcut file, and in the target field, after "[DRIVE LETTER]:\...\setup.exe", paste this:
/v"/l*v c:\SEP_INST.txt /qn", so that your whole string would look like
"[DRIVE LETTER]:\...\setup.exe" /v"/l*v c:\SEP_INST.txt /qn". This basically forces the verbose logging to a specific location.
Also, some basic things to make sure of before the install is run is that there is at least 700 MB free OS space available. Also, uninstall LiveUpdate from Add/Remove Programs before the SEP install. Also, perform a restart of the server right before the install. Of course only do these things proactively if you are still unable to generate a log file from the SEP install.