I've not run into this, and I don't use a VM. I've also never used the DeployCenter stuff and rely strictly on all the 3COM stuff. I've no idea what happens if you mxi and match DC-provided infrastructure like PXE or TFTP and the 3C-provided PXE/TFTP stuff. I think you should only have one or the other. Whatever the case might be, even if they're all installed, make sure only one of each service is running. There should be one DHCP server, one PXE server, and one TFTP server (whether they're all on the same machine or not) in your network.
My 3COM PXE Server (via the Control Panel, 3COM PXE->Logging tab->View log file). You'll need to view the log file, even if you have events logged to the NT Event Log boxes checked, as the log file contains more info. When I view my log file, I see "Data read: 1 hosts, 1 template" and not 0 hosts like you have in your log. Double-check the BOOTPTAB file and make sure it actually contains a line for matching the MAC. In fact, my BOOTPTAB file (and thus PXE server) just responds to (almost) everything as my network is reasonably secure and thus contains only the two following lines (the rest being comments or blank lines)
default:hn:sm=255.255.255.0:hd=\tftpboot\:to=3600:
All NICs:ha=00??????????:bf=mba.pxe:
in particular that "All NICs" line says to send the file "mba.pxe" to any system with an ethernet MAC address that starts with "00" that requests PXE service. So any MAC address from 00-0000000000 to 00-FFFFFFFFFF. I coulda just put all ???????????? instead of 00?????????? but I wasn't sure how it'd respond to an all-?. I've subsequently found out that all-? is perfectly fine but haven't bothered to change my BOOTPTAB file.
The MAC between [] marks should be the MAC address of the system requesting the service. There must be something on your network w/that MAC asking for DHCP and/or PXE. Check the MAC of the VM you're running on, just in case. The "(not found)" comment means that the MAC address requesting PXE service doesn't match anything in your BOOTPTAB file.
Where is your DHCP service? The Proxy DHCP box should only be checked if your PXE Service is running on a different computer than your DHCP service. If they're both on the same computer, it should be unchecked. You aren't getting to the point for TFTP, so wel'l skip that for the moment.
The "No boot filename received" is right, as your PXE service isn't sending anything. It hasn't successfully matched any MAC address in its BOOTPTAB file yet.
For now, all I can think of is double-check your BOOTPTAB file. Regardless of what this rogue MAC address is, the problem is also that you are NOT seeing the MAC address you do want to see (the new machine that you're booting up.) The machine doing the PXE booting (obviously it tries cause it did get an error) should show you stuff like its own MAC and possibly other info (some show UUID, etc). If you can manage to capture anything off that screen, that'd be good too.
PH