I create a FULL package for our servers - the whole shooting match minus email protection for AV.
SEP is smart enough that when it installs, it will disable the PTP anyway, so that's not an issue.
I've honestly seen no "huge performance hit" although ANY protection such as firewalls, or services that inspect packets will slow things down some.
So we run the firewall, IPS, and AV (again, it disables PTP sensing the server OS)
For domain controllers, create a group just for them, and set it up so that IPS doesn't detect DOS attacks at all. DCs take a LOT of pings from clients, and if there's much fragmentation on your network, well............ just don't run DOS detection on the DCs, but that's covered via a group specific for DCs.
All other servers go into the same group except for the SEPM servers, and I've relaxed those because I"m the only person who ever goes there, and they are dedicated to SEPM only, so there's less risk. I just have all the normal firewall and AV stuff on them as far as group configuration.
Remember - regardless of what you install, it's your management CONFIGURATION that can make or break a server. So plan carefully, manage carefully. You can force even an AV-only install to cripple a server if not careful.
Maybe what I do here contradicts Symantec's "best practices" or recommendations, but I have to tailor things to our needs, our risks, and the state ISO rules and policies. If we don't abide by their policies, and we get "infected" and it risks their computers, they can cut our agency off the state network in a heartbeat.
You need to tailor the installs to best suite your needs - and what you feel your risks are. Our folks are prone to, well, accidents....... and not being careful. So I deal accordingly.