Well the first thing that really got my attention here is that you stated that you were on RU1
I don't have the info handy at this moment, but I do recall that 12.1 RTM and RU1 and I believe that RU1 MP1 had some issues with honoring exceptions.
I would start with an upgrade (if this ends up going to support anyways, I don't think engineering will work on an older platform), get the SEPM to RU4, and toss it onto a couple of test clients. The upgrade may help; there has been engine changes themselves on how the individual components work and the newer verisons will be the first ones to get those updates should they be released via LU.
Now what I am curious about is are you removing the other components or are you just disabling/turning them off - there is a difference, drivers still load and still affect the system. Simply withdrawing or disabling features by policy do not unload drivers.
If this persists despite removing components of SEP - you can also go down to the CORE only set of SEP (which "has no technologies installed" - but isn't really true) - once we get that far, we can start individually looking at which drivers themselves is the cause.
Or if you would also be willing and tackle it on the policy end - scanning network files, etc. can cause issues too.