wow....you're not kidding. I just ran a test on this to assess the impact of fetching ACLs for SharePoint scans. We went from 11 GB scanned in about 24 hours (less than 500 mB/hr) to up to 9 GB/hr. Obviously there's something clearly amiss in that, as the discrepancy is just far too high.
Funny, with the old SharePoint scanner, I guess I had learned to accept that these scan types were just going to be slow regardless of what you did. With ACL Fetches off, this is rivaling the speed of file share scans. Yes, it would be better to have the ACLs, but at least you do still get the Created By and Modified By attributes.
Please post up here if they get a fix for you. I've already spoken with one of my customers about this, and we've come to the conclusion that we could easily do a quicker "inventory" scan with ACLs turned off, and then, if needed, target specific sites with a subsequent scan with ACLs turned on to get the more detailed info.
~Keith