The default "Group Rights" for all Groups on the "Access Rights" tab is "Full Access".  

To give a Limited Administrator access to only one or a few Groups, it is necessary to manually change all the other Groups to "No Access".  

We recommend/request that this get changed so the default rights are "No Access" to 1) make configuring a Limited Administrator simpler and less time consumming, and, 2) follow the normal Best Practices for security settings to "Deny all".