site server settings, highlight the server in question, and click configure.Then remove the task and package option (whichever is checked) When they check in they should uninstall the agents.
You then can use the delete option on this same page to remove the server. I only recommend doing step 1 to keep you from having other further issues, you might just be able to highlight and delete from this page and achieve your desired result.