If you haven't done so, create a detection rule for the piece of software you're installing. That way, when the Managed Software Delivery runs, it will check first to see if it's installed. If it's present, it won't reinstall.
You could also create a filter of machines that have the software installed, and exclude it from the Managed Software Delivery, so that as machines get the software and are added to the installed software filter, they'll be removed from the delivery policy.