Tips for Patching Microsoft Windows Servers
The following isn't necessarily the right way to set up and patch Microsoft Windows servers, but for us, it solved our problems and created an infrastructure where -- if changes needed to happen -- they only needed to happen in one location.
The challenges we faced were the following:
- Reboots one cannot reboot enterprise Windows servers any old time. Reboots needed to be controlled.
- Reboots of cluster or redundant systems when scheduling a specific time for reboots, one doesn't want to reboot all resources at the same time. Not good for AD or clustered pairs.
- VMWare how to patch and reboot both the host and the clients appropriately.
- Application specific patching how to account for specific groups of servers that may not get specific patches.
In order set up our patching procedures, all of the above items had to play into the eventual setup. I won't go into all the gory details of how we did it, but give a general overview to give you an idea of what to consider when setting this up.
Everything in Altiris is based around collections. Out of the box, there are many many collections auto-populated for you. You can use these or clone them in order to modify them for your particular needs. We decided to do clones and then set up specific collections that were manually updated. Taking into account the 4 items above, we created manual collections for all of our Domain Controllers. We have an even number of them, so 3 went into one collection and the other 3 went into a second collection. Remember, this is all about setting up the environment so you don't have all of the domain controllers rebooting at the same time. That would be bad.
We then set up manual collections to take care of ½ of all of our clustered and paired servers following up with a second collection with the other half. Next was a collection of just the Host VMware servers. Lastly, a collection that is a clone of "All Windows Servers," but with all the manual collections (created above) listed in the excluded collections area. That way, everything else is covered.
Microsoft Hotfixes arrive in the Tasks area. Before going down the path of setting up tasks, one should determine which Hotfixes one wants to deploy. To do all of them could be disastrous and time consuming. One suggestion may be to only do Criticals or Criticals and Importants. That way, you weed out the minor stuff quickly.
Setting up tasks is pretty simple. You run the wizard and assign it to the appropriate collections you created above. Once done, you shouldn't have the need to return to the task ever again. This would also be where you may want application specific collection (i.e. All Citrix Servers) in order to segment which collections should not get a hotfix.
Patch Management Policies:
At this point, you have your detailed collections and the tasks are set up. Nothing is going out yet though because we don't have the policy set up telling the client what to do.
For Patch Management, this policy resides in Configuration\Solutions Settings\Software Management\Patch Management\Windows\Software Update Agent Configuration\.
There is a policy by default named All Windows Computers with Software Update Agent Installed. Use this policy as the master and make clones of it for each required need.
For example: To set up patching of Domain Controllers, you would clone the policy and name it "Server Patching Domain Controllers group 1" or something like that. You would then add one of your Domain Controller collections from the above step and set times. Times in this case are set in two different spots. One setting is for when to actually run the hotfix and the other is when to reboot a system. You don't want the time between the two to be too far off as changes to files are taking place and can slowly degrade a system. We set it up to patch @ 5pm and reboot around 7pm, with group 2 patching @ 5pm, but not rebooting until 7:30pm when it can be assumed group 1 has rebooted and resumed functionality.
Follow the same general rules for setting up patching and rebooting schedules for the other "paired" collections, the VMWare host servers and the collection of all remaining servers. Last item is to enable the policy.
Some extra thoughts:
Collections can be set up manually or with SQL queries. They can also be created out of an import from AD if you have servers relegated to different OUs for each function.
We never had 100% luck in auto patching clustered pairs. The first one would patch fine and then fail over the cluster prior to rebooting, yet the second one (scheduled to reboot at a later time) would never fail back and thus not reboot. We would manually baby-sit these servers and after the first reboot, manually move the clustered resources over to the patched and rebooted system.
If applying patches to Exchange specifically, Altiris may choke. In our experience, Altiris doesn't stop all the required services to patch Exchange and the patch fails. Errant services like MOM or WMI may also prevent the hotfix from being applied. We resorted to manually applying those patches specific to Exchange instead of having Altiris deploy them.