Managing Queues That Get Backed Up in NS6

On a few occasions we've had issues where our queues (EvtQFast, EvtQueue, EvtQSlow) have gotten backed up.  This could have been caused by a database communication issue, a service hiccuping or something else.

Regardless of the cause, once you get the issue fixed and the events start processing, they may not be processing fast enough and, as a result, you start seeing one/many queues reporting busy in your Notification Server's event log.

Now you could stop the Client Message Dispatcher service, move all of the events from the queue folders into a temporary folder, restart the service then gradually copy the files back in so they process, but that often involves up to hours of babysitting the server.  One of our server's is starting to outgrow its hardware so this has been a task we've had to address a few times this past year, and when it happens it shoots your productivity.

I came up with a more automated way to handle this using Robocopy, which is a part of the Windows Server Administration Kit.  Robocopy gives you the ability to put a delay on file copy operations so you don't flood the event queues when you try to get the events to process again.

If you already have Robocopy installed on your NS, you can follow these steps:

1. Stop the Altiris Service and the Client Message Dispatcher Service
2. Move the contents of your event queue folders to a temporary folder
3. Restart the Altiris Service and the Client Message Dispatcher Service so new events can come in
4. Run the following command:  robocopy "D:\Path\To\Old\Events" "C:\Program Files\Altiris\Notification Server\NSCap\EvtQFast" *.* /IPG:250 /MOV /W:1

The command line in step four will copy the contents of the temporary folder containing all of your old events into the EvtQFast folder for faster processing.  The IPG switch sets a delay of 250 milliseconds between packets to help slow the injection of files in the copy process so any new events coming in will still get processed in a timely manner.  The MOV switch says to move the files instead of copy, and the W switch sets the wait time to 1 second if there's an issue with the copying.  This is in there as Altiris may grab the event before Robocopy adjusts file attributes on it.

The 250 milliseconds can be adjusted to fit your needs.  If there are a lot of new events coming in, you may want to bump this up to 500.  Obviously, the bigger the number, the longer the process will take.  I try to make sure the items in the EvtQFast folder don't start growing too quickly.  If they do, I Ctrl+C out of the command, adjust the IPG value and restart it.

There may be a few files leftover when Robocopy is finished running.  I usually drag and drop those into the EvtQFast afterward so those events process.