A Lesson in IT History
Helping customers during a crisis can be a matter of obtaining history. One of the best practices in technical support is keeping accurate and detailed records of all aspects of support incidents. These records can be invaluable in providing a quick resolution to customer problems if further incidents develop, and can potentially save the support specialist hours of diagnostic tests and troubleshooting. Looking up a customer’s detailed history, then, is a key to getting the best results.
Tyler, a Symantec Antivirus (SAV) support specialist, was on a call with another client when he received an inbound request from a large Business Critical customer. An emergency had nearly brought down the network of a multinational oil and energy company. Tyler handed off his current call to another representative and fielded the emergency call.
The oil company had recently undergone a planned, 10-day shutdown of its network. When the IT group in the data center brought the system back up, SAV began to download and deploy a full VDB definition set—over 30MB—rather than a much smaller incremental update to the SAV clients. At many customer sites, this would not have been a problem, but the oil company network was attempting to distribute the full 30MB VDB to more than 60 servers and 100,000 clients on its network. The distribution of that much data to so many endpoints was consuming bandwidth and nearly crashing the entire network. Many of the clients that needed the VDBs were in remote areas, including off-shore drilling platforms with limited and intermittent bandwidth. Without a fix, the network would be brought to a standstill trying to update remote sites over low-bandwidth WAN links.
Business critical background
The customer mentioned that the company had experienced the same problem earlier in the year, so Tyler accessed the company’s detailed support file. “Because they are a Business Critical customer, the site is extensively documented,” Tyler noted. “I was able to look up all the information I needed to set a plan of action.” Tyler knew the top priority was to fix the problem rather than run diagnostics. “Trying to find out why it happened was the lowest priority; solving the problem was top priority.” Based on the client history, Tyler was able to create a custom XDB that included the missing incremental updates in the Symantec test environment. “We broke apart the days of incremental updates that were missing from their main VDB, compiled those into a new definition set, and ran tests,” he recalls. Once he verified that the custom XDB would roll out successfully, he contacted the IT team at the oil company with instructions on how to apply and roll out the smaller, bandwidth-friendly update. The IT specialists at the energy company applied the update to a test server, and once they determined it would work without a hitch, they immediately deployed it across the entire network. The update was deployed successfully, and network performance returned to normal. “You could hear them breathe a sigh of relief,” Tyler recalls.
The period from first call to resolution was less than one business day, according to Tyler. The custom XDB was in the customer’s hands before 3 p.m. “Because of our past relationship with the company resolving a similar situation, we already knew what to do,” Tyler added. “Having all that history documented meant we could respond quickly.” Tyler followed up with the customer the next day. There were no other issues, so he closed the case. But every step of the process is in the customer file—just in case history repeats itself.