The Symantec Security Information Manager (SIM) appliance identifies critical threats in your environment by correlating hundreds of millions of raw events into an actionable list. It receives information on suspicious events from a wide variety of antivirus and intrusion detection systems as well as from Symantec's Global Intelligence Network of sensors and security knowledge.
At a large financial company, SIM was being used to monitor 100,000-150,000 devices. Although events were flowing into the product at only half of its maximum sustained rate, SIM was unable to properly keep up with the inflow of information.
Mark, a thirteen-year Symantec tech support veteran, responded to the call to Symantec Enterprise Support Services.
Scouring the scene for clues
"They were getting a high volume of events," says Mark, "which shouldn't be a problem because SIM is clocked to process 2,000 events per second (EPS). Here, we saw that the slow-down occurred at 1,000 EPS. I had to look closely to find out what caused the problem."
The customer's implementation was different than most, which led to possible clues. Instead of using the correlation and reporting features of SIM, the customer was using it to send the information to a third-party product and report it on a custom-made security Web site. The third-party product would search the SIM database and extract information on events.
Before he could find the answer to the problem, Mark had to go deep into the details of the implementation, spend time with the customer's support team, set up a test environment in his own lab to run possible scenarios, and examine screen shots of configuration pages going back to the beginning of the project.
Solving the mystery
At last he found the key to unlock the mystery. Circumventing the native correlation capability of the Symantec product to identify events worthy of an alert status, the company had configured SIM to treat every event from their Symantec AntiVirus servers as an alertable event. SIM normally processes events in batches but the configuration of the system prevented batching of events. As a result, SIM was processing every event singularly.
Mark went back to his lab to test the hypothesis. "I was able to see the same issue that the customer was running into when I configured the system as the customer had," he says.
Mark brought the problem to the Symantec development team and showed the documentation that he had been able to capture from his lab tests. Development created a patch that would enable SIM to batch events even if they were already designated as alerts. The team rolled the patch into the customer's environment and verified that the issue was gone.
Case leads to product improvements
The customer is extremely pleased with Symantec Enterprise Support Services, pointing to Mark's intervention as critical to its being able to keep its SIM implementation. "Not only is the customer satisfied," says Mark, "but we've put the solution into the next release, so now other customers who want to examine all their events have that option. That will help to make all our customers happy."