By Dan Bleaken, Senior Malware Data Analyst, Symantec Hosted Services
Introduction
Yesterday the U.S. Federal Trade Commission (FTC) shut down California-based ISP Pricewert LLC (also known as 3FN and APS Telecom), a notorious rogue internet service provider (ISP) that specialised in the deployment of botnets and the distribution of illegal, malicious and harmful content such as spam and child exploitation images. (http://www.ftc.gov/opa/2010/05/perm.shtm)
One of the largest and most active botnets responsible for spam activity, the Cutwail botnet, experienced several hours of downtime on the morning of June 5, 2009, following a preliminary injunction by the FTC earlier that week. Malware from the Cutwail botnet, also known as Pandex, was first identified in January 2007.
With between 1.5 and 2 million active bots, Cutwail was perhaps the largest botnet in history at its peak. Before the November 2008 shutdown of ISP McColo, Cutwail was linked to approximately 25% of all spam. By the end of May 2009 it was responsible for 35% of all spam. The Acai berry spam runs, which MessageLabs Intelligence reported on in May, have been among its larger spam runs.
By late June 2009, the Cutwail botnet had managed to recover to approximately one third of its original capacity, while still limping from the impact of this latest ISP shutdown, it wasn’t as badly affected as Srizbi, a rival botnet that was devastated by the closure of McColo, another ISP, in November 2008. The fact that the botnet was able to recover after only a few hours highlighted the progress that spammers had made since November’s McColo shutdown. Clearly, spammers have learned the importance of having a backup for their command and control channels.
History of ISP takedowns, and botnets affected
Security firms have battled botnets for many years, but only in the last 18 months, with takedowns of rogue ISPs (especially McColo), has the security community been more aware of opportunities to disrupt botnets, and grown more confident that this online menace can be successfully fought.
Botnets are distributed networks of ‘zombie’ or ‘bot’ PCs, infected by malware which enables them to be marshalled by cyber criminals primarily to distribute enormous volumes of spam and other malware and launch phishing attacks via email.
Botnets have been around for years. Back in 2003 – ancient history in internet terms – security firms often saw thousands of PCs infected with the Sobig mass-mailing virus, and later in that same year, when the Fizzer malware logged thousands of computers into internet relay chat (IRC) rooms. At that time this was perceived only as a problem for IRC admins. In fact, the infected PCs were being connected to chat rooms by their human bot-masters, waiting to receive command and control (C&C) instructions. It was only when MessageLabs Intelligence correlated the malware and spam traffic from each spam sending IP address, that the bigger picture was later revealed.
Over the years security firms kept botnets under observation, doing their best to inhibit their ever-increasing output of spam, but not knowing how to tackle their C&C infrastructure. Then in October 2008, the security community stepped up efforts to disrupt the botnets and start to pursue the ISPs that hosted them.
The first high profile ISP to go at the end of September 2008 was Intercage (aka Atrivo), linked to the infamous Russian Business Network. However, the most widely publicised was McColo, a San Jose-based ISP, which was found to deal almost exclusively with cyber gangs. McColo, set up by a 19 year-old Russian, was host to a botnet called Srizbi controlling 1.3 million IP addresses, as well as the Mega-D, Rustock, Asprox, Bobax and Gheg botnets.
In November 2008, community action resulted in McColo’s peering ISPs disconnecting it from the internet, largely because of an article written by Brian Krebs in the Washington Post [blog.washingtonpost.com/securityfix]. Taking down McColo came as a shock to the botnet gangs and spam levels dropped instantly by as much as 80 per cent, which represented an enormous victory against the spammers.
Srizbi was crippled, never to return and the other botnets were badly disrupted. Over the following two months, spam gradually recovered to previous levels, as the surviving botnets relocated their command and control channels, the criminals spawned several new botnets as well.
The operations behind many of these botnets were forced to re-evaluate how they functioned and sought to put more protection in place to prevent a repeat of the huge disruption caused by the take-down of a single ISP. Take-downs like McColo must cost bot herders and spammers hundreds of thousands of dollars - perhaps millions - in lost revenues.
When Srizbi disappeared, activity from the surviving botnets increased dramatically, seeking to fill the huge gap in the market left behind. At that time, Srizbi had been responsible for as much as 50 percent of all spam. After losing their botnet of choice, spammers rented capacity from other botnet operators in order to keep up their spam campaigns.
However, when the next major take-down of a dubious ISP took place, it was clear that the cyber criminals had already learned from the strike against McColo.
This time the security community’s target was another California-based ISP called 3FN (aka APS Telecom and Pricewert) which was hosting command and control channels for Cutwail (aka Pandex), one of the oldest botnets which had been spewing out malware since January 2007. By June 2009 Cutwail had swollen to more than 1.5 million active IP addresses in an aggressive recruitment drive.
3FN was put under a preliminary injunction by the FTC on Friday 5 June 2009 and Cutwail went with it. But within a couple of days, by Monday morning, Cutwail was back online and as strong as before.
Botnet gangs had made many refinements to their creations in the six months since the McColo take-down. Thus the organisations behind Cutwail were able to quickly reorganise after losing an important part of their botnet infrastructure. The fact that the technology was now much more flexible and robust allowed them to review the status of the botnet and return to business as usual in just a few days. It was clear that botnets now had a business continuity or disaster recovery plan of their own.
The botnet C&C mechanisms had shifted away from IRC towards HTTP. Instead of receiving instructions from one place, algorithms were built into the bots so they would look for random-looking domain names, which are purchased by the botnet gang each day, and from which the bots receive their commands. This ensures that the botnets aren't so reliant on one ISP.
But as the botnet controllers evolved their tactics, so did the security firms. One botnet in particular had grown significantly in the wake of the McColo take-down; a botnet called Mega-D (aka Ozdok). By November 2009, the algorithms behind the C&C mechanism used to issue the botnet with new instructions were broken by FireEye, a security company that specializes in combating botnets. It was now possible to predict which domain names were to be used by the botnet and to register them in advance of the botnet controllers. It was almost like cracking the Enigma code; and for the first time it was possible to know the botnet’s next move and to register these domains faster than the botnet controllers.
Mega-D appeared to be crippled, its spam-sending days seemed to be over, until a few days later, MessageLabs Intelligence identified large volumes of Mega-D spam being distributed from IP addresses that had not been used to send spam previously. This suggested that the botnet controllers had enacted their business continuity plan, seemingly with inactive sleeper bots or perhaps even a whole parallel backup botnet.
Disaster Recovery isn’t the only tenet that botnet controllers have borrowed from the world of corporate IT; they also use a technique called “fast-flux” hosting, which dynamically distributes resources across a number of continually changing IP addresses using a form of “round-robin” DNS. In the hands of a botnet controller, fast-flux can hide the true location of websites used to host malware, spam and phishing content, by hiding them behind the IP addresses of compromised, botnet-controlled computers, each acting as a web server or proxy.
Another technique that botnet controllers use to hide their botnets from the prying eyes of security firms is to expose only a small proportion of their zombies at any one time, cycling their use over a period of several days and limiting the amount of spam sent from each to minimise the risk of them appearing in backlists of known spam-sending IP addresses.
Until recently, botnet controllers had to recruit one PC at a time. But with the advent of “generic droppers” like Bredolab, and perhaps Conficker, larger botnets can be assembled to order, whether for a spam campaign or something even more sinister.
Cyber criminals can now purchase the control of thousands – even tens of thousands - of ready-compromised PCs, recruited en masse to their own botnet. For example, cyber criminals may pay for malware, spyware or botnet Trojans to be installed by the Bredolab operators who would then instruct the Bredolab botnet to “drop” the malware onto however many computers were needed, using computers that were already under their control. This takes botnet recruitment from a random, scattergun approach where PCs are infected at random, to a more commoditized recruitment campaign. The only limitation to the size of the botnet is how much the criminals are prepared to spend.
Now cyber criminals with enough cash can set up a botnet operation and be in business virtually overnight.
As botnets grow in sophistication and number, there is a danger of them becoming an extension of the hidden world of international and industrial espionage. The authorities in many countries are now concerned that attacks on government and business resources will become the next battleground in cyber warfare. The botnet could become the weapon of choice to disrupt infrastructure, and a lot less expensive - or traceable - than a ballistic missile.
The resources exist for states to be attacked from the inside
More concerning perhaps is that such an attack could come from within a country’s own borders and as such would be difficult to prevent. In an average week in the UK the UK Messagelabs Intelligence tracks at least 16,000 active spam sending bots. The actual number of active bots in the UK is likely to be much larger than this. One way to look at this is as a potentially massive sleeper cell. It would take hundreds rather than thousands of zombie PCs to launch a successful DDoS attack against a typical web server; however, cyber criminals often prefer to spread the workload across sever thousand computers to better avoid detection.
The zombie PCs that make up botnets are recruited largely from inadequately protected domestic PCs, but there are also numerous compromised business networks too. Conventional firewalls that don’t inspect HTTP streams –the preferred C&C mechanism for many contemporary botnets- are not sufficient protection, nor is conventional antivirus software on its own.
Businesses should minimise the risk of becoming part of a botnet by ensuring that they are protected by filtering internet traffic for spam and other malicious or harmful content before it reaches their corporate network. Many ISPs provide such value-add services now, as more pressure is brought to bear on the industry to tackle this problem as close to the source as possible.
Service Level Agreements (SLAs) are also critical here, for example, during a major spam run, how many false positives does the SLA allow for before too much genuine mail is be quarantined along with all of the spam?
All eyes are now on the cloud, whether private or public, to fight the spammers. Traditional desktop appliances are no longer flexible or strong enough to keep defences running around the clock, 365 days of the year, which is what is now required from businesses and individuals. Botnets are essentially ‘private clouds’ working together, distributing the infected software to PCs. Botnet operators are like SaaS providers, with all the will in the world it will be almost impossible to fully eradicate the problem using technology alone. The most effective way to alleviate the botnet burden is by turning the Internet against them using the fabric of the cloud as a catalyst to kill the botnets, making the internet a safer neighbourhood to stroll once again.