Monday, February 2, 2009

How to Blow Up a Cloud

Discussion seems to have started over a new kind of security threat, one that is unique to cloud computing. Christopher Hoff at Unisys first envisioned it (hasn't happened yet, as far as I know) and gave it the catchy name "EDoS": Economic Denial of Sustainability.

Actual conversation while I was writing this:

"What are you reading about?" asked my wife.

"Economic Denial of Sustainability."

"Ack Guk." Pantomimes sticking a finger down her throat. "I'm reading a murder mystery."

This sums up a little problem I've found when reading about this subject so far: What this "EDoS" mean gets lost, at least for me, behind a bunch of polysyllabic specialist words. So as an aid to clearing my own thoughts, I'm going to take a whack at explaining it here. I'll be trying hard to avoid jargon, so if you're knowledgeable don't rag on me for not using proper terms. Do rag if I get it wrong, of course.

First, a precursor concept: Distributed Denial of Service, or DDoS. (Now you know why EDoS actually is a catchy name, at least in certain quarters.)

DDoS begins when a bad guy uses a virus, worm, or similar kind of Bad Program to implant something that does his bidding into your Aunt Sadie's computer when she innocently connects to the Internet without protection (firewall, virus checker, etc.) for five minutes. Yes, according to numerous experiments in security labs in numerous companies, that's all the time it takes. The Internet is a dirty place.

Enormous numbers of computers are infected with such things. Groups of infected computers owned by a single bad guy or group have been estimated to consist of tens or hundreds of thousands of computers. Having infected them with his own stuff (and, amazingly, often then fixing the vulnerabilities so nobody else can take it away from him), the bad guy can then send commands out to all those computers to get them to do something for him.

Do what? Well, since we're presuming this is a bad guy, it's unlikely that he or she will use all those computers to help create a cure for cancer, analyze data searching for extraterrestrial intelligence, or any of the other laudable things that many people run when their computer isn't doing anything else. More likely, he'll use it to send spam.

Another typical bad guy use is to launch a Distributed Denial of Service (DDoS) attack on a web site: The bad guy orders all those computers to continually send repeated requests to some targeted web site. Hundreds of thousands of computers doing that all at once will at least clog the site, making it impossible for legitimate users of the site to get in. It may well cause the site to crash, going offline completely.

DDoS attacks are real, and have occurred "in the wild," meaning they happened for real, launched by genuinely bad guys, attacking real web sites, and are known about. They're a type of attack that's rather hard for a company to keep hidden from the public. I should look up some examples and cite them here, but I'm feeling lazy. Google "DDoS" or the expansion. DDoS has been around for a long time.

So, basically, DDoS is a bad guy causing a huge number of computers to whack on a web site until it falls over.

Now, cue Cloud Computing.

One of the definitions of Cloud Computing (defining it is a widespread hobby) is the use of large bunch of computers as a common shared pool of resources to run one or more applications. Most of the focus of this area is on using someone else's large bunch of computers to run your programs, so you are effectively "renting" the computers rather than buying them.

With the right kind of application – and web serving, in most cases, is the right kind – you can rent only what you need for what you do, expanding if your business goes up, and contracting when it drops. This is very cool, since you don't need to outguess your market and plan long-term investments in computers, software purchases, buildings to house the computers, people to run them, and so on. In the jargon, it turns CapEx into OpEx – capital expenses into operating expenses. This is an especially good thing nowadays.

Clouds are proving to be a great way to meet temporary demand. A star example of this was the night the NY Times converted 11 million articles into PDF, making web-available a huge historical archive of NY Times articles. This was done by Derek Gottfrid, who rented computing from Amazon; the books-and-everything-else-seller turns out to be the major cloud computing provider, selling its spare cycles as the Elastic Compute Cloud (EC2). I'm told it cost Derek around $300, small enough that he put it on his personal charge card. Buying the computers and installing them would have been ten to a hundred times more costly and taken rather a bit more than 24 hours.

Of course, the "elasticity," expanding and contracting on demand, can be automated so it responds to needs directly, and immediately. This is rightly considered a major feature of clouds that have it. In many clouds, it's the application software itself, potentially the best judge of its own busy-ness, that signals the cloud provider to add more oomph.

With that buildup, you can see where this is going. DDoD and cloud computing: Put them together.

Imagine a DDoS attack suddenly targeting a web site running on Huge Big Momma Towering Thunderhead Cloud Computing Corporation. Elasticity is on full auto. What happens?


Your application expands like a giant balloon on a compressed-air tank.

It grabs additional computers as fast as Huge Big Momma can fill them up with your application, which can be plenty fast. There may be no crash or even slowdown of legitimate users; the Huge Big Momma web site we are imagining is, indeed, huge, and it handles lots of people's applications. This is a good thing, since the same WHOOMP may happen if you have a really successful one-day sale.

Of course, HBM's bill-o-meter goes into overdrive, charging you for all the computers used, all the communication into the cloud, etc. (This, by the way, is a reason why Huge Big Momma is highly motivated to fill computers with applications in a big hurry.) Since you can be sure the infected computers aren't ordering anything, you potentially lose money very rapidly indeed.

A more insidious form of this attack can also be used, very simply: Just don't do it all at once. Instead, keep unobtrusive steady trickles of bogus requests coming in, just enough to increase the rent.

What this trickle attack accomplishes is eroding the economic advantage of the cloud, threatening a major justification for using cloud computing. As far as I can tell, this trickle attack is what Hoff actually had in mind when he coined the term EDoS.

Note that such a trickle attack may well not affect traditional self-owned web sites at all, since it would just soak up otherwise unused capacity (of which there usually is a lot), and add a tad of delay to legitimate users' requests. But since it is a trickle, it wouldn't be noticed. So nobody will bother mounting such an attack.

Are either the sudden or the trickle attacks real threats? Of course. They can be done, so they will be.

Sudden attacks can be blunted by attaching some rational policy to full-auto expansion, like "don't expand more than X% without checking with a human," with provision for prearranged one-day sales or the like. You do, however, need to know to do this. And, with application code itself doing the expansion, the cloud provider needs something to put on the brakes, since that code may have a bug. Bug-produced ballooning is a far more likely than EDoS, actually.

The trickle version is a bit less straightforward to deal with. It implies (and so does the sudden attack) that the traffic monitoring already done to avert other types of threats will have to be extended to cover these cases. Perhaps this can be done – and I'm definitely not a security guru, so this may be completely bogus – by looking at how similar the requests are. The messages spewed by robots may well have similarities to each other, or at patterns in their timing or source locations, that are distinguishable from human-generated requests. Work will be needed to discover those patterns, and Hoff is to be thanked for pointing out that this is a new style of Internet vulnerability arising in future cloud use.

My net is that EDoS is just another round in the never-ending black-hat – white-hat thrust and parry which, unlike a real sword fight, seems never-ending.

(P.S.: I swear to $deity that when I made up the Huge Big Momma name I was not thinking of any other company whose initials might end in an M preceded by a B. Really.)

(P.P.S: Lots of software-related posts recently. I promise some hardware will come up soon.)


Eric Knipp said...

The cloud presents all kinds of security concerns in a new light. In fact, I think those of us following the cloud "trend" tend to think about security in terms of mapping existing threat models onto cloud architecture. Your take points out a serious threat that couldn't even exist in prior deployment models. Great point, thanks for sharing it.

Anonymous said...

>I'm told it cost Derek around $300
It was 100 computers for 24 hours @0.10 per hour for each computer. So, $240 dollars. A bargain...

Ryan said...

The "trickle" EDoS seems like a bit of a marginal attack. If implemented on any business that is actually a going concern, it seems all it could do without attracting notice is raise the costs of hosting by, at most, a multiple of the "base" cost, not a magnitude.

Are there really legit businesses that are economically viable when cloud hosted, but not when cloud hosted at 2x the cost? That seems like a fairly small group of businesses.

Roland Dobbins said...

All DDoSes are 'EDoSes', by definition. 'EDoS' is a dumb, redundant name which nobody with clue uses (see my comments on Hoff's post about this). It hasn't caught on, and it won't, so you should probably stop using it, heh.


All the various attack modes you describe, and then some, are in use today and have been for years.

You're correct in that DDoS is the big security elephant in the room nobody wants to talk about in regards to cloud computing - DDoS kills the cloud dead, totally defeats the model.

Fortunately, there are many Best Current Practices (BCPs) at all 7 layers of the OSI model, from architecture to implementation to operations, which greatly improve the resiliency and ability of applications/infrastructure/services to withstand DDoS. Unfortunately, most organizations don't follow/implement them, so trivial DDoS attacks like the recent RoK/USA attacks end up taking down important services and having a disproportionate impact.

The cloud model will finally force folks to implement the BCPs; however, it's going to take failure on an organization-by-organization basis for this to happen, since people seem fundamentally unable/unwilling to learn from the mitakes of others.

Dong Seong Kim said...

Is the EDoS used in any literature?

Greg Pfister said...

@Dong - I don't personally know of uses of EDoS. Google scholar can't find any. Plain Google only finds a few blog & news articles. So net, I'd say no.

Post a Comment

Thanks for commenting!

Note: Only a member of this blog may post a comment.