Adding Hardware is not Always the Cheapest Option

July 26, 2010 by Stephan Schmidt

15 Comments

If your application grows and you need to support more customers, there is the common wisdom to buy more hardware. Or solve performance problems with more hardware. But the simple math says there is a point where investing in software or OS optimization is much cheaper than buying more hardware. There are essentially three ways to optimize and scale your web application – assuming it can scale.

Buy more hardware

Optimize hardware – buy better optimized hardware

Optimize your application – or your software stack

Suppose we have 10 servers and want to add 100% customers. Then we need to buy 10 more servers. Suppose we have 1000 servers and want to add 100% customers. Then, for the same growth, we need to add another 1000 servers. As this gets expensive very fast (exponentially)- not in the beginning of your startup but after some time – you need to look harder and harder into software optimizations. This is also the reason why Facebook benchmarks all it’s new hardware to squeeze out every last bit of performance. If you have 10 servers and you squeze out 1% of performance, this is equivalent to 0.1 servers. If you have 10000 servers and you squeze out 1% more performance that’s equivalent to 100 servers.

See the example for costs for 10% customer growth:

Server Costs Adding Hardware is not Always the Cheapest Option

There are other factors to think of. Most often when talking about scaling, people talk about how their application scales linear. This correlates to your application having O(n) scaling. And in theory this is fine. In reality it’s probably f*O(n). And different architectures, although scaling with O(n), have different linear factors. It does matter if f is 2 or f is 20. What f does your web application have?

You can leave a Reply here. Of course, you should follow me on twitter here.

You can share this post!
Do you want to tell others about this article? Use the social bookmark icons to submit this artice to the service of your choice. Thanks.

About the author: Stephan Schmidt has more than 15 years of internet technology experience and 10 years experience in agile. He was head of development, consultant and CTO and is a speaker, author and blog writer. He specializes in organizing and optimizing software development helping companies by increasing productivity with lean software development and agile methodologies. Want to know more? All views are only his own.

6 Tweets

Comments

hgabreu

July 26th, 2010

“As this gets expensive very fast (exponentially)- not in…”
The way you explained is seems like a linear growth, not exponential, isn’t it?
100% of 100 = 100 and 100% * 1000 = 1000?

stephan

July 26th, 2010

Although I’m not the best in mathematics*, I would not call that linear but exponential.

*) as in got my diploma in mathematics but didn’t choose to study mathematics

Nick

July 26th, 2010

The examples you give are linear =P. 100 servers cost 100x more than one server.

An example:

Let’s say you have 1000 customers and one server.

If you could just add 1 server and be able to support 2000 customers that’d be awesome (and linear growth). Eexponential growth would be in order to double your customers you needed to add 10 servers.

stephan

July 27th, 2010

@Nick: The cost for adding 100% capacity is growing exponentially.

1 server => 1 server
10 servers => 10 servers
100 servers => 100 servers

1,10,100 is not linear.

The correlation between customer growth and hardware growth is constant.

stephan

July 27th, 2010

Or better: If your growth is exponential, your server costs are exponential, while your software optimization costs are constant.

Marc

July 27th, 2010

Thanks, that you clarified the relationship between growth of servers and growth of users in your last comment. The servers are only growing exponentially if your users/load are also growing exponentially and not by some mysterious law ;)

Maybe correct this also in your post? I mean, happy company which doubles their userbase regularily, but it’s by no means a law and thats somehow the gist of your second paragraph…

stephan

July 27th, 2010

@Marc: Constant growth in percent is exponentially in numbers. I think this is clear from the post – perhaps go back and read it again? If it is still not clear I’ll add a paragraph.

Yuvi

July 27th, 2010

Doubling the amount of hardware as the initial amount of hardware increases is indeed exponential. But only because the problem has been set up that way.
I would set it up as the cost of adding capacity for n more users, which is linear in n, at least in this model.

I do see your point however, that software optimization need not necessarily grow more expensive with the amount of hardware backing it. In the real world however, who knows what the growth in costs is? I don’t. Programs often grow in sophistication with the number of users, so the program isn’t stable across this growth. Plus there are a few thresholds after which deployment costs jump.

stephan

July 27th, 2010

@Yuvi: You’re right, it will get harder and harder for software optimizations. There are low hanging fruit for the first 10% but then it gets harder and costs more money, until buying hardware or tuning the OS is cheaper again until tuning software is cheaper again and so on :-)

What people wrote somewhere else:

codemonkeyism

July 26th, 2010

Just blogged “Adding Hardware is not Always the Cheapest Option” http://bit.ly/bqMIE3 #scaling