If your application grows and you need to support more customers, there is the common wisdom to buy more hardware. Or solve performance problems with more hardware. But the simple math says there is a point where investing in software or OS optimization is much cheaper than buying more hardware. There are essentially three ways to optimize and scale your web application - assuming it can scale.
- Buy more hardware
- Optimize hardware - buy better optimized hardware
- Optimize your application - or your software stack
Suppose we have 10 servers and want to add 100% customers. Then we need to buy 10 more servers. Suppose we have 1000 servers and want to add 100% customers. Then, for the same growth, we need to add another 1000 servers. As this gets expensive very fast (exponentially)- not in the beginning of your startup but after some time - you need to look harder and harder into software optimizations. This is also the reason why Facebook benchmarks all it's new hardware to squeeze out every last bit of performance. If you have 10 servers and you squeze out 1% of performance, this is equivalent to 0.1 servers. If you have 10000 servers and you squeze out 1% more performance that's equivalent to 100 servers.
See the example for costs for 10% customer growth:
There are other factors to think of. Most often when talking about scaling, people talk about how their application scales linear. This correlates to your application having O(n) scaling. And in theory this is fine. In reality it's probably f*O(n). And different architectures, although scaling with O(n), have different linear factors. It does matter if f is 2 or f is 20. What f does your web application have?