Why NoSQL Will Not Die

Reading the recent flamory piece "I Can't Wait for NoSQL to Die" from Ted Dziuba, I thought the author is wrong on so many levels. Or as Jeremy writes:

Well done, Ted. I laughed to myself a few times reading your post.

Not that I'm a NoSQL zealot, see my The Dark Side of NoSQL, but Ted is hilarous. On to our first laugh:

Never mind of course that MySQL was the perfect solution to everything a few years ago when Ruby on Rails was flashing in the pan.

No it wasn't, without heavy memcaching MySQL never worked for websites. Or:

Well, no. Did you know that Cassandra requires a restart when you change the column family definition? Yeah, the MySQL developers actually had to think out how ALTER TABLE works, but according to Cassandra, that's a hard problem that has very little business value. Right.

It seems to me, Ted has never worked with real live MySQL applications. ALTER TABLE is a pain, for tables with several millions of rows it will take hours to alter a table, mostly because MySQL creates temp tables. Which is no problem if your domain and market is static - as I assume Teds is - or if your MySQL schema is meta. But for others this is hell. All the while it does locks and your website is heavily impaired during the change. Even dumping the table, recreating it and importing all data (which is faster than ALTER TABLE) takes usually hours. You can work around this with hardware, SSDs, a clever slave setup, but you need a MySQL wizard to get this working.

The real solution to schema changes with high volumes of data is not to have a schema at all in your store - something most NoSQL databases support. This is mostly done by storing XML or (B)JSON into the store, and the store does not care about your schema. Your app then needs to deal with different versions of a schema (with at least two) and migrate data from the old schema to the new one between reads and writes (NoSQL to the rescue: Store JSON data with a version string, read old version, change, write new version). Or deal with optional values from the beginning, something a lot of code already does with sparse filled social media data. A background job can also migrate data piece by piece to a new version. With this setup, schema changes are easy, without a complicated slave setup or downtimes.

The problem with RDBMS doesn't end there. In a post to High Scalability Joe Stump writes:

Precompute on writes, make reads fast. This is an oldie as a scaling strategy, but it's valuable to see how SimpleGeo is applying it to their problem of finding entities within a certain geographical region.

I wrote in more detail about this in "Essential storage tradeoff: Simple Reads vs. Simple Writes" and how RDBMS wrongly optimize wirtes (I know about materialized views).

m3mnoch speculates about the reasons for Teds laughable post:

it doesn’t look like he’s ever done anything for a large, mainstream audience. i bet he still thinks getting slashdotted or techcrunched is the definition of “a lot of users.” [...] my point is, this isn’t 1998 anymore.

Agreed, even my humble WordPress blog on one server survives this. The reason MySQL works for Ted is:

because i totally believe that google adwords runs on mysql. IT’S READ-ONLY! that’s what mysql is good for — lots of read-heavy, cacheable data you can map against other read-heavy cacheable data.

Back to Ted:

You Are Not Google. The sooner your company admits this, the sooner you can get down to some real work.

He's right. You are not Google, so you will not have those MySQL wizards around who write patches for InnoDB. And he's wrong. You will get into scaling troubles with MySQL far before you're as big as Google.

DBAs are a reason for NoSQL

DBAs should not be worried, because any company that has the resources to hire a DBA likely has decision makers who understand business reality.

Another real gem. One of the reasons people go for Cassandra is that they do not need as many DBAs as with MySQL. Clustering and Scaling works out of the box for a wide range of scenarios - cases which you would have needed a MySQL wizard to reach. In Joe Stump words, CTO and co-founder of SimpleGeo:

How much time are your DBAs spending administering your RDBMSs? How much time are they in the data centers? How much do those data centers cost? How much do DBAs cost a year? Let’s say you have 10 monster DB servers and 1 DBA; you’re looking at about $500,000 in database costs.

The cost of RDBMS operations

And more about the cost of operating RDBMS with large websites:

I’m running a 50 node cluster, which spans three data centers, on Amazon’s EC2 service for about $10,000 a month. Furthermore, this is an operational expense as opposed to a capital expense, which is a bit nicer on the books. In order to scale a RDBMS to 6,000 reads/second I’d need to spend on the order of five months of operation of my 50 node cluster. [...] I’m happy to put my $/write, $/read, and $/GB numbers for my NoSQL setup against anyone’s RDBMS numbers.

SQL databases will survive, but for a much smaller niche (transcational data) than today. For sure NoSQL will not die in the near future: They support schema changes better, they scale better for write heavy applications and they are cheaper to scale all in all.

Other NoSQL posts on CodeMonkeyism:

The dark side of NoSQL

There is a dark side to most of the current NoSQL databases. People rarely talk about it. They talk about performance, about how easy schemaless databases are to use. About nice APIs. They are mostly developers and not operation and system administrators. No-one asks those. But it's there where rubber hits the road.

The three problems no-one talks about - almost noone, I had a good talk with the Infinispan lead [1] - are:

  • ad hoc data fixing - either no query language available or no skills
  • ad hoc reporting - either no query language available or no in-house skills
  • data export - sometimes no API way to access all data

In an insightful comment to my blog post "Essential storage tradeoff: Simple Reads vs. Simple Writes", Eric Z. Beard, VP Engineering at Loop, wrote:

My application relies on hundreds of queries that need to run in real-time against all of that transactional data – no offline cubes or Hadoop clusters. I’m considering a jump to NoSql, but the lack of ad-hoc queries against live data is just a killer. I write probably a dozen ad-hoc queries a week to resolve support issues, and they normally need to run “right now!” I might be analyzing tens of millions of records in several different tables or fixing some field that got corrupted by a bug in the software. How do you do that with a NoSql system?

  1. Data export: NoSQL data bases are differently affected by those problems. Each of them is unique. With some it's easy to export all our data, mostly the non distributed ones (CouchDB, MongoDB, Tokyo Tyrant) compared to the more difficult ones (Voldemort, Cassandra). Voldemort looks especially weak here.
  2. Ad hoc data fixing: With the non-distributed NoSQL stores, which do posess a query and manipulation language, ad hoc fixing is easier, while it is harder with distributed ones (Voldemort, Cassandra).
  3. Ad hoc reporting: The same with ad hoc reporting. The better the query capabilities (CouchDB, MongoDB) the easier ad hoc reporting becomes. For some of those reporting woes Hadoop is a solution. But as the Scala Swarm author Ian Clarke notes, not every problem is applicable to map/reduce. Either way you need to train customers and their expectations as they have become addicted to ad hoc reporting. This is not only a technical question, but a cultural one.

One solution is to split data that needs to be queried or reported (User, Login, Order, Money) and data which needs best performance (app data, social network data). Use a tradition SQL database for the first kind of data, and a fast, distributed NoSQL store for the second kind of data. Joining will be difficult, you need to support more different systems and skills are an issue. But the three problems can be solved this way.

What is your NoSQL strategy? Please leave a comment, I would like to know.

[1] they plan a distributed query language for ad hoc reporting in distributed environments

Essential storage tradeoff: Simple Reads vs. Simple Writes


Photo by Robert Scoble

When I studied computer science with a focus on databases, the holy scripture said only to use normalized data. For my exam I had to study 2NF and 3NF. And now it seems this is all wrong for large scale operations. From my education, I still have a strong feeling towards the pureness of normalized databases - and detect this feeling in many developers. In reality denormalized schemas are the way to go.

Why the former rise of normalized schemas for relational databases? Normalized data is pure. There are no repetitions, no duplication and data can easily be added, changed and removed. The database enforces a consistent state. But to read data from several tables you need to join them, which means complex and - often - slow reads. The tradeoff.

The rise of NoSQL storages

Joining doesn't scale to millions of concurrent users, and it's rumored that some companies ban joins completely. NoSQL databases, which include key/value stores and document databases, drop the notion of normalized data. Those with the biggest momentum are Cassandra, Project Voldemort, Tokyo Tyrant, Redis and CouchDB.  They store data or documents in denormalized chunks which leads to easy reads. But if the data needs to be present in several documents, this means complex writes.

Simple Reads vs. Simple Writes

Essentially you exchange simple-writes/complex-reads with complex-writes/simple-reads. In most modern web architectures with many more reads than writes - depending on your business model - it's no wonder people prefer the second tradeoff.

An example

Let's take a look at an - contrived - example. Suppose we want to store a shopping list. Each shopping list can be tagged and also have friends who can see the shopping list. With a normalized database one would perhaps model this problem like this:

Normalized example for databases

Compare the normalized form with a document schema:

Document based example for NoSQL
We added the tags and friends (in human readable form) to the shopping list, and the friends to the user document.

Read and write use cases

Let's examine two use cases:

  1. Adding, removing and changing the name of a friend
  2. Reading the shopping list

With the normalized schema, adding and removing a friend to a shopping list is easy: Just insert a row or remove a row in the Friends table. Changing the name of a friend is also easy, just change it in the User table. Reading the shopping list at least involves a join with Tags and so isn't a one-chunck read access. If we want to display the friends for a shopping list too, there even will be a second join.

With the document schema, adding and removing a friend is more complex. We need to read and write the shopping list document. We also need to read and write the user document. Reading is simple though. Just read the shopping list document, there is all the information to display a shopping list with the tags and the friends. Changing names is most difficult, we would need to change all shopping lists. This is the primary argument for normalized databases. But in reality, how often do users change their name compared to viewing a shopping list? And most of those slow operations can be done in a second, asynchronous step (see picture above).

Be aware of this tradeoff and plan or change your architecture accordingly. But what do you think?