Is this a dark side of the more distributed NoSQL options? Certainly. Every option has its dark side. The dark side of the less distributed solutions is that they’re more vulnerable to single-node failures. The dark side of the memory-centric systems (such as InfiniBrag) is that they have limited capacity. There are data-model tradeoffs, consistency-model tradeoffs, etc. all of which have to be considered.

Jan Lehnart of CouchDB has said that NoSQL is less about SQL or ACID than about choice. I mildly disagree, since I think we already had all of the choices he mentions, but I do think NoSQL is mostly about giving people motivation to *think* about the choices and tradeoffs they’re making when it comes to how they handle their data.

http://blog.datagraph.org/2010/04/rdf-nosql-diff

Hmmm, ok, you do need to use a sql-like language, or Visitors, to traverse your objects.

However, I can’t see another way, even if you do use unstructured; if you have data that is related, you need to walk it in some way.

http://blogs.oracle.com/datawarehousing/2010/01/integrating_hadoop_data_with_o.html

The impedance mismatch for your typical blog and an RDBMS comes when you consider having 2 tables…Posts, and Comments in an RDBMS vs having a collection of Posts each of which has an embedded collection of Comments in, say, MongoDB.

One of these naturally looks like your object model and one does not. One of which requires a lot of ORM machinery to traverse the gap between your datastore and your application….the other requires very little.

I think part of the reason a lot of people still think RDBMS are a good for ALL type of problems (excluding the very large scale ones where very few people still think they’re a good fit) is that we’re just so damn used to them.

I, for one, am happy to let my SQL knowledge become a bit less useful in favor of a happier development experience.

“If you have scaled up to the point where you need NoSQL, Ad Hoc queries are already out of the question.”

a.) Many companies need NoSQL early in their life cycle, and then still have not enough investment into a data warhouse and BI solution
b.) Many companies that are (too) large have complicated processes leading business users to just go to the DB department requesting ad hoc reports
c.) If your CEO comes to you requesting some information NOW for the next investors call, you tell him “this out of the question?” Cudos to you.

“ad-hoc repair is another matter – does it not indicate you haven’t thought through the operational usecases on the data?”

I don’t know about you, but I have strived for a zero bug strategy for the last years, and still there were bugs in the software we produced. Glad to hear that you do not have bugs because of edge cases no-one thought about or which are caused by the interference of different features in different departments meeting live data and users.

” Keeping olap/ad-hoc queries off the active database is a best practice for relatively small systems, [...]”

Not sure what your point is here.

Also not sure why you ignored “Also see the quote in the post.”

beg away, but you need to refute the argument and not just handwave. Keeping olap/ad-hoc queries off the active database is a best practice for relatively small systems, never mind the outliers than non-relational systems target.

ad-hoc repair is another matter – does it not indicate you haven’t thought through the operational usecases on the data?

The impedance mismatch that exists is between sets and objects. There is nothing about a blog post that makes it more natural to be thought of as an object. Sets of blog posts and all the regular relational juice is still applicable here.

When you go NoSQL you are deciding the forego flexibility with your data in favor of raw speed and scalability. Don’t fool yourself into thinking that NoSQL is more “natural” just because you don’t like the leaky ORM abstractions.

But in general, I agree that most developers jump into NoSql without operations consideration. Hopefullly that’s just a symptom of the novelty of these data stores and will work itself out as they become more established.

As mentioned above already, MongoDB, while requiring a new set of skills, does provide all three of the capabilities you mention.

Some examples are OneTick, MonetDB, Vhayu, KDB.

Our application developed in Rails was designed to use sqlserver because no other product do what we need like reporting service.

I would have loved to use an open source database and make the ecosystem more consistent, but really, i dont know any system that can compete from far with reporting services.

The most common forms of web applications – blogs, forums, CMS, wikis – are all examples of structured document storage, and have a natural fit with NoSQL and an impedance mismatch with SQL.

In such cases key-value stores offer higher productivity in development, and higher runtime efficiency.

CouchDB does support “temporary views”, which are ad hoc views that you can build and execute on the fly. However, temporary views are not recommended for production use, because they need to be built before they can get you the data you need. This could take hours, or days depending on the size of you database and the processing power of your database server. Temporary views are meant as a way to test new views in development which will eventually be saved into a design document, and not for running ad hoc queries.

10 million rows of normal data is not NoSQL territory.

I agree. I see noSql as something to start using when the existing relational datastore can’t handle the load or maybe for quick solutions.

1. Agility: Are they agile? If a user at a later date wants a new query that spans across 3 or more entities can I as easily do that with noSql as I can with Sql?

2. Conventions over adhoc design: If a developer can put anything into a database, they will. This means that as years go buy the datastore becomes murky. I’ve worked with object-oriented database, XML database and old-school IMS hierarchy database and I can tell you that as time goes by these can start to look very ugly.

So, you may choose to split your data among (pseudo-)functional/semantic paths as you suggested, or just put everything in your NoSQL store and write behind to your RDBMS … in both cases, RDBMSs will be part of your architecture for long time.

That’s one reason why NoSQL is indeed a bad name: a NoSQL store doesn’t imply no RDBMS at all.

Just my two euro cents,
Cheers,

Sergio B.

“Certainly in the businesses I’ve been involved in, ad hoc reporting has been phenomenally important and to lose that capability would be a deal breaker when considering moving to a different platform.”

Yes, experienced that too. But sometimes adhoc reporting is important (realtime), other times it’s just curiosity, like looking into your email inbox, twitter, …. and a 2 hour gap due to hadoop e.g. is tolerable (for the company, perhaps not for the curious employee)

Aside from that, your third point is a bad one.

Whilst it may be true that your customers expectations are that ad hoc reporting is simple for you (when using traditional SQL, for instance) because each time they ask for something they get it within five minutes, and that they can be trained to expect it to take longer, it’s no good when your boss needs to make a business decision and it will take several days to get him the numbers he needs.

Certainly in the businesses I’ve been involved in, ad hoc reporting has been phenomenally important and to lose that capability would be a deal breaker when considering moving to a different platform.