but the problem is frameworks and ORM are also developped for relationnel datas

for example in PHP (yeah I know ! but now with frameworks and architecture it’s powerful for “web2.0″ or social, see drupal or facebook!) there is a very good ORM like Doctrine for data components.
but if you want to use these new model, I suppose you have to throw all ORM you liked so muche and make yourself all data work. paintful !

don’t know if it’s supported in Rails ?

for me the problem is not if we can use these database, but if they are integrated in application frameworks.

shoppinglist.friends[0].name = ‘Johnny’

And you can also store references to other columns, so if your list of friends can be a list of references to other documents describing users (kinda like a relational database would do), so all you would need is, too, to update the user document.

You can also have server-side functions written in javascript to do these kind of updates, so that you need not transfer info around server client, just call the server-side function and it will do the update locally. You can also send plain javascript as a query, like an UPDATE, to do what is needed.

Whatever you think of the relational model as a logic user model, remember that it is intended as only a logical model.

Of course pointers for foreign keys, co-locating related entities, and relaxing constraints, can help performance. Those things are a part of the physical model.

The weakness is that to a greater or lesser degree, the SQL products do not support some
of the physical models you want. That seems to me to be purely an implementation issue.

http://www.julianbrowne.com/article/viewer/brewers-cap-theorem

My solution for the moment is sharded databases, with a healthy dose of de-normalized aggregates sprinkled throughout the model, which are kept in synch with triggers or controlled data access via stored procedures. There’s no chance of data being different in two places.

And there’s no edict on how to write queries – if you join 20 tables and it runs fast, fine. If you need to write parallelized queries to return raw data from one table at a time and then join them on an application server, then that’s fine too.

I think a lot of people new to this game don’t get how good a SQL database can be. On a single shard (a cheap 2U box), I can run 5 million customers, which is something like 150Gb of data, and everything runs fast enough to keep people happy. In a NoSql system, how many nodes would you need to support both transactional and reporting applications for that much data?

For ad-hoc reporting we BI experts are currently facing the arising of coulumn based and column indexed storages, mainly in combination with in-memory technology.

As developer one does data modelling in his / her best way, in preparation for access things are split into columns and links between. Far away from 3NF or documents.

I supose it’s worth having a look at during your further investigations. ‘am looking forward.

Regarsd
Rayk

In terms of chunking, I find extremely complex “materialized views” are very useful. I haven’t used the Oracle system, but I’ve written some pretty complicated update routines that I would be surprised to see in standardized packages. So I think your goals can be best realized (in most environments) with sophisticated systems comparable to materialized views – documents that update automatically based on an underlying verifiable data system.

“I think it’s worth mentioning that some SQL databases (Oracle for example) provide out-of-the-box solutions for this problem as well and arguably in a more seamless manner. For example, creating a materialized view based on a complex query gives you the benefits of a denormalized schema.”

There are other benefits of chunks in documents and values though. Distribution comes to mind, which is inherently more difficult (might I say impossible above certain levels) to achieve with Oracle. And of course you can have denormalized data in every SQL database.

But they only scale with massive caching. Today many applications experiences millions of concurrent users. Then you need to trade consistency with scalability.

As I’ve said to Stefan, the downside is “mainly the lack of adhoc reporting and adhoc data fixing.”. Which sometimes is a major issue.

So for user data, payment data, orders it’s still a good idea to keep your data normalized.

Cheers
Stpehan

I will likely stick to Normalized Data for the majority of my applications, but I will have to keep an open mind towards DeNormalized data as well, I suppose.

Cheers
Stephan

PS: And many more things

very neat and clean presentation of the trade-off those two principles.
Unfortunately there is more involved in such a decision, but hopefully you’ll get into that later ^^

Chears
Stefan