I’ve spent the last 4 years developing Lotus Domino applications – most of this time I worked on legacy applications.

And I have to say that most of you people will be hugely disappointed by the NOSQL world (Domino is one of the Original NOSQL database systems :)).

Why? Because in NOSQL databases schema is implicit and hugely dependent on context. Meaning that when you work on data that is 10 years old and that had schema changed 10 times in between can be HUGE PITA!

I have learned though that “schema” (although implicit) should always be well documented – especially the transitions between them.

When you sift through this kind of data and start getting “errors” like date fields with prosaic values, you realize how important it is to keep track of your design :).

What I’m trying to get across is: There is no silver bullets. If RDBMS doesn’t cut it for you then neither ORM neither NOSQL wont.

Yes there are problems out there that are more suited for this type of DB than the other.

BUT! There is no substitution for discipline, getting your basics straight and for developers having an good ol’ experienced DBA behind their necks.

I believe ORM’s are great for prototyping, maybe some smaller stuff – and that’s it. Its not that they wouldn’t be able to “get it done”. Its just that when shit hits the fan – you really won’t know how to help yourself out.

And cocky young developers (who don’t believe DBA’s) have a tendency to do just that! :)

“There is nothing wrong with the mapping part.”

Then you don’t need use an ORM, iBatis or another simple mapper is sufficient.

People use ORMs because it transparently handles associations and generates all queries. The mapping part is the simple one.

With rowMap in Spring JDBC one gets a lot of things.

The way you put it is coloured and when reading I thought you where completely right. But then I started to think about the alternatives.

When you are using Spring JDBC you have to write your own SQL and do the mapping yourself. This will prevent much of the N+1 mistakes, sure. But why does it prevent these mistakes: Because we are writing SQL again.

The problem doesn’t come with the ORM as mapper, the problem is that people let the ORM-framework come up with queries! We need to write the queries ourself! There is nothing wrong with the mapping part.

IMHO it would be best if we had ORM mappers that take SQL queries and maps the results to objects. Then the people who still understand SQL could write the queries, and the resulting objects can be used throughout the program.

ORM isn’t a thing of the past, SQL-generation might be.

The other day I heard one of them say, “I used NHibernate to solve a problem. Now I have two problems.”

The ORM should be able to automatically map plural table names.
The ORM should have no mapping files and have default conventions that make sense.
The ORM should support standard ANSI SQL not some crazy variation.
When an error occurs, not only should the SQL be displayed but also the parameters. The SQL should be nicely formatted.
The ORM should detect when a field length is too long and display the field name and the text that was tried.

I wish Hibernate would do all that.

These days, a flagship product that is heavy on full range of database interactions (especially query) is using iBATIS. We’re finding iBATIS strikes a pretty good balance. We hand write the SQL (and often call stored procedures), but iBATIS assist with the tedium of mapping inputs and result sets to Java objects. It also nicely integrates with Spring Framework, where Spring is the bean factory facility for BlazeDS services, DAOs, iBATIS, and JMS message processing beans.

iBATIS is a good way forward for large scale projects that don’t want to compromise on the SQL layer that is used.

Well, here are the news: bad developers will mess up ANY tool that you can imagine. Anyway, thanks for the provocative text, but I was expecting some alternatives instead of just negative comments.

Kind Regards

http://blog.devtrain.fi/2009/10/18/kauklahti-making-orm-simple

My solution is simple extension to JdbcTemplate to support joins and CRUD generation for class hierarchy. Also ResultSet extraction into objects is provided.

../genericdaotest/src
jsmith
jsmith@smith.com
dto.vm
dao.vm
applicationContext.vm

select tripid, carrentalid, provider, city, pickupdate, returndate, cartype, rate, bookingstatus, lastupdated from carrental limit 1

select tripid, carrentalid, provider, city, pickupdate, returndate, cartype, rate, bookingstatus, lastupdated from carrental where carrentalid = :carrentalid
insert into carrental (tripid, carrentalid, provider, city, pickupdate, returndate, cartype, rate, bookingstatus, lastupdated) values(:tripid, :carrentalid, :provider, :city, :pickupdate, :returndate, :cartype, :rate, :bookingstatus, :lastupdated)
update carrental set tripid = :tripid, carrentalid = :carrentalid, provider = :provider, city = :city, pickupdate = :pickupdate, returndate = :returndate, cartype = :cartype, rate = :rate, bookingstatus = :bookingstatus, lastupdated = :lastupdated where carrentalid = :carrentalid
delete from carrental where carrentalid = :carrentalid

Then I tried Rails 3 years ago and now I’m absolutely in love with ActiveRecord. Paired with the power of ruby there’s no way I would write my own DAO again (or use another platform – hehe). If properly used, it generates SQL that could hardly be written better “by hand”.

In this particular case, I see “hard to change” as a feature. When you change a type in an existing schema (something that’s pretty rare, to be honest), you have to do some extra work in order to give the database as much information as possible. In return, you get static safety, optimized queries and many other features that we have learned over the past decades in the RDBMS world.

Consider this analogy: in a Java program where all the objects are of type Object, changes are very easy to make, but you will most certainly break a lot of code doing so.

–
Cedric

Basically everything written here:

http://codemonkeyism.com/dark-side-nosql/

applies to them.

Sometimes I think we need something completely different. Should relational data be “mapped” to objects? What would a relational domain model look like?

Base: extended by the implementation classes. Contain all the getters & setters, which you are free to override in the implementation classes. You never modify the Base classes directly. They are always fully auto-generated. If you want, you can even customize the class that generates the Base classes, so they all have the same customized core functionality.

Implementation: allows you to customize the getter/setter for any property, and to add your own for virtual properties, etc. The Implementation objects are also host to your save & delete methods. Implementation classes are not regenerated/overwritten when you regenerate the Base classes due to a schema change.

As for selecting data from the database, this is where Propel starts to shine. It actually generates another class for each table called a “Peer”. They are essentially containers for static methods for fetching rows of that table. They come with a few built-in queries, which you can add to. Example:
PersonPeer::retrieveByPk(42);

You can also select using “Criteria” objects.
Example:
$c = new Criteria();
$c->add(PersonPeer::FIRST_NAME, ‘Derek’);
$people = PersonPeer::doSelect($c);

You can also select using straight-up SQL if you like, and then populate your Implementation objects from the result set. Although you lose database portability by doing this (versus Criteria), you gain the benefit of having very straightforward easy-to-read queries.

Altogether, it gives you quite a flexible abstraction layer, which has helped me out of more than just one or two tricky situations.

For example, I’ve customized the Peer objects so that they automatically cache my result sets, and the Implementation objects so that they automatically save the History of all object changes.

Using Hibernate/JPA+Spring and some Java annotations I wrote the DAOs of the Parancoe meta-framework:

http://docs.parancoe.org/reference/html/persistence.html

Using these DAOs developers have a very little interaction with the ORM system: they only need to define the mapping, and the queries (using HQL or Criteria API), nothing else. They become productive immediately, and the possibilities of being wrong using the ORM are reduced at the minimum.

I’ve been developing over three years with Lotus Domino / Lotus Notes platform with it’s own databaseformat NSF which is a NoSQL database. Adding fields, sorting, data replication with many other instances of the database were such a charme. Our complete CRM was implemented as many databases in this hierarchical structure and it worked very fast on huge amount of records and data. Everything was documented in it even binary files (.doc, .xls, .ppt and so on).

The only “problems” we had were that we’ve encountered a “missing link” with the “classical” business reporting systems which mostly do only support relational structures. And sadly with many consultants blaming that databases because of they only know relational ones. But if they had to change their tables (datatype migration, add one field, …) there was a need to take their application offline. With the NSF database we just developed (using DAOs) in a testenvironment and deployed it to production. All changes were replicated automatically to other instances of the database.

The downside is highly increased application code and logic, because you need to deal with NULL values.

From my experience though, over years NULL values also creep into RDBMS.

Bad code may not be a problem with ORM per se. The implementation framework should make it extra-hard to do it wrong, and should not hide details under three levels of abstraction within the framework itself (this is what people call “magic” [in a bad sense] when they do).

For example, Hibernate’s Level-1 Object cache may be fine for some use-cases but in general, trading lazy-loading/sync’ing for freedom from Level-1 blues appears to be a good balance. “Hibernate Lite”, anyone ?

Another gripe is with XML and annotations. Why not create DSL-like API for declaring entities and associations? (Think SQLAlchemy)

http://blog.mongodb.org/post/215738382/databases-should-be-dynamically-typed

Take a look at the comments on the blog post, and also on Hacker News:

http://news.ycombinator.com/item?id=887605

Sorry, but I’m still not overly impressed with the alternatives so far…

Just like you, I have seen a lot of horrible code written in the ORM layers, but do you have any evidence that these problems will disappear with NoSQL and other alternatives?

At any rate, you can’t dismiss a technology just by observing that it can be misused. After all, C++ is still going very strong today…

–
Cedric

I think the real way to go is integrate queries into the language, like LINQ does. Though not constrained to ORM’s, it proves tremendously powerful as you get type-safety, syntax help etc. unlike the nasty misuse of annotations and string concatenation known as JPQL.

The same is true for web frameworks (or any framework). Is Grails or Seam bad because there’s too much “magic”? Should you prefer plain Servlets?

That being said, if you want to have a simpler ORM framework, you should take a look at iBatis. That’s a very low-level framework that doesn’t have the complexity of a full blown ORM tool, but still offers a good way to map Java classes to a database.

btw i use hibernate a lot and i am still happy with it (it does fit my needs). and if persistence is not a part of your core business, you should stay with hibernate or any orm you use today…

so i think “orm is dead for somebody”…

I switched the group to doing the majority of effort in Scala. Mostly using a tweaked version of the “pimped Connection” too -> basically, just doing straight SQL programming.

The end result: vastly improved quality and performance. For our situation, it’s all we needed as well, and as the system flushed out, the devs just have a better idea of what’s going on. And I’m talking about “this part in memory, this part in the DB” sort of awareness. They just missed and mixed up everything when using an ORM.

In the hands of a master, Hibernate can be impressive. But in a company where the devs are constantly hacking in features, I found it a nightmare.

“Time-to-market is at least one reason why we still need ORM. I would recommend ORM for prototyping.”

I do agree on that one, yes. If you need a short time to market or do a VC presentation, use an ORM. Or better use Rails ;-) and drop it afterwards.

Ah the Java argument again. We do differ on that one +g+

I do not think Hibernate will be an obvious choice for projects in 10 years, not so sure about 5 years.

I’ve seen ORMs in multi-million line projects and in smaller ones.

They do help write less boiler code, but people often do misjudge the amount of SQL code they really need beside CRUD.

And many queries I have seen are aggregates. Where a simple count(*) or sum(*) would be enough, developers fetch all objects via Hiberante into a List and then call size().

And there are many more defiancies: The lack of interface support in Hibernate for example often prevents good code. The tight coupling of objects to normalized database schemas often prevent domain object refactoring. And I could go on.

Nevertheless, I think ORMs are established and used in many new projects. Even companies that can’t rely on 3rd party ORMs, build their own.

On the other hand, sometimes a good ORM can even predict and optimize the generated SQL queries. Granted, the manually crafted SQL queries can’t be easily beaten, but for most scenarios you don’t need them and if you do, you just break the abstraction yourself (and that’s possible if your ORM choice was wise).

Spring JDBC absolutely rocks, but can you deal with the time-to-market demand on a medium / large project with that? Time-to-market is at least one reason why we still need ORM. I would recommend ORM for prototyping.

Key-value data stores are going to mature and get better or probably we will begin to see hybrids. Till then, I guess we just need to grin and bear ORM.

Well, you’re not offering much of an alternative, are you? :-)

Sure, ORM’s have well documented deficiencies, but they’re really the best solution we have right now, and to be honest, they perform much better than what you claiming.

As for these other solutions that you mention (NoSQL, Polyglot Persistence), they are far, far, far from having proven themselves in even the simplest scenarios, and their shortcomings have already been widely documented.

Hibernate and JPA will still be around and dominant in 5-10 years, as will Java.

–
Cedric

I’m sorry, but if I have a PERSON table with a FIRST_NAME and a LAST_NAME, why, in this day and age, do I need to put together a class named Person with fields named firstName and lastName and glue it all together myself? Not everyone is building amazon.com and Facebook. In fact, few are. And for the vast majority of us, ORMs like HB or AR are more than sufficient. Rolling your own is so ‘98.

Yes and no. No because you will not use the magic of ORMs like session managment, change management etc. Yes because you will need to manage your associations by yourself. But association managment with ORMs has the same problem as RMI: It seems to be transparent, but it is not (lazy exceptions and remote exceptions).

Of course your points are valid, but really the problem is: what do you gain by not using an ORM? You are solving the problem that you have in some of the cases with an ORM for all SQL Statements. You loose the power of refactoring your database with your domain objects. You have to track the changes you do yourself.

You are basically building your own specialized ORM. Again.

As to Scala and non sql databases. I say: Nice ideas, but by far not enough ressources available for usage in many projects. Hell, often we have to fight for using Java 1.5 because big companies are still stuck at Websphere 6.0