There is nothing in the relational technology that obliges storage and search to be unified. It is the POJO/DAO mindset that combines them. This mindset is the very essence of the long-bemoaned object/relational “impedance mismatch.”

My preferred persistence framework is Oracle ADF Business Components for Java. Suggest you take a look at it for a database guy’s perspective on persistence programming.

Best,

Andrew

“I was hoping that someone somewhere smarter than me had figured out how to bridge querying disparate systems in a better way.”

Uh, smarter, than I’m most possibly not the right person.

Perhaps the joins are only needed for reporting, if that is the case it would be best to write the data also into a OLAP for reports.

Like http://mondrian.pentaho.org/

The decidedly brute-force, ugly, smelly, hairy, nasty and yet strangely cool approach I took at one point was to make a kind of query builder that, in conjunction with some simple lookup and filtering facilities implemented on Storage instances–but again, not true searches–whittled down the sets of items from each Storage to be combined (so the Student Storage was able to filter using a simple where clause/predicate, and the Class Storage was able to do the same). Then I loaded those sets into a temporary database (H2–obviously could have been anything) and did the more complicated joining there (minus the simple where clauses/filters that were used to get me the candidate sets). (A tip of my hat to a former colleague for first exploring this approach.) The result, of course, was dog slow and could not be used on enormous datasets, but performance was never a priority and the client understood that they would pay dearly in performance costs for this approach. I was hoping that someone somewhere smarter than me had figured out how to bridge querying disparate systems in a better way.

Thanks for the links to the papers; very interesting reading.

Have you read the transaction apostate paper and the Amazon dynamo paper? Sometimes it seems it isn’t even possible to have data on one machine to join it.

http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html

http://www-db.cs.wisc.edu/cidr/cidr2007/papers/cidr07p15.pdf

But if you have new insights and a solution to the problem of data joining of disparate stores, please drop me a line.

Obviously, a Compass/Lucene-type search handles a huge number of cases–people tend to like to search by keywords, and so a coarse-grained search/locate strategy like that makes a lot of sense. But in some of the applications I work on, careful targeted queries that join bits of two entities together–a classic SQL join–are also needed.

Have you found a convenient way to expose a *common* SQL-like query mechanism across items that use different Search implementations?