Scala vs. Clojure

I’ve been taking a look at Clojure, and to be honest, I’m a bit underwhelmed by its syntax. Oh, it’s not all the S-expressions, those I kindof like, but the Clojure syntax is full of lots of other weird constructs which make it look less Lisp-y. Square brackets for vectors, curly braces for maps, *hash* curly braces for sets, hash parens for function literals and then % to replace parameters within. It’s all very squirly and seems to me to be not at all according to the Lisp philosophy.

With that said, the idea of a purely-functional Lisp (other than Scheme) is very appealing. The unification of data structures brilliance incorporated; and for a language without objects, Clojure has some remarkably solid integration with Java.

I would bet money *against* it being the next dominant language on the JVM, but I don’t think Clojure is going away any time soon. Inevitably, Clojure is going to find a niche in supporting and scripting bulkier infrastructures than have a hard time with certain features (particularly concurrency). It should be interesting to see how things develop.

Interesting opinion. For me it’s more about S-expressions. as the syntax is mainly Lisp I think – you’ll like it or hate it. I tend to the second opinion. For most people in the enterprise it’s hard to read and hard to grasp.

“[...] but the Clojure syntax is full of lots of other weird constructs which make it look less Lisp-y.”

Could make it more appealing for people outside Lisp though.

“I would bet money *against* it being the next dominant language on the JVM”

As said before, I don’t really think there will be a language replacing Java on the VM – Java fits the enterprise model too well. But if I should bet, it probably is Scala.

Clojure could replace Lisp dialects though – if people start to overcome their hatred for the JVM.

“It should be interesting to see how things develop.”

Definatly.

Well I think that is more about Groovy vs Scala. Clojure is too exotic for a typical developer.

@Lite: Not sure. Groovy doesn’t bring that much to the table for enterprise developers (MOP isn’t that interesting) that Scala doesn’t and type inference looks superior to non-typed references in Groovy to achive the same: less type annotations. Groovy will only make it in the enterprise when static types are enforced (only my personal opinion, no proof just a bet.)

The Java ecosystem is huge, and I have to believe there is plenty of room for many successful languages on the JVM, none of which is likely to displace Java.

It certainly seems counterproductive for Clojure and Scala to battle, as they are both on the same team, in some sense, in trying to foster a more functional programming approach.

That one is statically typed and one dynamically will probably partition potential users naturally.

“It certainly seems counterproductive for Clojure and Scala to battle, as they are both on the same team, in some sense, in trying to foster a more functional programming approach.”

I’m not sure about that. In the end it’s about what people use – I don’t think there is this “we all together broaden the FP approach to more people”. When it comes to success most open source projects I know want to be successful – sometimes even agressive (see Spring vs. Jboss – agreed both want to make money)

Mindshare isn’t divisable I guess.

“That one is statically typed and one dynamically will probably partition potential users naturally.”

Yes, might be.

The best solution should be not to replace Java is learn from both languages Scala and Clojure and apply the concepts to Java. As C# it learns from F#.

Java language have to continue to evolve, if not it can stay in the past.

Regards.

Note: Im thinking that Scala and Clojure still difficult to use for the average Java programmer so better get the good ideas from Scala and Clojure and apply them to Java, All the Java community should get together and do pressure to the JCP so it becomes a reality, at least we need “Closures or even Delegates”.

Personally, I see Clojure and Scala as complementing each other, not in competition. I have a hard time believing that Clojure could be used *exclusively* to create a serious, full-scale application. Don’t get me wrong, Rick, your presentations are great, but I honestly don’t think that I have the sheer mental ability required to keep track of a dynamic system of truely enterprise magnitude. That’s where Scala comes in…

Conversely, Clojure is really, really good at expression high level actions, particularly one requiring a high degree of concurrency. Try as I might to borrow all the good ideas (brilliantly persistent datastructures, STM, etc), they still fit more naturally in Clojure. Besides, as a dynamic language, Clojure can be a lot easier to work with for some things.

In short, I see the enlightened systems of the future built solidly on a Scala foundation using Clojure and sometimes even JRuby or Groovy to control the higher-level logic. There’s something to be said for mindshare, but I think from a purely technical perspective, none of these languages could ever completely oust the others.

P.S. Depressingly, I think that most of the enterprise world will keep right on building “unenlightened systems”, meaning that they will pick the least common denominator (Java) for a long time to come.

I think “Java” will be replaced. Note I put the name in quotes deliberately. The original Java is already dead, and version 1.7 will further stomp on its corpse. Inner classes, generics, annotations, and now closures. Those aren’t intrinsically bad features but each was crudely bolted onto the deliberately non-extensible core of the original language. Java’s new features almost have to make it less Java-ish. So, basically the problem for “Java” is that the simplicity that sold it to the enterprise has been thrust aside. Is Scala harder? It has slightly more features but the syntax is more regular and can absorb new features with more grace. I could see it winning.

I don’t think Clojure has much chance of winning in risk-averse big business, but I see it being an ongoing way for small, clever businesses to distinguish themselves.

So you are saying a programming language makes a business distinguish? I dont agree.

What about Facebook? It use PHP and not Java or Scala or Lisp. And thousands of startups and business use simple tools without a fuzz and are the winners right now. I dont think the programming language matters, are the developers the key. The brains behind all those ideas are the winners, Programming languages are just a simple tool.

So I just hope that Java include closures because we will have Java maybe for another 10 years to develop, minimum.

Being underwhelmed with syntax is a good thing – is there a need for extra bulk when you have the expressiveness in 3/4 structures? To my mind lisp, and clojure are very elegant in this respect. I think this is particularly important when it comes to tooling, parsing, and understanding the source.

Clojure has namespaces which allow you to partition your code. Clojure has removed objects and types but kept polymorphism (multimethods) and ad-hoc hierarchies, which provides much of the convenience of oop without having to specify and lockdown type in advance – which much better fits the dynamic model. On thinking about Rich’s perspective i now believe the separation of data structure from function is better on the whole than oop; just take for instance on the fly redefinition of functions – when there’s no state to worry about it’s a no brainer, and so useful for interactive debugging.

Tooling is coming along, for the enterprise developer enclojure for netbeans looks great, with source debugging etc. For the hacker emacs is available with inferior lisp mode and slime/swank integration.

In practice I haven’t used clojure on a mega scale project but it’s easy to see how it would be done – and we haven’t yet mentioned the benefits of clojure’s concurrency model!

Hi,

One of the reasons why Scala does not implement Iterator directly is that Iterator contains a remove method and that goes against the general design of the Scala collections where mutators only appear in certain subclasses.

Scala has made some sacrifices in the name of interoperability, but in this case I think the right choice was made in providing wrappers in scala.collection.jcl instead. Combined with the implicits in scala.collection.jcl.Conversions, it’s easy enough to work with Java collections in Scala code.

I agree that passing native Scala collections to Java should have similar support, but it’s not particularly hard to do that yourself.

Ismael

@Ismael: “One of the reasons why Scala does not implement Iterator directly is that Iterator contains a remove method and that goes against the general design of the Scala collections where mutators only appear in certain subclasses.”

Nevertheless I prefer the Clojure way. And I would prefer better integration – with perhaps an exception thrown for remove – than the need to either have implicits that create new objects all the time or use wrappers by hand.

Tight integration with Java – as shown in Clojure and e.g. Groovy – is needed for success I think.

“[...] it’s easy enough to work with Java collections in Scala code.”

No, it’s hard. You always need to think about it. Currently I read “Practical API design” – an excellent book – and the author makes a proposal for more cluelessness when using APIs. “Don’t make me think” as Krug said or the “Principle of Least Surprise”.

I haven’t written lots of Scala but I’m already annoyed because of the sub par integration with Java (collections e.g.)

And I’m not convinced: I still think it’s not about the remove method. The sub par integration in Java is due to the – now dropped idea – of Scala as a language for both the JVM and .NET. Therefor Java functionality is duplicated in Scala whereas Clojure is integrated.

Stephan,

You are free to have your preference of course, but given Scala’s focus on static typing _and_ immutability, I am not surprised that they would rather not have a mutator that throws a runtime exception in most cases at that level. It penalises all Scala users due to Java’s weak type system and questionable collections library design.

“Success” can be defined in many ways. I believe integration with Java is very useful, but every design decision is a matter of trade-offs. If “tight integration” was so important, then Scala would never use symbolic names for example (try to use them from Java).

“Principle of Least Surprise” and having to think about things are totally different things. Please don’t stop thinking while you’re programming. Not if you want to write good quality code (good abstractions require thinking). Having said that, an API should be as easy to use as possible (and not more).

You are only looking at it from the point of view of someone who is writing small amounts of Scala code that interact mostly with Java code. Try to think of the other case too, someone who is writing mostly Scala code and interacts with Java code in well-defined module boundaries. For the latter case, the current situation works pretty well.

Supporting JVM and .NET is certainly a very important reason (I never said that Iterator.remove was the only reason), but I have no idea where you got the information that the idea was dropped. Last time the question came up, Martin Odersky said that the .NET port is being worked on.

“For the latter case, the current situation works pretty well.”

Nope it doesn’t. Whenever you interact with JDBC, JPA, JAX-RS, JMS and dozens of other libraries you need to think just to pass and get parameters.

I know a lot of FP developers are in the “write everything myself” camp, I’m more in the “write as few code as possible” camp – and reuse existing code. If you’re writing everything in Scala and you have enough money to pay for such a task instead of assembling your application from existing libraries, then I guess the Scala approach is fine.

“(good abstractions require thinking).”

I especially don’t like the “abstractions” hand waving in functional circles. Most applications just pass data. Perhaps I find some examples of “good abstractions” in the future.

Yes after some looking I found a recent .NET page on the Scala website. Which doesn’t make me more confident in the future of Scala on the JVM. With .NET the outlook for proper Java integration looks dim. Which isn’t a problem for you I guess, but for people who want to stay in JVM land it’s a disadvantage.

We’ll see. For now I keep using Scala, with more impact at companies perhaps the Scala focus will change from academia to industry.

Stephan,

Please stop assuming things and stating your opinions as fact.

Here are some facts for you:

- I do not have a FP background. In fact, Scala is the first language that is somewhat functional that I have written a substantial amount of code in.

- We use Scala at work with JPA, Spring, Wicket, Compass, etc. So, this not an academic exercise and we also believe in re-using as much (good) code as possible.

- I have written 0 lines of code for .NET since I left University.

- Having conversions done automatically is a one-liner (import scala.collection.jcl.Conversions._).

- Our codebase is still mostly Java with a few Scala modules, so I am talking from experience on a medium-sized project.

It’s fine to disagree, but please limit your assertions to opinion instead of fact and don’t try to discredit the opinion from others based on false information.

Thanks,
Ismael

I’m very unimpressed with the principle of least surprise. If there’s a reason you should stop using the old stuff, there’s a reason the new stuff should NOT do the same thing in the same way. Or else it will fail in the same way.

Concurrency and immutable data are the perfect example. Mutable data is much simpler – in the degenerate single-threaded case. In the presence of multiple threads it’s LESS simple. If you are not forced to work through your immutable-data surprise at the outset, your pleasant sense of familiarity will drag you down into the familiar threading swamp.

Ismael,

Please stop assuming things and stating your opinions as fact.

It’s fine to disagree, but please limit your assertions to opinion instead of fact and don’t try to discredit the opinion from others based on false information.

Thanks,
Stephan

PS: I really don’t want to sound rude, but I won’t discuss things with people who resort to personal attacks. Thanks for understanding this.

@Julian: The principle of least surprise is not about doing things the same way. It’s about being not surprised when using something because it’s consistent. Having Scala Iterators, Lists and Arrays and Java Iterators, Lists and Arrays in one application in several APIs is surprising.

About threading an mutable data: Perhaps it’s surprising to you, but most threaded web applications written in OO languages with mutable state don’t share any data. They are request scoped and attach each request to a thread. So for web applications the idea of immutable data is – today – not relevant. The web way of writing applications have made sharing-state-multi-threaded-applications irrelevant (beside same small parts like caches, connections pools etc.)

Things change when data is exchanged like in twitter scenarios. But most often messages buses are used who use copied data for transfer.

New architectures with sveral agents on each layer could scale better – not proven – on multiple cores but would need to split each requests into tasks fo multiple agents handling them (as seen with multi layered JMS queues in some architectures) compared to having each core a thread group (see Join/Fork by Doug Lea).

@Ismael: To not drop the discussion – because I’m really interested in others opinions – consider 20 developers working on an application.

Wouldn’t it be easier for collections to just work, instead of needing to convert them (with a one liner as you’ve said) or thinking about two types of collections (Scala and Java) in your head? (I’m very stupid you see so I need to think all the time “Is this a Scala list or a Java list or a Scala Iterator?”)

@Rich: Nice presentation btw, I’ve just taken a look.

http://blip.tv/file/812787

After some more thought, perhaps Clojure is more relevant to Lisp communities while Scala is to Java shops. So you’re right about that.

First of all, I don’t see why you claim I was rude. Your previous comment made a few comments about “FP developers”, “write everything myself” and “academia”. I just asked you to stop assuming that people who were talking about Scala were in this camp. Then I stated some facts about my involvement with Scala to help you understand where _I_ was coming from. And yes, those _are_ facts.

However, I never claimed that my _opinion_ that the current design is a decent trade-off is a fact. Neither do I claim that your opinion has no merit. As I said from the start, it’s a matter of trade-offs, and I just happen to prefer the current approach. The way I see it, Scala and Java mix well if you work at module boundaries. You avoid many issues that way. So, when I am working on Scala code, I don’t want to suffer mistakes that were done in Java several years ago (the collections API was introduced in 1.2, for example). In addition, as time goes on, more Scala modules and libraries can be written and perhaps some will have even less contact with Java code.

I should also add, that eventually many wrappers will be written that make it nicer to work with Java libraries. This does not mean that all Java code has to be rewritten. The latest version of Scala includes scala-swing as an experimental library, for example.

Hope that clarifies things,
Ismael

I used scheme at the university, I couldn’t stand the syntax , I know the strongest point to lisp is the syntax but for me I could not get it, So I went with C, Java, and scripting languages and I think thousands of people are in the same camp as I am, Not in the lisp camp, So I think Scala have a better future but Scala or Java could learn few things from Clojure.

I like groovy but I was amazing with a sample code stephan showed in his blog and it was very similar to groovy but statically typed and have performance so I think better Scala is the way to go, it is just my Opinion.

Regards.

@Stephan: the “servlets” model of unshared state certainly works… until you need to share state. At which point you either push your concurrency down onto a database (which doesn’t really solve the problem, just moves the bottleneck), or you have to use any one of several mechanisms to run shared state in the container (which is just like sharing ordinary data structures between threads in an app with a main() method, only more clumsy because of the added indirection).

I basically consider the “container” model itself to be a failed idea.

The company I work for made 100Mio $ a year in the past on this failed model.

What information do you want to share between sessions?

Comment to

http://ndanger.org/blog/2008/10/08/adopting-functional-programming-or-just-why-did-people-start-using-python-anyway/

“@Tuna-Fish: I had a small post about Scala vs. Clojure on my blog recently:

http://stephan.reposita.org/archives/2008/10/02/scala-vs-clojure/

I think concurrency is not a real problem to solve in nowadays applications. The request-per-thread model (with worker threads working the queues) works fine for most people, and scales very well – even for synchronous IO, see e.g. Talkinator

http://mailinator.blogspot.com/2008/08/benchmarking-talkinator.html

“In my benchmarks now, on my quad-core desktop the talkinator server can push about 39000 messages per second. Keep in mind this is processed messages as in decoded, packaged, and queued for particular recipients.” (From the mailinator guy who runs the whole mailinator application on one server)

This might change, but I doubt it. The “FP is better for multi-cores” doesn’t hold with some scrutinizing.

Peace
-stephan”

From the Talkinator post:

“…you cant do Thread-per-connection and for Java you soft-max at around 12000 threads (in my testing) and hard-max at 16383 (JVM just wont make more…)”

“In effect, Talkinator is a synchronous IO system which can support tens of thousands of open connections, running on just a few hundred threads (on a LAN it only needs a few tens of threads).”

“Messages are queued for each user.”

This isn’t a request-per-thread model. In fact, the author goes to great lengths to describe *why* he didn’t take such an approach. If you really look closely, you’ll see that the system he is describing is actually a crude, home-grown version of an Actor concurrency system. I guess higher-order concurrency abstractions really are useful after all…

In theory, one-thread-per-connection solves a lot of problems in a *server* application. However, threads in Java are extremely expensive, even using Linux and NPTL (as the Talkinator article explains). If Java had green threading (think: Erlang), then maybe this would be a practicable approach. Unfortunately, even with green threading, this technique is still completely inapplicable to desktop applications. The reason concurrency is such an issue these days is desktop apps are now being expected to take advantage of multiple cores. The thread-per-connection model is inapplicable, and just managing threads by hand gets very tricky very fast. Hence: actors, fork/join, STM and a horde of fanatical pundits banging out the rhythms of functional programming.

Daniel, yes you’re right. I did understand the model the first time I’ve read the post but made a mistake by quoting the the post. It’s more of an one-thread-per-active-connection system (not connection in general).

I still have the opinion though that the for web based applications who do not need to share state (most of them) a request-per-thread model works fine (If the application needs to share state, it’s another thing – see Twitter ;-)

“Unfortunately, even with green threading, this technique is still completely inapplicable to desktop applications.”

Yes you’re right on this one too. But as I haven’t (with some RCP exceptions) developed desktop applications for the last 15 years, I neither have a interest or a knowledge of desktop applications.

Though I take offense that every time someone implements a queue model he is developing “actually a crude, home-grown version of an Actor concurrency system.”

Or when everytime someone uses stateless, side effect free code he is “copying” FP.

Actors use queues.

FP uses (mostly) side effect free and stateless code.

Others use queues.

Others use side effect free and stateless code.

Those are principles that are used by FP, they are not FP. Those principles are used by actors, they are not actors.

(Oh and I’m for actor based code, I currently use them for comet based back messaging to the browser, see the slides in the latest post, and I do like side effect free and stateless code – which I have used for years in Java as a principle and now like in Scala (@Pure would be nice, I always found IO in Haskell fascinating))

But that aside, I think there are too many FP zealots around. I don’t like fanatical people any more – not that you are one ;-) And lots of them (who come from Rails and other languages) will jump ship when the next hype thing comes around.

Ah, I see what you’re saying now! :-)

FP certainly isn’t the be-all, end-all of programming methodology. I agree whole-heartedly that there are too many people zealously pushing it as the only way to go. Immutability does tend to make concurrency a bit easier to work with, not to mention lending itself nicely to thicker abstractions in that arena, but as you say, it doesn’t solve every problem.

I think that good developers will naturally gravitate toward immutable, side-effect free code just from an intuitive design standpoint. After all, it does make code a lot easier to reason about in most situations. Such developers are also usually aware of the problems associated with asynchronous execution, and so design their systems accordingly. The really problematic folks are the ones who (as you say) jump from bandwagon to bandwagon, never stopping to learn a paradigm long enough to turn it into a craft.

Concurrent program design requires thought and experience, two traits that are in precious short supply.

“I think that good developers will naturally gravitate toward immutable, side-effect free code just from an intuitive design standpoint.”

Yes, (not only because that makes me a good developer, hehe).

One developer (a very good one) I know started to make all declarations in his Java code final, yes they gravitate to immutability.

“Concurrent program design requires thought and experience, two traits that are in precious short supply.”

Yes. And deadlocks don’t help (in code you swore that no deadlock would be in there).

I hope I can write some “Real world Scala and FP” like Jonas does (but I guess I could never write such good posts as Jonas did – or you do, even if I don’t agree with them). To give substantial arguments instead of empty phrases – though I think Scala has more substance in blog posts than e.g. Erlang (perhaps because of the wagon jumpers) and is relativly fanatics free.

Ignore my comments please, I had no idea about Clojure, I thought it was just another CL wannabe but I gave a shot to Clojure, It is awesome.

It is very easy to use,Compiles to bytecode, the syntax is robust, it is a dream come true. It follow the Lisp culture but the great things about it. I liked a lot and from now on I will continue to use Clojure.

I did many example with Swing and is great how short of code and exciting is to program Swing with Clojure, I think it is much better with Clojure than even with groovy builders.

I think when someone really try Clojure for real can say that it have a bright future. it just need of course more docs. By the way is coming one new book in the pragmatics bookshelf for Clojure, I will get it.

Congrats to Clojure team, it is a beauty this programming language for the JVM.

@Chekke: Great to hear :-)

@Chekke: Clojure is indeed a beautiful work of software engineering and definitely will breath a new life into JVM. I am very impressed so far. It also has a secret weapon in that its creator is also a very likable fellow, and quite good at communicating esoteric knowledge to the vocational comp sci set.

It’s “gung ho” not “gun ho”. http://en.wikipedia.org/wiki/Gung-ho

Like several people who commented on this post, I started out thinking Scala would be the Java.Next language. Since then, having spent some time on Jython, JRuby, Scala and then Clojure I have very strong feeling it is rather Clojure which is the top contender.

Every new language has to bring about a disruptive paradigm change to succeed. What Java did to C/C++, Ruby did to Java/EE etc. The kind of intellectual leap a software engineer would expect at this time is found in Clojure, which gets lots of thing right. Scala is too similar to Java and while solving some of Java’s problems (verbosity) only compounds some on the other hand.

Dear Julian Morrison, Stephen, Ismael,

I think you all and me live in the same world yet I don’t understand more than 20% of what you all are talking about. But I want to say my opinion to you….

Use either python/Ruby. After seeing your article, I looked at Clojure and found it looks like garbage. Scala was ok. I also think that if either Scala/Clojure why not Haskell. (of course you are talking about JVM…but JVM is not the end of world!! Parrot will be much better than JVM in future)

These languages also are almost platform neutral. Ruby after 2 years, will be the tuffest chalenger for Java.

Thanks

Another 2c worth from a longtime Java developer getting started with Scala and Clojure:

I was excited about Scala; I bought several books and wrote some smallish programs. The boilerplate-saving constructs are great! Java really needs properties and closures (or anonymous methods, whatever).

Integration with Gigabytes of Java libraries is another of Scala’s big strengths. But Scala’s complex and pedantic type handling make interfacing a bit rough around the edges. I was able to write a lot of “Java without semicolons” but when I went for some of the more advanced functional constructs, I ended up writing what looks like Perl or line noise. Beyond a certain level of terseness, I feel readability
suffers again. Also, various “features” that make Scala suitable as a “DSL construction kit,” such as the (sometimes) optional parentheses, braces, periods, etc – those features make the language idiosyncratic and lead to a Perl-ish inconsistency of coding style. Whenever there was a problem involving types, I found myself staring at masses of crpytic brackets and parentheses. Wait, which one was the lispy language again? I completely failed to understand Lift, a candidate “Killer app” for Scala. Finally, I didn’t see any big advantage in Scala’s
wrapping of much of the Swing API.

Then I tried Clojure. It’s been 20 years since I wrote any Lisp code, and the parentheses took some getting used to. Also, FP wasn’t a self starter, and I often hung up my REPL trying to print out infinite lazy sequences. But I was able fairly quickly to write working code, and it was clean and terse code, and it often worked almost right off the bat. Some of my longer-running code turned out to be accidentally”
multiprocessing: It kept 2 or more cores busy although I hadn’t explicitly used any concurrency features. Java integration is smoother and more complete than in Scala. It also turns out I don’t miss objects at all.

I’ve gotten accustomed to parentheses and they don’t bother me any more. It’s nice not having to think about data types and still getting fast code. FP is fun once you get some practice, and Swing code in Clojure is nice and terse. Lastly, though neither is perfect, I found Clojure’s API documentation a bit easier to read. Scala might do well to copy the “Cheat Sheet” concept!

Programming in Scala feels like work; programming in Clojure is fun. That’s got to count for something!

Thanks Carl Smotricz for the insight about Perl-ish Scala. I’m more confident to spend more time on Clojure than on Scala now.

Thanks Carl Smotricz for the insight about Perl-ish Scala. I’m more confident to spend more time on Clojure than on Scala now.

However, can you explain why “Java integration is smoother and more complete than in Scala”?

Everything which tries to implement the Java Collections could be considered abhorrently broken, because the Java Collections are – in fact – abhorrently broken.

Some of the claims from the Clojure sound like that Subversion slogan a few years ago: “CVS done right”. Although the world already knew that there is no single, meaningful way to do CVS right.

I guess I belong to the camp which considers _not_ implementing JCA a plus :-)