All posts by stephan

Stephan Schmidt has been working with internet technologies for the last 20 years. He was head of development, consultant and CTO and is a speaker, author and blog writer. He specializes in organizing and optimizing software development helping companies by increasing productivity with lean software development and agile methodologies. Want to know more? All views are only his own. You can find him on Google +

Real Life Bug Prevention in Scala

Using Scala for the last 5 years to run eventsofa I’ve became a better programmer creating less bugs. Most of the code is Java++ Scala compared to more canonical functional Haskell-like Scala. So I’m interested how Scala used in this way prevents bugs that I would make in Java, Python or Ruby.
There have been two things that made a huge impact, Options and tagged types.

Lets explore both concepts with a small example:

Option vs. Null

How does one signal that there is no invoice or there is no user? In Java this is often done with Null or throwing some kind of InvoiceNotFoundException. With Nulls for error handling (Some Thoughts on Error Handling) code looks like

Code with exceptions would look similar. This code is noisy and because developers are not forced to check for Null, sometimes developers do not test for error conditions signaled with Null. The result are hard to find NullPointerExceptions (I also blame you Java, for not printing more information in NPEs, take a look at Elm!)

In Scala one would use Option to signal that there is no user, with returning Some(User) if a user was found and None if there wasn’t.

Code using this does not need to check for the result type of the method call and split the code path. One code path is used for both the error and the success case. Error handling can be deferred to a later point and errors can be accumulated.

or

and with some for sugaring

In Scala it’s easy to compose Options compared to composing Nulls – which don’t compose. Also the code is easier to read with less noise, especially the for version is easy to read even with more complex dependencies. Code that is easy to read and easy to understand results in less misunderstandings and less bugs. With this error handling by developers is improved as they do write more and better error handling code. From my experience with Scala the usage of Option instead of Null or Exceptions leads to a lot less lazyness bugs.

Tagged Types

The usage of String for accountNumber and userId is problematic.

For developers it’s hard to understand how these Strings look like or how to get or create a correct one. There might be different userIds in different String formats and it is easy to plug the wrong userId formatted String into a method. A little bit more nuanced the problems shines in

If you find this code, your first thought is “What is percentage?”. To represent 11.5% the Float value could be 11.5 or 0.115. This has biten me in the past when I’ve assumed 10% percent for a coupon was represented by 0.10. As a result millions of users got mails with coupons with a value 0.10% because the developer of that method represented 10% with 10.0.

Where this also rears its head is with money.

First Once and for all: Do not use double for money, second is this a Netto or Brutto amount? A bug I’ve seen several times is when variables representing netto amounts are plugged into methods expecting brutto amounts.

The classic approach would be to create abstract data types (ADTs), classes, case classes or data types classes (Value Objects).

The downside with this is (Never, never, never use String in Java (or at least less often :-)) and the reason developers don’t use this often enough: developers need to write more code plus Percentage is harder to understand, reuse and calculated with than Float.

In Scala I’m using Tagged Types.

These have several benefits. It’s easy to see that Percentage is a Float, creation can be documented (10% represented by 0.10) and controlled plus the API is easier to read. All of this leads to less misunderstandings and from my experience therefor to less bugs.

Other examples are

Tagged types have made my code much better than before.

How about other Scala features to prevent bugs? As I am writing server side, request isolated code, immutable data strucutures didn’t help to prevent concurrency bugs in the last years. My only concurrency is fire and forget usage of actors for logging, database side effects, sending mails etc. Limited concurrency is used to get data from different data sources, neither of this use case share data though.

Sealed traits have helped with checking if switch statements are exhaustive, especially in larger code bases this helps add features. Case classes prevented hashcode bugs as they generate correct hashcode methods by themselves.

All in all my Scala code has a reduced level of bugs due to some Scala features compared to my code in Java, Ruby or Python. Any new language I would need to support both Option and tagged types.

Exploring Delegation in Kotlin

I’m a huge fan of interfaces in Java and also of composition over inheritance. Inheritance is some magic wiring with tight coupling that creates lots of friction when evolving a code base. I’ve written about interfaces before several times, for example here or here. Looking at Kotlin I wanted to see what I can do with interfaces and composition.

Take this example:

Why would one introduce a HasName interface in this case? It reduces dependencies and coupling. This makes reasoning about code easier and speeds up compilation, especially incremental compilation.

How would we use the HasName interface? A function that checks for a good name could look like this:

Now the function depends on the whole of person, not just the name part. The function can not operate on a lot of things, just persons. What about other things with Name, like Dogs? Rewritting the code to

makes the function more reusable.

Inside our Person class we have the code for the HasName functionality. It woud be nicer to be able to reuse the functionality from somewhere else.

In Kotlin we can delegate Interfaces to objects:

This looks a little unnatural to me, as the user of the Person class needs to know about the NameMixin. See if we can do better

looks cleaner as the consumer of Person does not know about NameMixin.

Kotlin can also use data classes (Thanks to Christian Helmbold for pointing this out).

If we want to have more control, we can use a Factory inside Person.

The name of the companion object, in this case Name, is optional but helps to structure the factory methods.

The mixin can be accessed with this:

Using a companion object with a factory method is better, because it gives us more control in the creation of the Mixin. But the control is not optimal. I wish we had something like

where I have access to name and more control over it. It would also be nice to have a way to access other Mixins from a Mixin. But overall, some nice functionality in Kotlin.

CEO to CTO: What is your RewriteRatio?

I’ve been thinking a lot about the interaction of CEOs and CTOs over the last years. On that topic I gave talks, educated CEOs and held seminars. What type of interactions do CEOs have with technology? What are KPIs that make sense? What are levers forthe CEO? What questions should a CEO ask? What should CEOs know about technology?

I still have the feeling after many years of technology success for many CEOs technology is a black box, that either works or explodes, with nothing in between. Do I get a good return of investment from IT? Should I have more developers? When talking to CEOs many feel not in control when it comes to technology.

Recently after talking to Jens Schumann from OpenKnowledge about the life cycle of frameworks, I had an idea for a lever and a possible KPI around technology. This would be the ratio of the effort of migrating the current system to a new technology vs. rewriting the system from the scratch.

Say the ratio is 50%, this means migrating this system to a new technology is 50% of the effort of writing it new. 150% means migrating the system is more expensive than writing the system from scratch. I would assume many systems are between 80% and 120% with the majority at 100%. If no guidance is given, most code is written in a way that ties it tightly into frameworks.

RewriteRatio obviously depends on the technology the system should migrate to. e.g. migrating from Scala Lift to Scala Play is easier than migrating from Ruby with Rails to Javascript with Node. If you switch the programming language it’s probably always the same as writing from scratch, as essentially this is what you do.

Why should a CEO ask this question? Systems come to the end of their life cycle, or parts of these systems do. Web frameworks are no longer supported, in Javascript we have seen Backbone, Knockout, Angular, Ember and React in rapid succession, with older frameworks massively losing momentum and community. So this always is a hidden risk in your books potentially in the millions. Rewrites also have a huge impact on the delivery of new features. Often rewrites block the delivery of new features for months or even years. During some company phases rewrites are no problem and even planned for, e.g. during a first to market phase financed with external money. During some other phases rewrite costs can be painful, e.g. when there is no longer hyper growth and profit margins are thinner. The less they are, the better.

If not asked, CEOs often assume there is no need for a complete rewrite but only upgrades or migrations. Technology people often think everyone knows that we need to rewrite sooner or later. Or they just assume they’ll leave the company before that date. So often code is written in a way where migration is not easily possible.

The RewriteRatio can also be used as a communication device and alignment between CEO and CTO. e.g. the goal could be to maintain a 50% rewrite ratio. The CTO then manages according to this corridor. Developers know how much effort to put into abstracting technologies or packaging business logic away with anti corruption layers or if fast and furious is ok with the CEO (100% RewriteRatio).

RewriteRatio is just one lever or KPI which helps CEOs manage IT. I will explore more in upcoming posts.