What Developers Need to Know About Agile

Agile is mostly driven driven by managment and consultants, seldom bottom up by developers. Some developes are suprised about agile and don't know how to react. Lots of literature is written about agile, but very little with the developer in mind. What do you need to know about agile as a developer?

I'll primarily talk about Scrum as the most successfull (in term of adoptions) agile method, but most applies to other agile methods.

What developers need to know about agile

  1. Much less crunch time due to better estimation, more detailed and better planning. Agile is often pull instead of push, so developers pull their work from a product owner, contrary to work pushed onto them by managment (see below). Because features are no longer pushed and there is no abitrary deadline, there is a lot less crunch time. Crunch time happens if agile teams underestimate the work or overestimate their ability. Over time estimation gets better and crunch time at the end of a sprint vanishes.
  2. More working on features. Developers in agile are not working on whatever someone wants, whatever comes to managments mind in the morning after getting up. Agile has better planning and the backlog in Scrum leads to less chaos and more predictability. But because stories and iterations are short, product management is more agile as before.
  3. Less meetings. All meetings in Scrum are pre-defined meetings and include daily standups, estimations and planning. There are much less ad-hoc meetings, meetings are time boxed and are shorter, all meetings have a clearly defined purpose with defined work results. So I consider meetings in Scrum not "meetings" but real group work. There are especially less panic status meetings because there is no crunch time.
  4. Pull not push. Developers pull features (stories) from a product owner during a planning session. They pull as many as they can do. Requirements are not pushed from customers or product management. Pull as many as you can do, strife for increasing your work.
  5. Be a critic. Agile expects you to be a critic with requirements. Your opinion is needed, there is no push in requirements. Martin Fowler writes in Conversational Stories:

    In terms of coming up with stories, what this means is that they are always something to be refined through conversation - and that developers should play an active role in helping that definition.

    Good stories follow the INVEST principle: I - Independent, N - Negotiable, V - Valuable, E - Estimable, S - Small, T - Testable. Negotiable means your opinion as a developer is needed.

  6. Agile makes you happy. There are problems in agile, and things developers need to adapt to and usually do not like at first. But in all teams I've worked with, satisfaction went up after the introduction of Scrum and most developers - though not all - want to stay with agile after it's introduction.
  7. Responsibility as a team. The team wins or fails, not individual developers. The team has committed itself to stories, all developers have promised to finish those stories in the next sprint. Don't take this lightly, it's a major shift in responsibilites. Help each other, it's no longer "We failed because A didn't do B." If the team failed, you failed.
  8. Agile is about working, high quality code. The result of agile is working, high quality code. Quality is king, pull only as many stories as you can do with high quality. Do not sacrifice quality. The pull principle (see above) gives you enough leverage to not compromise on quality. Live for software craftmanship.

Agile is a different way for thinking. Embrace agile as a developer, it will change your life and make things easier (and only some things harder) and make you love your work and your profession more.

Scala Goodness: Structural Typing

Structural typing in Scala is the way to describe types by their structure, not by their name as with other typing. Structural typing reduces coupling and the need for inheritance. In Java you would mainly use interfaces instead of structural typing.

I go with the same examples as in "Scala Goodness: Compound types" so it's easier to compare both solutions.

class Dog {
   def call() = println("Comes to you")
}
class Cat {
   def call() = println("Doesn't bother")
}

def call(c:{ def call():Unit }) = c.call();

c:{ def call():Unit } describes a type which has a call method that returns nothing - like void in Java. The method does take all objects of types that satisfies this constraint. We can now use call on both classes:

scala> call(new Dog)
Comes to you

scala> call(new Cat)
Doesn't bother

You can also give your structural type a name with type aliasing:

type Callable = { def call():Unit }

def call(c:Callable) = c.call();

With the same classes from above we get:

scala> call(new Cat)
Doesn't bother

scala> call(new Dog)
Comes to you

Both ways work with more than one method:

def callAndFeed(a:{ def call():Unit; def feed():Unit }) {
  a.call();
  a.feed();
}

class Dog {
  def call() = println("Comes to you")
  def feed() = println("Eats")
}

Calling with a new dog results in:

scala> callAndFeed(new Dog)
Comes to you
Eats

Structural typing is very useful if you are not able to modify classes to make them implement a trait or interface or if you want to reduce coupling and increase reuse. How does this relate to interfaces? The benefit of interfaces instead of structural typing is how they describe the roles of a class.

class Dog extends Feedable with Callable

instead of

class Dog {
   def call() = println("Comes to you")
   def feed() = println("Eats")
}

Scala glory!

See also:

Be careful with magical code

Let's talk about programming magic. It seems hip to use the most powerful language or framework and power often goes hand in hand with magic. Many kinds of magic are available, especially in languages which are renown for their magic, like Ruby. But there is a lot of magic even in Java:

  • Class loading
  • Garbage collection
  • Terracotta (which is still a wonderful solution to the coherence problem)
  • Remote procedure (RMI) or webservice calls
  • AOP
  • ORMs and in particular Hibernate

I've been a long fan of ORMs - especially Hibernate - but I'm no longer, so I wrote "Orms are a thing of the past". Particulary because of their magic. The same goes with AOP. I've biten by Spring AOP, but it could be any AOP, with the more powerful like AspectJ being more dangerous. Some methods changed, Eclipse wasn't clever enough, pointcuts didn't find the right methods anymore and the system was broken in production.

Leaky abstractions manifest themselves often because of magic. Early Java examples of magic which can go wrong are garbage collection (GC) and RMI. Garbage collection works like a charm, up until it doesn't work anymore or a full GC run stops your application for several minutes. RMI, another early one, breaks in ways you do not anticipate:

calc.add(2,4)

Reading this code, one would assume very little can go wrong. But if it's a RMI call it can throw some nasty exceptions at you. As I've commented in my post "ORMs are a thing of the past":

But association managment with ORMs has the same problem as RMI: It seems to be transparent, but it is not (lazy exceptions and remote exceptions).

What is programming magic? Is it Vodoo? Arthur C.Clarkes third law says:

Any sufficiently advanced technology is indistinguishable from magic.

which makes magic something you do not understand. Any sufficiently advanced programming language or framework. As a sidenote, programming magic is bordering and related to cargo cult programming.

Why is magic even used?

Magic is good. It helps developers safe time, work with higher abstractions, write less code, read less code, have more aesthetically pleasing code and it makes you feel powerful. And it seldom does bite the one who wrote the code. Problems often arise in maintanance, with developer turnover and new developers needing to understand the code. Magic most often bites operations, and because developers and operations often do not talk, it seems like a SEP.
As I'm working more in operations in the last years, I have been biten by Java code. I have no experience with Rails ops - someone can fill me in? - but assume it is the same in Rails and PHP as it is in Java operations.

Why does magic bite you?

Because

Magic is non-linear

Repeat: Magic is non-linear. It works for a wide range, then suddenly breaks.

  • High Load under special circumstances
  • Many, many more queries than you thought of
  • Locking in unpleasent ways
  • Exceptions you didn't think of

And because you do not understand the magic it's hard to know the breaking point. Assume

customer.getAddresses() 

which differs a lot when it's proxied with an ORM, compared to a local method call. Perhaps it produces n+1 queries, it might be lazy, might take a lot of time under load or might have non-linear and unknown caching behaviour.

What you can do

You can and should use magic, but need to balance with maintanence and operations:

  1. Know where it's happening
  2. Confine it to defined points
  3. Really understand what's going on

"Confine" today is a very strong concept to me, and most of the code I write or review confines magic to clearly understood place. I've also learned to write a small anti-corruption layer around all external frameworks, but this is better explained in another post. Taking the Hibernate example - and I will get flamed for this - you might use:

dao.getAddresses(customer)

instead of

customer.getAddresses() 

With a simple implementation this looks like:

class HibernateCustomerDao extends CustomerDao {
  def getAdresses(c:Customer) = c.getAdresses
}

Why is this better? A developer knows it's non-linear when he sees dao.getAddresses(customer) compared to customer.getAddresses(). And sooner or later you will need to optimize performance or change caching. Easy to swap with JDBCTemplate for performance reasons, to add caching etc.

Other kinds of magic?

There some more kinds of magic you need to beware of. All the time there is new magic coiming up. Is Project Lombock magic? Probably not. Is type inference in Scala magic? I'm not sure yet, there is some discussion. Are implicits in Scala magic? I'm pretty sure they qualify. Beware of magic. Use it to your advantage, know it, confine it and really understand what's going on.