the blog for developers

7 Good Rules to Log Exceptions

I’ve been helping to debug some nasty problems and bugs lately. It occurred to me that some best practices on how to log exceptions go a long way towards easier debugging. Some of the best practices I’ve learned to log exceptions are compiled in this post.

1. Only log technical exceptions not user exceptions
User exceptions are either ok and need not to be logged (“login name already exists”) but shown to the user, or no exception at all (“user has no credit left”). Technical exceptions are those you need to debug (“no file storage left”, “could not book product”) and react to. If you log everything you will probably get too many log entries to have a meaningful reaction to exceptions in your log. You should inquire into every exception in your log files and find the cause for it (“is it a bug?”). Too many exceptions will make you sloppy with exceptions in your log files (“nah, just another exception”).

2. Store data in your exceptions to make them easier to log
Taking the exception “could not charge money from account” you should store the context of the exception just like Junit does (“expected but got …”) to make debugging easier

CannotChargeMoneyAccountException(Money moneyInAccount, Money toCharge, Account account)

The message could be: “Tried to charge 20 EUR from account 1234567890 but 10 EUR available” compared to “Charge failed”. This makes it much easier later to log the exception in a meaningful way. Be careful to create no memory leaks though.

3. Log the description of your exception
Very bad example from Sun: The ClassCastException didn’t show you what class you did want an object to cast to for a long time.

Now it even detects

final String s = "Hello";
final int x = (Integer) s;

and tells you

T.java:4: inconvertible types
found   : java.lang.String
required: java.lang.Integer
          final Integer x = (Integer) s;

During runtime the exception thrown by Java is now:

Exception in thread "main" java.lang.ClassCastException:
java.lang.String cannot be cast to java.lang.Integer

Much better than before.

4. Output all causes to your exception
If your exception has an exception wrapped as a cause, log all causes. Some logging frameworks do this for you, some don’t. Be sure to have all causes of your exception in the log file. Be sure the beginning of all relevant stack traces in your log, not scrambled ones.

5. Log to the right level
If your Exception is critical, log it as Level.CRITICAL. You need to decide what critical means for you (most often it means losing money). For example if a booking didn’t work, or a user could not register due to technical problems then you have a CRITICAL problem you need to solve.

Monitor your log files for critical exceptions. You’re losing money.

Have your own exception implement isCritical() or a CriticalException interface and test when logging the exception in your wrapper to log it on the right level. If your logging framework hasn’t got an appropriate level, create one.

6. Don’t log and rethrow
Logging and rethrowing an exception is an exception anti-pattern.

catch (NoUserException e) {
  LOG.error("No user available", e);
  throw new UserServiceException("No user available", e);
}

Don’t do it. Your log files will then contain the same exceptions several times on several stack levels. Only log the exception once.

7. Do not log with System.out or System.err
Always use a log framework, it can handle logging better than you.

I hope those rule help you with your exception logging and enable you to easier debug your problems.

Thanks for listening. As ever, please do share your thoughts and additional tips in the comments below, or on your own blog (I have trackbacks enabled).

Update: For some Hackernews comments: Logging everything is fine but leads to ~50gig/day, 300gig/week, 1.2tb/month log files for a moderate site. Your grep won’t work very good for that. Splunk will obviously help of course but is only free for 0.5gig/day, a 1/100 of the log files you will get.

You can leave a Reply here. Of course, you should follow me on twitter here.

You can share this post!
Do you want to tell others about this article? Use the social bookmark icons to submit this artice to the service of your choice. Thanks.

About the author: Stephan Schmidt has more than 15 years of internet technology experience and 10 years experience in agile. He was head of development, consultant and CTO and is a speaker, author and blog writer. He specializes in organizing and optimizing software development helping companies by increasing productivity with lean software development and agile methodologies. Want to know more? All views are only his own.
Leave a reply.

Comments

I had seen so many instances of logging exceptions as you mentioned in #1 which makes debugging almost impossible.

#2 is a new learning for me. Thanks for the tip.

stephan

@Albin: I’ve seen #1 several times too

>> 6. Don’t log and rethrow

I did that recently and then had to go and remove the clutter :)

Alexey

I would add: Logging is a michanism of error hiding unless you check you log regularly. The best way to do this is to automate it.

stephan

@Alexey: Absolutely, you need to check your log. Lots of people don’t check their logs for errors.

Andrew

Good list – thanks.

Rule #5, there’s a br tag at the end of that hyperlink to OWASP… following it in firefox, it’s a bad title in the wiki.

stephan

@Andrew: Thanks, fixed, was a newline that wordpress changed to a BR.

[...] 7 Good Rules to Log Exceptions | Code Monkeyism 1. Only log technical exceptions not user exceptions User exceptions are either ok and need not to be logged (”login name already exists”) but shown to the user, or no exception at all (”user has no credit left”). Technical exceptions are those you need to debug (”no file storage left”, “could not book product”) and react to. If you log everything you will probably get too many log entries to have a meaningful reaction to exceptions in your log. You should inquire into every exception in your log files and find the cause for it (”is it a bug?”). Too many exceptions will make you sloppy with exceptions in your log files (”nah, just another exception”). (tags: programming exception) Possibly related posts: (automatically generated)links for 2008-07-21Daily Bookmarks 03/26/2008Tags I: Use With Caution [...]

3bit

Sometimes the Anti-Pattern “log and rethrow” makes perfect sense. I case of EJB and remote calls you might want to log the Exception on the server side, but also the client has to be notified of the Exceptional state. If the client has no logging enabled or the log is lost, you can at least fix the server component.

Yes 3bit, that’s an interesting case. But it should be more of an exception than a rule.

stephan

@3bit: Yes, then it’s the top layer. At the top layer (EJB at the remote interface or the web layer or the REST layer) you need to log and rethrow.

Great post!

Regarding #1: I agree functional and technical exceptions shouldn’t be stored in the same log file, however user exceptions are useful for helping reproducing issues. IMO they should be logged to a dedicated file (e.g. functional.log).

stephan

@Tiago: I wouldn’t. I would probably log a “User X entered wrong password” but not the exception. There is a difference. The exception has a stacktrace and several causes. User exceptions are quite often and I don’t think most organisations can handle the load of data when logging every exception.

@Stephan: You’re right, I wasn’t clear enough about that. I didn’t mean to log the stack traces in the functional log, just the user exception message as you said. Cheers.

[...] 7 Good Rules to Log Exceptions Code Monkeyism (tags: programming) [...]

[...] 7 Good Rules to Log Exceptions [...]

Armin

Hello
it’s very good
thank you sir

Bob Follek

#2 “Store Data…” is excellent advice, but make sure the context info you log isn’t an audit problem. In some businesses, e.g. U.S. financial services, writing a full customer account number to a log file might be a red flag. Consider using just the last 4 digits, etc.

stephan

@Bob: Very good comment, the same goes for passwords or (in Germany) other personal information.

french_c

stephan: Regarding 3bit and “Don’t log an rethrow”.

Log and rethrow Exceptions at system borders is an interesting case. Nevertheless it is a requirement. Otherwhise you end up with a system that is perfectly healthy according to your log (and response time) while being totally broken.

At system borders I tend to log every RuntimeException including its full stack trace (at error level) and every checked exception as part of the API interaction log without a stacktrace (at info level). The “API interaction log” is a simple log statement that tells me which method was executed and whether the execution was successful or not.

Since most remote APIs require centralized API handling code that takes care of exception transformation, stack trace removal, etc. it is easy to add my logging requirements.

stephan

@french: Insightful comment as always, thx.

french_c

@Bob/@Stephan:

Did you ever talk to labour unions regarding log statements? If not, and your software should run in Germany, you better do:)

Of course you need to be careful logging social, medicial or health related data. But you should also be careful logging user interaction data, which is strictly prohibited in a coporate context (in Germany).

There might be a chance that your labour union will support user interaction tracking that can be used to benchmark employees. But if you are an employer: Better ask upfront! ;)

stephan

@french_c: Oh, don’t tell me about unions. I had a startup which did knowledge management by putting tags (Java, Oracle, …) on people. Unions love that !

[...] http://codemonkeyism.com/7-good-rules-to-log-exceptions/ Categories: DevOps Tags: Log Management, logging Comments (0) Trackbacks (0) Leave a comment Trackback [...]

And: allow your logging framework to be re-configured at run-time. For example, if something CRITICAL goes wrong, and it’s not immediately obvious why, you can then log more detail without having to redeploy your app; particularly important for compiled languages like Java.

Leave a Reply

What people wrote somewhere else:

Additional comments powered by BackType

Guide to CodeMonkeyism

Over the last 4 years I wrote many articles on this blog. To make it easier for you to find the relevant ones, I've organized them into topics.

Top 10

6 reasons why my VC funded startup did fail

Go Ahead: Next Generation Java Programming Style

Java Interview questions: Write a String Reverser

The dark side of NoSQL

7 Bad Signs not to Work for a Software Company or Startup

Is Java dead?

Scala vs. Clojure

Never, never, never use String in Java

No future for functional programming in 2008 – Scala, F# and Nu

Clojure vs Scala, Part 2

Java Developer

Is Java Dead?

Go Ahead: Next Generation Java Programming Style

Be careful with magical code

All variables in Java must be final

Never, never, never use String in Java

Bending Java: More readable code with methods that do nothing?

NoSQL Guy

NoSQL: The Dawn of Polyglot Persistence

The dark side of NoSQL

Essential storage tradeoff: Simple Reads vs. Simple Writes

Sharding destroys the goals of your relational database

The unholy legacy of databases

Startup/CTO

Development Dream Teams

6 reasons why my VC funded startup did fail

American vs. European style of Software Development

12 Things to Reduce Your Lead Time and Time to Market

The high cost of overhead when working in parallel

Essential storage tradeoff: Simple Reads vs. Simple Writes

Job Seeker

Another Good (Java) Interview Question

7 Bad Signs not to Work for a Software Company or Startup

Java Interview questions: Write a String Reverser (and use Recursion!)

Java Interview questions: Multiple Inheritance

As a Manager: What I value in developers

Top 10 Tips (+1) to Get a Pay Raise

Agilist

What Developers Need to Know About Agile

5 Practices Better to Change in Your Scrum Implementation

Scrum is not about engineering practices

ScrumMaster and ZenMaster: The joke of certification

What is Trans-Scrum?