December 6, 2010

Exception data evolved

As detailed in my earlier post about exception signatures I and my colleagues take exceptions from our production servers seriously. In addition to grouping them by signatures (which helps a lot and makes triage much more pleasant) and logging them in our internal bug tracking software we also try to add relevant debug information whenever we throw an exception.

The BCL team in their infinite wisdom added the Exception.Data property. This property is simply an IDictionary which allows storing key and value pairs of any type. By default this is an empty collection which means you don’t have to worry about it being null.

Typical usage

public void LogonUser(string username, string password, string domain)
{
    try
    {
        DataStore.LogOnUser(username, password, domain);
    }
    catch (Exception exc)
    {
        var ae = new ApplicationException("Underlying logon failed, see InnerException", exc);
                
        ae.Data["username"] = username;
        ae.Data["domain"] = domain;

        throw ae;
    }
}

When the exception reaches our internal trac site it might look something like the image below (in our case the data dictionary will actually be added as a comment but you get the point).

example-data

This has proven to be is incredibly useful for debugging purposes but it is a bit tedious to actually write the code. Storing the exception in a variable just to get access to the data property just didn’t feel right to me and writing the same boilerplate .Data[“xyz”] = xyz was just boring.

Step 1 – The AddData extension method.

I started off by creating an extension method for System.Exception called AddData. AddData looks a bit like this (very simplified, see the end of the post for the real deal)

public static Exception AddData(this Exception exception, string key, object value)
{
    exception.Data.Add(key, value);
// whohoo, chaining! return exception; }

This allowed me to save a few keystrokes over the first method and I would end up with something like this instead

catch(...) {
    throw new ApplicationException("Underlying logon failed, see InnerException", exc)
        .AddData("username", username)
        .AddData("domain", domain);
}

It might not look like much but it sure helps when you want to add debug data to an already existing exception, it saves you from having to store the reference and so on but it still doesn’t solve the problem with having to write the argument names twice. This isn’t only a nuisance when writing the code; it’s very easy for the key and value to get out of sync if you’re doing refactoring since refactoring tools will only change the variable name, not the key.

My ideal solution would be something like this

catch(...) {
    throw new ApplicationException("Underlying logon failed, see InnerException", exc)
        .AddData(username)
        .AddData(domain);
}

And have the AddData method automatically infer the proper key name but since that’s not possible I had to find another way.

Step 2 – Taking a cue from ASP.NET MVC

We continued using the AddData extension method above for a couple of weeks before it dawned on me that I could use anonymous types to skip the parameter name duplication. Anonymous types have the great ability of being able to “infer” property names when initialized.

var x = new { username = username, password = password }
// Is the same as
var x = new { username, password }

With this in mind I wrote an AddData extension method which accepts a single object and then uses reflection to iterate over all properties and adds their property names and values into the exception data dictionary.  This allowed me to rewrite my code yet again.

catch(...) {
    throw new ApplicationException("Underlying data store logon failed, see InnerException", exc)
        .AddData(new { username, password });
}

Neat, isn’t it? This is the exact same technique that ASP.NET MVC uses for route-declarations, attributes in html helpers and more.

Reflection? Isn’t that horribly slow

Not like in the olden days. It comes with a cost but you shouldn’t be too worried since you’re probably not throwing exception often enough for it to matter anyway (and if you are, then you have bigger problems).

Teh codez

using System;
using System.ComponentModel;
using System.Diagnostics;

namespace freakcode.Extensions
{
    /// 
    /// Extension methods related to instances of System.Exception and inherited objects.
    /// 
    public static class ExceptionExtensions
    {
        /// 
        /// Adds the supplied debug data to the exceptions data dictionary and returns
        /// the exception allowing chaining.
        /// 
        /// The exception type, you should not need to specify this explicitly
        /// The exception.
        /// The key of the debug value to be inserted into the exceptions data dictionary.
        /// The value to be inserted into the exceptions data dictionary.
        /// key is null
        /// An element with the same key already exists in the Data dictionary
        public static T AddData<T>(this T exception, string key, object value) where T : Exception
        {
            if (exception == null)
                throw new ArgumentNullException("exception");

            if (key == null)
                throw new ArgumentNullException("key");

            /* Key or value is not serializable (or key is null). The default internal structure which
             * implements the IDictionary is going to throw an exception in Add() so instead of 
             * throwing another exception while preparing to throw the first one we silently ignore the
             * error. Unless we're building in debug mode that is, then we'll fail. */
            if (value != null && !value.GetType().IsSerializable)
            {
                Debug.Fail("Attempt to add non-serializable value to exception data");
            }
            else
            {
                exception.Data.Add(key, value);
            }

            return exception;
        }

        /// 
        /// Adds the each property name and value from the supplied object to the exceptions data dictionary and returns
        /// the exception allowing chaining.
        /// 
        /// The exception type, you should not need to specify this explicitly
        /// The exception.
        /// An object from where properties will be read and added to the exception debug data collection.
        /// key is null
        /// An element with the same key already exists in the Data dictionary
        public static T AddData<T>(this T exception, object values) where T : Exception
        {
            if (values == null)
            {
                // Some really nasty things can happen if you start throwing exceptions in the middle
                // of throwing exceptions so unless we're in debug more we'll just silently ignore it.
                Debug.Fail("Argument 'values' was null!");
            }
            else
            {
                foreach (PropertyDescriptor descriptor in TypeDescriptor.GetProperties(values))
                    exception.AddData(descriptor.Name, descriptor.GetValue(values));
            }
            
            return exception;
        }
    }
}
kick it on DotNetKicks.com

Licensing information

December 5, 2010

Calculating week numbers in Google Spreadsheets Revisited

In my previous entry Calculating week numbers in Google Spreadsheets I explored and presented a way of retrieving the week number of a given date. A couple of weeks ago a guy named Frederik Claes send me the best bug report from a non-programmer that I've ever received. He pointed out an error in an edge-case date that wasn't included in the test suite. After having looked at the problem and read up on the problem I found two errors.

First: the algorithm

The algorithm I used didn't comply with the specifications and for some dates it would return an invalid week number. While it had the advantage of being easy to read and verbose it was obvious that it would have to be replaced. An improved algorithm is currently being discussed on the talk-page of the wikipedia article and after a couple of verifications I ended up reimplementing my script using it instead.

Second: Time zone conversion

This is a classic example of a Works On My Machine problem. It turns out (and for good reasons to) that Google Apps Scripts run in a (potentially) different timezone from the actual spreadsheet. This is a major problem when dealing with dates since what looks like 2009-01-01 in the spreadsheet could very well have been converted to 2008-12-31 when it reaches the script. While I haven't had the time to fully investigate the possible workarounds I think I've found an acceptable way of transforming the date within the script to it's proper timezone.

The script

I've sent the script to Google for review so hopefully it'll be available through the Apps Script gallery within a couple of days. Until then you can copy the script from this blog and insert it to your sheets.

// by Markus Olsson (http://www.freakcode.com). 
// See http://www.freakcode.com/projects/google-spreadsheet-week-numbers
// The script is licensed under the MIT License. Basically you're free to do whatever you want with it. Attribution not necessary but appreciated.

// Update 2011-05-02:
//   Now returns current week when no arguments are given.
//   Adding attempt to automatically convert non-date objects, see code
//   Using 2 space indentation since that seems to be the proper indentation level for apps scripts
//
// Update 2010-12-05: 
//   Using the algorithm suggested by DR J R Stockton on http://en.wikipedia.org/wiki/Talk:ISO_week_date#Algorithms and
//   some of the JS implementation provided by DR Stockton on http://www.merlyn.demon.co.uk/weekcalc.htm#JS.
//
//   This fixes the erroneous week numbers reported by Frederik Claes.
//
//   Also fixed proper conversion between spreadsheet and script time zones since scripts now have explicit 
//   time zones (they didn't when I first started this script).
//
// This method is provided as an easy to use extension to the function namespace of Google Spreadsheet. 
// Since it was written a couple of people have reported that it is possible to get the week number from a
// date cell without resorting to scripts formula:
//    =VALUE(MID(TEXT( A1; "yyw") ; 3 ;5)) 
// Use whichever method you prefer, I think =WeekNumber(A1) is more readable but that could be because of the
// work I've put in into this script ;)
//
function WeekNumber(date, inIsoStringFormat)
{
  // Allows this method to be called without arguments to return the
  // current week. Shorter version of =WEEKNUMBER(TODAY())
  if(arguments.length < 1)
    date = new Date();

  // For a short while when I first started writing this script date cells 
  // wasn't given as pure js date objects so you had to convert them first. This 
  // hasn't been an issue for me for ages but some comments on my blog suggests 
  // that explicitly convertering into a Date instance has helped them. Maybe they're
  // using old spreadsheets, maybe not. But I can't foresee that adding this extra 
  // safeguard should cause any major problems (famous last words).
  if(Object.prototype.toString.call(date) !== '[object Date]')
    date = new Date(date);

  var activeSpreadsheet = SpreadsheetApp.getActiveSpreadsheet();
  var spreadsheetTimeZone = activeSpreadsheet.getSpreadsheetTimeZone();  

  // Google apps will automatically convert the spreadsheet date into the timezone
  // under which the script is running, this will revert it to the spreadsheet date and
  // at the same time truncate hours, minutes and seconds.
  date = new Date( Utilities.formatDate(date, spreadsheetTimeZone, "MMM d, yyyy") );
    
  // Get the week day where 1 = Monday and 7 = Sunday
  var dayOfWeek = ((date.getDay() + 6) % 7) + 1;

  // Locate the nearest thursday
  date.setDate(date.getDate() + (4 - dayOfWeek));
  
  var jan1 = new Date(date.getFullYear(), 0, 1);

  // Calculate the number of days in between the nearest thursday and januari first of 
  // the same year as the nearest thursday.
  var deltaDays = Math.floor( (date.getTime() - jan1.getTime()) / (86400 * 1000) )

  var weekNumber = 1 + Math.floor(deltaDays / 7);
  
  if(inIsoStringFormat)
    return jan1.getFullYear() + "-W" + (weekNumber < 10 ? "0" + weekNumber : weekNumber);
        
  return weekNumber;
}