It seems you got the syntax wrong.
The syntax for if-statements in expression dialogs is as follows:
if(condition to be evaluated, then use this expression, else use that expression)
sum(if(date=2012, CostField2, if(date=2011, CostField1, CostField)
The last CostField is the "else-statement for the second if".
If the year is neither 2012 nor 2011, then it will go into the second if-statement's else statement.