7 Asymmetric loss functions
In Chapter 3, Richard McElreath mentions context-specific loss functions that may be highly asymmetric, e.g. whether to order evacuation given the hurricane speed (page 60 in the 2nd edition). My experience is that students don’t really understand the example, so let me try to give a more applied example with a situation virtually everyone is familiar with.
To me, in most of the cases the question of whether the loss function should be symmetric or asymmetric boils down to whether we are fitting the model to infer causal structure or to make real-life predictions. In the former case, we want the model to be as accurate as possible in matching the data, so over-predicting and under-predicting are equally bad. Hence, our typical symmetric loss functions that I have described earlier. However, when you are using a model to make real-life predictions the loss functions is typically asymmetric, because predicting too many or too little has very different real-life consequences.
Imagine that you are hosting a party or a conference reception and you want every guest to have a piece of cake. How many pieces do you need to order? That depends on how many guests will be present. Unfortunately, you cannot know for sure, as some may not come for various reasons. From your prior experience you may know an approximate number of guests, so the question is how many pieces of cakes do you need to order and what are the consequences of making a mistake? Let’s say you are off by five pieces. If you ordered too many, you wasted money, as five pieces of cake is left uneaten. If you order too few, you have five hungry and unhappy guests. Neither outcome is ideal, but which one is more preferable? Most people order extra pieces of cake just in case, because the asymmetry of the loss is very obvious. Or, an even more extreme example, would you be rather five minutes early or five minutes late for your train?
In short, if you are using your model for real-life predictions, you loss function is likely to be asymmetric.