Last night, trying to catch a cab in a Manhattan rain, it occured to me that it really shouldn't be so surprising that so many of our systems fail exactly when we need them most. I was standing blocks from the nearest subway or bus and had no umbrella; I needed a cab to get home. But so did everyone else. Result: no cabs available for anyone. The real surprise is that we expect those systems to work at such times.
The number of available cabs is a (somewhat) efficiently determined quantity. At any moment, the supply of taxis matches the demand for private transportation. But when demand spikes, there is a scarcity of cabs. The trouble is that these demand spikes only occur infrequently - they are tail events. But more than that, they are wrong-way risk events. Not only is demand higher in terms of quantity demanded, but demand is also escalated in terms of how much people need cabs (think of this as the minimum price a person is willing to pay having gone up as well). Thus, the time I need a cab the most is the exact time it will be hardest to find.
It is in many ways a futile exercise to apply central measures to tail events. The real surprise is that we expect those systems to work.
Any system which is based on the center of the distribution will fail to give satisfactory results when tail outcomes appear, such as rainy nights or the United Nations convening. On the other hand, any system which is designed to operate smoothly 90% or 95% of the time must, by definition, be based on the center of the distribution. To deal properly with tail events, a system must either a) ignore the center or b) undergo a regime shift to deal with the tail (which is just a sneaky way of saying it should be two systems, one for the center and one for the tail). Taxis operate based on the center; therefore the system breaks down in the tail. However, tail events for cabs are generally forecastable - I would think that supply could adjust relatively easily and undergo the necessary regime change. If I drove a cab and I saw it was raining, you bet I'd be out looking for extra fares.
The BLS birth/death model which I discussed yesterday is another example. Designed to represent the manner in which businesses change over time, it must be built in accordance with normal (i.e. central) behavior. It therefore fails anytime the economy experiences sharp growth or contraction - tail events.
And how about value at risk? The analogy should be obvious by now. On the one hand, the distributional assumptions for VaR come from historically observed data. Thus, the system is based on the center. On the same hand, VaR doesn't tell us anything about the tail event - it merely defines what constitutes such an outcome. By the time you experience the tail, you're beyond VaR's ability to help you. So VaR, for all its comfort and reassurance, is another system which fails to account for the periods it will be needed most. Better risk measures deal exclusively with the tail, which is to say the outcomes they are being used to evaluate.
Anyway, I got a cab after a few minutes. It was my yellow swan.