Posts tagged as:

model

The rise of VaR

October 1, 2009 in Finance, Quotes, Risk

Simon Johnson and James Kwak take a look at how VaR got to be so popular in the first place. They make the insightful observation that a bad (or at least an incomplete) model can gain acceptance not only because of its simplicity but, oddly, because of its output as well.

Indeed, VaR succeeded not just because it seemed to capture risk accurately (“losses exceeded only 5% of the time” and so on), but because it provided the answer that financial agents were looking for. Most cynically, its greatest disadvantage – failing to look at what actually happens in crisis times, rather than just defining the crisis itself – turned into its biggest sell point when it came to market adoption. In a bizarre twist, the model was chosen because it gave the right answer; not because it answered the right question.

It reminds me of a passage from the ever-insightful Hitchhiker’s Guide to the Galaxy:

“I checked it very thoroughly,” said the computer, “and that quite definitely is the answer. I think the problem, to be quite honest with you, is that you’ve never actually known what the question is.”

“But it was the Great Question! The Ultimate Question of Life, the Universe and Everything!” howled Loonquawl.

“Yes,” said Deep Thought with the air of one who suffers fools gladly, “but what actually is it?”

But you don’t come here for HHG2G quotes (or do you?). Here’s the key excerpt from Johnson and Kwak’s analysis:

David Colander made this point about economic models: The sociology of the economics profession gave preference to elegant mathematical models that could describe the world using the smallest number of parameters. “Common sense does not advance one very far within the economics profession,” he says.

A similar point can be made about VAR models. Sure, maybe all the financial professionals who design and work with VAR know about its shortcomings, both mathematical and practical. But nevertheless, using VAR brought concrete benefits to specific actors in the banking world by helping them rationalize bad bets. If common sense would lead a risk manager to crack down on a trader taking large, risky bets, then the trader is better off if the risk manager uses VAR instead.

Not only that, but imagine the situation of the chief risk manager of a bank in, say, 2004. As Andrew Lo has argued, if he tried to reduce his bank’s exposure to structured securities such as collateralized debt obligations, he would be out of a job; VAR gave him a handy tool to rationalize a situation that defied common sense but that made his bosses only too happy. And at the top levels, chief executives and directors who probably did not understand the shortcomings of VAR were biased in its favor because it told them a story they wanted to hear.

{ 0 comments }

Search forecasts

September 8, 2009 in Internet, Math

Google Insights recently rolled out a new feature: 12 month search forecasts. The forecast comes from a relatively simple decomposition of the search volume into trend, seasonal and residual components. The model’s out-of-sample performance is tested on the most recent 12 month period; if that prediction proves accurate, then the model is accepted. Here’s what it looks like when searching for “Google” (blue), “summer” (yellow), and “weather” (red):

The “Google” volume shows a clear macro trend with little seasonal impact; “summer” is of course the opposite. “Weather” proved too unpredictable and no forecast was generated.

A complete description of the methodology is available in this paper by Google (pdf link).

Click here for a live view of these trends.

{ 0 comments }

Modelling interactions

August 18, 2009 in Math, Politics

Andrew Gelman’s latest post highlights the importance of interactions. He includes this breakdown of where people fall depending on political party, ideology, and income:

Consider the income dimension. Among liberals, the income curve is flat no matter whether the person is a Democrat, Independent or Republican. For conservatives, however, income has a large effect – in fact it becomes a strong predictor of political party. Thus, in modeling the impact of income on party, we must consider the income-ideology interaction. Without it, we would overstate the impact among liberals and understate it among conservatives.

It is not enough to merely include ideology as a separate variable in a linear model, however. That would be tantamount to presenting two distinct graphs instead of the three-way graph above – one of party vs income and another of party vs ideology. The interaction of income and ideology is explicitly ignored.

Instead, one must consider what essentially amounts to three different income variables: one for conservatives; another for moderates; and a third for liberals. These three variables would each have different coefficients, and so the model could properly capture the joint impact of income and ideology.

Be warned, though: interactions can quickly lead to overfitting, as they increase the number of variables geometrically. An exploratory analysis like the graph above or a compelling alternative hypothesis is a necessary prerequisite to using interactions in a model; if an interaction isn’t justified, you probably shouldn’t use it.

{ 0 comments }

Deconstructing the Gaussian copula, part III

August 11, 2009 in Finance, Math, Risk

(Parts I, II and II and a half of this series are also available.)

In the first two parts of this series, I respectively addressed some misperceptions about the Gaussian copula and described its common use in CDO pricing. Part III focuses more on the model components and the intuition driving them.

I am a staunch supporter of a “models are just the tool” viewpoint, an opinion more elaborately and memorably stated by George Box as, “All models are wrong, but some are useful.” With that in mind, what you will find here is not a campaign against the Gaussian copula itself; merely its blind application to certain problems in finance. I find it as difficult to blame this model alone for the 2008 recession as I find it hard to blame the sinking of the Titanic on its hull design (new research actually suggests the rivets were more at fault) – while it certainly contributed to a general sense of invincibility and well-more-than-advisable risk taking, it is naive to think that in the absence of this notorious model, 2008 would have turned out just fine.

As I recently (and strangely, given my campaigns against it) stated about VaR, the Gaussian copula does exactly what it is supposed to do – the error lies in its interpretation and its application in the first place. I join Paul Wilmott in his crusade for less equations and more common sense among quantitative financiers: getting the number is good but explaining it is better.

Gaussian Dependance

Copulas are nothing more than descriptions of how two or more random variables relate to each other. To be more specific, copulas refer to the co-behaviors of uniform random variables only; but any distribution may be transformed to the uniform case via its CDF, and that is the appeal of copula models: they describe dependance without concern of the marginal distributions. The Gaussian copula, we may conclude, doesn’t necessarily have anything to do with normal distributions as we typically think of them (i.e. in the “normal distributions are useless in finance” sense)! Rather, it describes the sort of dependance that arises when a bunch of normally-distributed variables are correlated with each other.

Gaussian dependance isn’t easy to describe like a Gaussian distribution is. For the latter case, just think of a bell curve. The former is more difficult to identify, so here’s a picture of a two uniform random variables with a Gaussian dependance structure (click to zoom):

A first observation is that the dependence is regular (meaning even) and smooth. It lacks any significant clustering. More importantly, it lacks a property called tail dependence. Tail dependance is the probability of observing extreme observations in all random variables at once. Strictly speaking, it measures the probability of observing joint tail events. As you move further out in the tail, that probability converges to 1 in the limit for structures exhibiting tail dependence. It is extremely surprising and counter-intuitive to learn that the Gaussian copula lacks tail dependence. In plain English, this means that tail events in the Gaussian copula are asymptotically independent of each other – and that is the chief problem with using Gaussian dependence in finance.

In finance, extreme events co-occur all the time, as recent memory bears witness. If risk management is the process of ascertaining, measuring, and avoiding those situations, then doesn’t it seem a little odd to use a model which is explicitly unable to account for them? Tail dependence is a necessary condition for a dependence model in finance. The Student t copula exhibits it and is only marginally more difficult to implement than a Gaussian copula; but simplicity is king and there was obviously a decision made at some point that tail events didn’t require consideration, anyway. It brings to mind my favorite VaR metaphor as an airbag that always works, except in a crash.

Correlation

Another element of the Gaussian model which does not carry well to finance is the idea that linear correlation is a sufficient statistic for the dependence distribution. Consider these two plots, each of which shows two variables that, by construction, have a correlation of 0.7 to each other. First, a Gaussian dependence structure (this looks different than the above plot because the former was the copula itself, as indicated by the uniform marginals, whereas this is a full copula-derived multivariate distribution):

Next, a dependance structure exhibiting lower tail dependance (this is from a Clayton copula and is a stylized depiction of behaviors more characteristic of finance). You can plainly see the impact of the tail dependance, in contrast to the Gaussian plot above:

The two distributions are very obviously different, and yet if you merely measured their correlation you’d describe them in exactly the same way. Correlation alone is insufficient to describe more complex dependence structures such as those observed in finance. And yet, it is the only descriptive statistic of a multivariate Gaussian distribution.

Financial covariates tend to resemble the second plot – when a large negative event occurs in one, it more than likely will occur in the other. This, by the way, accounts for some of the skewness in financial distributions – it is possible to have two perfectly normal distributions whose combination is nonetheless skewed if the dependence structure exhibits tail effects like this.

Again, we have a call for clarity: it is imperative for the underlying dynamic of any model to resemble the behaviors of the system in question.

The Single Correlation Factor

In a CDO pricing framework based on the Gaussian copula, not only is correlation the sole determinant of the dependence structure, it is assumed to be the same for every name in the basket. This has caused much alarm. Certainly, using more factors would provide a more accurate model – allowing different industries to have different correlations, for example. Unfortunately, this comes at the cost of model accuracy.

It is very important, where possible, for a model to have no more than one unobserved input for every output. Think of a Black-Scholes option: future volatility can not be known, so we plug in whatever value gets the model to spit out the current market price of the option (a “the market is always right” approach). If there were two volatilities (say, a short term value and a long term value), we would be unable to create a consistent model, for there would likely be an infinite number of volatility pairs that would satisfy the market price. For every additional parameter, we need one more output metric to match. If we could match an option’s price and also it’s delta, just for arguments sake, then there is probably a unique combination of two volatilities for that output space.

This is why using multiple correlations is problematic not just from a fitting standpoint, but from a model integrity standpoint – if you take the thousands of necessary pairwise correlations and estimate just a handful of them incorrectly, the model could deliver completely spurious results.

(For a very concrete example of this, consider pricing a mezzanine or senior CDO tranche, which requires two correlation inputs. Without knowledge of the corresponding equity tranche price – and consequently the attachment point correlation – this becomes a very difficult puzzle indeed).

However, in my mind this is one of the more minor problems. That’s not to say it isn’t an issue, but I’d much rather have a single-parameter tail-dependent model than a multi factor Gaussian one. Why? Because it’s more important to me that a model captures downside risk in some regard than that it captures the distribution’s central dynamics more faithfully.

Correlation (again)

We’ve discussed why correlation is insufficient to describe the CDO dynamics, and also why a single-factor model may lack fidelity. But in some ways, the entire discussion is slightly off base. Correlation (as I’ve alluded before) is an implied measure – it is whatever plug gets the model to output the “right” price.

There is a raging debate about how similar correlation is to Black-Scholes volatility, but I think for the purposes of this exercise we can highlight their similarities (though I will not necessarily agree with that under more rigorous terms). both are plug values; both have intuitively “correct” ranges but can not be directly measured or observed; both are the single unobserved input in the most simple pricing models of their respective derivatives.

Because of this, a lot of our reasoning on the problems with correlation goes backwards, since we begin with the premise that correlation is arbitrary and/or unmeasurable, and therefore conclude that a correlation-based model must fail. However, in practice we actually start with a tranche price, and work out the implied correlation value from that price. So I don’t really care if my correlation comes out to 60% or 70% because I’m not going to read too much into that figure – it’s just a parameter that will keep my model ticking consistently with the market, all else equal.

“But wait,” you say, “that’s the dumbest thing I’ve ever heard!” What if the market price is arbitrarily high and implies a correlation greater than 1 (or just 1, since the input is bounded)? Then that’s great, you get the price right in that instant, but the second you try to measure any sort of risk or even price it the next day, you’ll fail because a correlation of 1 doesn’t reflect reality at all. Moreover, take this to its logical conclusion: why not have a model whose sole input and output is just the price. In this scenario, you would see a tranche trading at 20, and set your “model” to 20 (the implied price). Tomorrow, your model still says 20 – so when the actual tranche trades for 19, you need to adjust your “model parameter” (i.e. price) down. Obviously, a ridiculous situation and it speaks to the critical need for any model to balance a reasonable representation (even if a simple one) of reality with an acceptable range of input parameters.

To reiterate, this is why I would prefer a simplistic one-factor tail dependent model to a multifactor Gaussian one.

Other Copula Models

All of this must raise the question, why are we stuck using the Gaussian copula?

And like so much else, the answer is: because its easy.

As mentioned, the Student t copula exhibits tail dependence and is only slightly harder to build than the Gaussian variety. So why not use it? The dark secret (unless you read part II) is that single factor Gaussian copula models are really just massive simplifications of copula-derived mathematics. The engine itself relies on arithmetic and an integral – nothing that would suggest a copula model on the surface. It is the mathematically friendly properties of the Gaussian distribution that make this possible (though frankly, it seems to me a t implementation shouldn’t be much farther off). More obscure copulas, like those in the Archimedean family, don’t necessarily follow “real world” behaviors in high dimensions, as it pertains to finance.

Moreover, like all problems of this ilk, CDOs suffer from a massive curse of dimensionality. In such situations, familiarity is key – in fact, it is sometimes the only hope of finding answers in the massive cosmos of sparse data.

Finally, Gaussian copulas have a nice property – they are easy to explain (keep in mind, lately such explanations aren’t much at all). In particular, the error rates are easy to quantify – we can be 99.975% sure of an outcome. Knowing a concrete chance of failure, even if that probability is completely bogus, makes the model easy to accept. More complicated copula structures, by contrast, are harder to work with (read: make it harder for risk managers to promise certain error rates within certain error bounds).

Finally, more complicated does not necessarily mean better. Even after all I’ve written, a pinch of common sense applied to a single factor Gaussian model might do more wonders than a more advanced model in the hands of a naive user.

Here endeth the lesson.

{ 0 comments }

Photo finish: the Netflix prize

July 28, 2009 in Math

A month ago, the million dollar Netflix prize was finally won by a coalition of leading teams called Bellkor’s Pragmatic Chaos, who blended their respective methods into a super-algorithm that finally crossed the 10% improvement barrier.

…or was it?

The 10% mark sent the competition into a final, 30-day countdown, during which time other teams could submit scores. After 30 days, the contest would be over. With just one day remaining before the contest ended, a bunch of other teams turned the tables on the leaders and formed their own blended coalition, The Ensemble. The Ensemble eeked out a 0.01% improvement over BPC’s score, landing them squarely in first place.

A scant 20 minutes before the deadline, BPC submitted a new entry which tied The Ensemble for first place – but with just 4 minutes remaining in the years-long competition, The Ensemble put up a new, marginally-higher score, sealing first place.

…or did they? (yes, two plot twists!)

It looks preliminarily like BPC may have won the competition. The contest was structured to avoid a common problem in statistics: overfitting. Overfitting occurs when a model is trained to a dataset in such a way that it is not able to describe similar data outside the original set. Overfit models are useless for forecasting.

The Netflix prize avoided overfitting by providing a two datasets, a training set and a test dataset: competitors used the training set to build and optimize their model, but scores were based on the model’s fit to the test set. Thus, an overfit model would fail. But it didn’t end there – scores were actually based on only half of the test set. These scores determined when the contest ended, but not the winning team; the winners would be the team that had the best score on the hidden half of the test set once the evaluation period ended.

So, The Ensemble managed to get the highest score on the public half of the test set but it seems that BPC may actually have the marginally higher score on the hidden half, which is the one that really matters. The implication in BPC’s blog post (which has a much better summary than I) is that The Ensemble overfit the test set – I don’t think that will end up as the most accurate assessment, however. The Ensemble will surely score highly on the hidden half; overfitting would mean they couldn’t describe it well at all.

I still find it somewhat amazing that in a contest that lasted for three years, the conclusion boils down to a few minutes and mere decimals. But I guess that’s what happens when you tell a bunch of nerds you’ll give them $1mm to do what they love to do anyway…

{ 0 comments }

Deconstructing the Gaussian copula, part II

July 9, 2009 in Finance, Math, Risk

(Parts I, II and a half and III of this series are also available.)

Recently, I addressed a great deal of misinformation regarding the Gaussian copula and it’s role in the 2008 crisis. I would like to try and follow that up with a succinct description of the copula and its use in CDO pricing. (This may seem a defense of the math behind the process, but you know I’m just setting it up for a fall.)

Introduction

David Li’s contribution to quantitative finance was the rapidly-standardized “single factor Gaussian copula” CDO pricing framework. The real crux of the problem was the “single factor” part – not the Gaussian copula itself (though we won’t pull any punches here). In an extraordinarily broad sense, a copula is a mathematical function that describes how two or more random variables interact. “Correlation” is a simple way of describing the copula, which should give the function some intuitive grounding. But let’s back up a second and figure out why we even need a copula in the first place.

Aside: Why Copulas?

If you try to model the behavior of many random variables, you need a multivariate distribution. The most mathematically friendly distributions are from the Gaussian family, including the familiar bell (or normal) curve. This is why such models are prevalent in all manners of statistics. For most purposes, the model is not only easy to work with but asymptotically correct (which is a nice feature, to put it mildly). However, there are some areas where the model choice is more for pragmatic reasons than justified ones – finance being prime among them. Indeed, financial distributions do not behave normally, but only recently have tools been developed that can describe them – and even there large joint distributions are daunting.

So, it is unsurprising that the Gaussian copula arose as a natural choice for modeling the joint distribution inherent to CDOs – which are essentially just collections of many intercorrelated credits.

But I’m getting ahead of myself. (This is much easier to discuss than to write about, I think, because you can guage your audience’s comfort which each boldfaced section before moving on. I hope, brave reader, that you are still there.) Lets talk about CDOs.

CDOs

A CDO is nothing more than a collection of various bonds, all held together in a basket. The principal risk of a CDO is default: the chance that one or more of the bonds will not survive to maturity. To isolate this risk, it is instructive to think of the CDO as a basket of sold CDS contracts, rather than a basket of purchased bonds (and indeed, “synthetic CDOs” are nothing more than CDS portfolios and have rapidly gained market share from bond portfolios). Thus, the buyer of a CDO needs to draw two conclusions regarding the basket:

Will any of the credits default?
When will all of those defaults occur?

The first point is obvious; the second gets at the heart of the problem. Both the timing and the correlation of defaults matter. If the CDO basket is comprised disproportionately of financial companies, then default by one may imply a greater likelihood of default for the others; a more diversified basket may not exhibit such dependencies.

This issue is compounded by the introduction of tranches – a staple of the CDO industry. Again, it is helpful to consider a CDO as a basket of sold CDS. The most junior (or “equity”) tranche has, by definition, sold insurance on the first few issuers to default – say, the first 3. The next tranche does not experience a loss until the 4th issuer defaults. The key here is that when a portfolio is tranched, investors have not sold CDS on specific issuers by name, but rather by time of default. They can not know ahead of time which issuers they are effectively responsible or on the hook for.

Bathtub Correlation

To understand why tranching compounds the correlation problem, think of the CDO as a rectangular bathtub interspaced with mines that represent each issuer’s default. The CDO investors are aboard a boat on one side of the bathtub, and need to cross to the other side. If the boat hits a mine, that issuer defaults, and the explosion of the mine will damage the boat. The equity tranche has an extremely thin hull and will sink quickly; the senior tranche has a thick hull and can withstand many blasts without taking damage. Finally, the boat moves across the bathtub via geometric brownian motion – which is to say, randomly.

In a low-correlation world, the mines are dispersed uniform randomly across the bathtub; hitting one mine does not imply or necessitate hitting any other. With high correlation, the mines cluster somewhere in the water; hitting one mine makes it relatively certain that another will be hit.

As a consequence, equity investors prefer high correlation. They are indifferent to hitting just a few mines or many, as they are wiped out in both situations. Therefore, they prefer the mines to be clustered, as this leaves more clear paths across the bathtub. In contrast, senior investors prefer low correlation – they can withstand glancing off a few mines, but hitting a cluster would wipe them out.

From this intuitive example, it should be clear that not only the timing of the defaults, but also their expected clustering (i.e. correlation) is important when valuing a CDO tranche.

Correlation in the Guassian Copula

Let us first draw the connection I’ve sketched out already: CDOs are composed of many issuers that may interact with each other; and a multivarite normal distribution is a common method of describing such behavior. So far, so good.

Like any Gaussian multivariate model, the Gaussian copula takes as parameters the correlation of every pair of variables under consideration. (In other words, to make the model work, you need to “explain” to it how every issuer interacts with every other issuer – these are the parameters.) Thus, the number of parameters increases with the square of the number of variables being considered – specifically, there are $\frac{N(N-1)}{2}$ parameters. If you had a CDO of 100 names, you would need to compute 4,950 parameters to describe their behavior! It doesn’t take a statistical degree to appreciate the flimsiness of a model which relies on such assumptions – it’s just too many to estimate reliably. Clearly, the traditional model simply won’t do.

Enter David Li, whose principal contribution to this field is to boil 4,950 parameters down to just one.

Shocking! Dastardly! The decision that caused the 2008 crisis! Well, not really. Though I am full of doubts about the validity of the Gaussian copula for this task in the first place, I do not think that the compression of its parameter space is the chief culprit by any means.

What Li was suggesting amounted to this: instead of modeling the intricate inter-corporate correlation structure, in which financials are highly correlated to each other but bear little semblance to utilities, which themselves are very similar, he said why not just model everything at the average correlation of the CDO names? Actually, he just said that one correlation level will be enough to describe the CDO price – he did not say it was the average (I just added that to make the notion more tolerable at first glance). He didn’t care if you chose a higher or lower correlation than any pair in the whole CDO exhibited; his claim was that there was some single number that would get the model to output a price that matched the market.

Before we get up in arms about this let’s remember that most financial instruments are priced this way. One or more variables of the equation are left free to change, such that for some level the model will output the “correct” (or market-observed) price. With options, this is called volatility; with swaps this is the fixed rate; with bonds this is the yield – I particularly like the last example because most people assume this is limited to derivatives. It’s not, “real” securities exhibit this problem too — for stocks, it’s called a P/E ratio.

So, we’ve boiled correlation down to one parameter which can take any value, but forces all issuers to have the same correlation to each other AND (this is a much more important caveat) exhibit a Gaussian dependance structure.

Now What? This Is Getting Boring.

Ok, let’s price a CDO.

If I have CDS prices for all the issuers in my CDO, I can back out the probability of each issuer defaulting. (That’s a whole other lecture, but please take my word that if we have the price of default insurance, we can calculate the probability of default. Otherwise I’ll go on for another 2000 words…) This answers my first question: will defaults occur? Combine that with a correlation number and I can answer the second question: when will all the defaults occur? So now I can price the CDO, right? Unfortunately, no.

The default probabilities backed out of the CDS data are conditional default probabilities, meaning they have the market’s 4,950 correlation factors baked into them. Company A may be doing fine, but it’s very correlated to company B which is not so healthy. The result is that company A’s CDS will exhibit a relatively high default probability even though that’s more B’s fault than A’s.

In statistics, we like to deal with independent or unconditional probabilities, because the math becomes dramatically easier. So the conditional probabilities extracted from the CDS are not so useful, and must be transformed into independent probabilities. To achieve this goal, we do something that I think is very clever:

We set up a model in which defaults are driven by a shared “market factor” and an idiosyncratic factor, similar to a regression with one dependent variable and an error term, hence the name “single factor model.” Now, I know I just said there are two factors, but one is specific to each individual issuer, so it doesn’t count as one of the model factors — if this troubles you, chalk it up to statistical nuance. Anyway, the two drivers are weighted by a correlation term; as correlation increases the market factor dominates, and as it decreases the idiosyncratic factor dominates.

Now, suppose for a moment we knew the value of the [random] market factor. In this case, default would be driven solely by the idiosyncratic factor (since the market factor is fixed, and we have chosen it such that all names either are – or are not – in default). The idiosyncratic factor is, by definition, independent across all issuers. Therefore, we have artificially created a scenario in which defaults are independent for each issuer by conditioning the market factor on a certain level. More specifically, we have generated a set of conditionally-independent default probabilities. Now, repeat the process for every issuer and every market factor level. The result is a complete picture of how every issuer behaves in every possible situation. From this, the unconditionally independent probabilities can be extracted.

(If that isn’t quite clear, suffice to say there’s a bit of math behind it. Interestingly, the math is surprisingly simple, but with the exception of the number of factors in a Gaussian model I have promised not to write out any equations in this post, so in the absence of symbols I hope you will accept my reasoning.)

So now, we have the probability of every issuer independently defaulting at any given time – with that information, it is relatively straightforward to figure out the expected loss on the portfolio. In fact, it’s mainly arithmetic at this point: the value of the portfolio is just the probability-weighted average payoff of all the issuers.

And that’s really it – that’s how the Gaussian copula is used to price a CDO, or a collection of sold CDS on many issuers. We calculate the default probabilities from the CDS, then we use the Gaussian copula to tell us how they relate to each other. You’ll notice that I never actually mentioned the copula when discussing the probability model – that’s because you don’t really need it. It happens that the copula math simplifies nicely into something that is almost, but not quite, entirely unlike a copula (hey! a Douglas Adams reference!). However, the copula-based approach is more informative, even if copula-specific math per se doesn’t enter the picture.

And why is this so bad?

A few of the modeling decisions I’ve described above are unquestionably poor ones, though it may not be obvious how to improve them. Here is my brief rundown:

The Gaussian dependence structure – what’s wrong with it? What alternatives are there? Why are they better?
The single factor – is it really sufficient to describe the behavior?
The single correlation number – is it sufficient to describe the behavior? Can we reliably estimate more relationships? Is correlation the right metric in the first place?

I’ll attempt to answer all these and more in part III…

{ 9 comments }

Interviewing Myron Scholes

May 16, 2009 in Finance, Risk

Speaking of LTCM (and of this Sunday’s Times Magazine, for that matter), here’s an interview that’s going to run with Myron Scholes, who comes off like a bad comedian. The questions are poor and the answers arguably worse. Let’s take a look, shall we?

The second question: “You’re known as the intellectual father of the credit default swap…” Saying this about Scholes is a bit like calling Edison the intellectual father of the personal computer. The Black-Scholes framework introduced modern derivative pricing, yes, but the leap to credit default swaps was independent of their research. Moreover, only the most basic tenets of the BS model are used in pricing CDS. In fact, the math behind a Credit Default Swap is far simpler than any equity option. A high school student could price a CDS; a college student would need to pick up stochastic calculus and multivariable differentiation before he could work through the Black Scholes derivation.

The third question: “In 1997, you shared the Nobel in economics for your Black-Scholes theory. Is the name intended as a riff on the black-hole theory?”

Really?

It’s called Black-Scholes because Fischer Black and Myron Scholes wrote a paper together in 1973, not because Myron wanted to create an intergalactic pun. In fact, he shared the prize with Robert Merton because Merton was the first to expand on the math of pricing options, also in 1973. Merton actually coined the term “Black Scholes model” in his paper; had he not been so modest it would certainly be known today as BSM – which is how many academics refer to it anyway.

The fifth question and answer: Q: “In retrospect, is it fair to say that the idea that banks could manage risk was a total illusion?” A: “What you’re saying is negative. Life is positive too. Every side of a coin has another side.”

Thanks for the insight, Myron.

The seventh question and answer: Q: “Some economists believe that mathematical models like yours lulled banks into a false sense of security, and I am wondering if you have revised your ideas as a consequence. ” A: “I haven’t changed my ideas. A bank needs models to measure risk. The problem, however, is that any one bank can measure its risk, but it also has to know what the risk taken by other banks in the system happens to be at any particular moment.”

A perfect opportunity wasted to point out that models are just the tool. Instead, Scholes here implies that each bank measured its risk correctly but misjudged every other bank. But besides the fact we know this to be false, such a situation wouldn’t necessarily result in the financial destruction we observe today. That would actually be far preferable to the current state of affairs!

Let us repeat the assumptions of the Black-Scholes framework:

Investors can borrow and lend at the risk free rate
Stock prices follow geometric brownian motion with constant variance
There are no transaction costs, taxes or dividends
You can short sell as easily as you can buy
Your option is European

If any of those are violated then the model does not reflect reality. 1-4 are always violated; 5 is up to the investor. Scholes should have aknowledged this instead of running on about how his ideas have extreme utility.

The eighth question and answer: Q:” What good is a theory of risk management if it applies to one tree instead of the forest?” A: “Most of the time, your risk management works. With a systemic event such as the recent shocks following the collapse of Lehman Brothers, obviously the risk-management system of any one bank appears, after the fact, to be incomplete. We ended up where banks couldn’t liquidate their risk, and the system tended to freeze up.”

This reminds me of the saying “VaR is like an airbag that works perfectly every day, but breaks as soon as your car crashes.” Most of the time your risk management works? No – most of the time you don’t need your active risk management! Your risk management only has to work a couple times a year and if it doesn’t work those times then it doesn’t work at all. Risk management systems “appear incomplete” because they are incomplete and insufficient.

The sixth question and answer: Q: “The writer Nassim Nicholas Taleb contends that instead of giving advice on managing risk, you ’should be in a retirement home doing sudoku.’” A: “If someone says to you, ‘Go to an old-folks’ home,’ that’s kind of ridiculous, because a lot of old people are doing terrific things for society. I never tried sudoku. Maybe he spends his time doing sudoku.”

It’s hard to say whether I hate Scholes’ answer or agree with Taleb more strongly.

{ 0 comments }

Projections of future losses may appear closer than they are

May 7, 2009 in Finance, News

AK pointed me toward this piece from The New Yorker, “Stress Test Results: In Line With Other Estimates,” which I excerpt here in its entirety. I’ve bolded the last sentence:

From the moment the Treasury Department announced its plan to stress-test the country’s nineteen biggest “bank holding companies,” the process was dismissed as a whitewash. Critics argued that the tests were not going to be difficult enough, and that, as a result, the projected losses for the banks would be too small, making their balance sheets look healthier than they actually were. But Treasury’s report on the stress tests says that the banks’ total losses through 2010 (including the losses suffered in 2007-2008) could total $950 billion. That’s close to the I.M.F.’s estimate that total losses for U.S. banks from the financial crisis will be $1.1 trillion. And when you take into account the fact that the I.M.F. was estimating losses for all 8000-plus banks in the U.S. system, while Treasury’s number is only an estimate for the losses of the aforementioned nineteen biggest banks, Treasury’s projections look very similar to the I.M.F.’s. That at least suggests that the government used reasonable projections of future losses, and did not, as many feared, rig the tests to make the banks look good.

No, it does not. It suggests that the government’s loss projections line up with the current IMF projections and nothing more. Any inference that those loss projections are “reasonable” hinges on the assumption that the IMF’s projections are themselves reasonable. And they may be, but let’s keep in mind that the IMF’s estimate was revised upwards by 25% just last week. Moreover, last spring the IMF expected global GDP growth of nearly 4% in 2009.

I think the safe conclusion is that to this point, the IMF has systemically underestimated the severity of this downturn. (An implausible alternative is that the second derivative has been accelerating downward, but I’m told that in fact the reverse is true. Isn’t it fascinating how people embrace calculus when it might be able to resurrect their portfolios?) Barring further evidence, it appears safe to carry that assumption forward. We could say that the stress test affirms the IMF view, or vice versa, but we can make few judgements about how accurate either expectation actually is.

However, my point isn’t to argue that the stress test necessarily set the bar too low. I have always held that the stress tests would not give the banks a free pass, because I view the tests as the last opportunity for the executive branch to forcibly inject capital into these ailing institutions. There are two reasons: First, as I’ve stated here recently, the government can not afford to suggest in any way that the banks are solvent, only to have them incur further massive losses. This would undermine or eliminate the credibility of the Treasury. Second, I do not think that taxpayers (and, consequently, Congress) are willing to expand the bailout program, which means the only way to force banks to acquire additional capital is via a mechanism that essentially amounts to peer pressure: when the Government announces to the world that you need capital, how can you afford to ignore their claim? The stress tests are a tool to that end, and the result – about half the banks need some amount of capital – is not surprising from that perspective.

Is the test accurate, however? I’ve made the point before that because the economic assumptions of the test are tracking reality, we may be afforded a rare opportunity to see how close the expected losses mirror reality. Of course, there are major problems with ascribing accuracy to a probabilistic model based on a deterministic outcome, but statistical squabbles aside I’m quite curious to how that plays out.

As for the IMF, their expected result proved so rosy that last week they had add a healthy dose of depression. Maybe now it’s depressing enough, or maybe they had it right before – we don’t know. But the fact that the stress tests have come out in the same ballpark merely means that industry models, given similar inputs, yield similar outputs. The frustrating conundrum remains: garbage in, garbage out. Models are just the tool (if I use that phrase enough times, maybe people will start to believe it) and in the absence of hindsight we have no basis to judge reason or accuracy.

Yet.

{ 0 comments }

Touché.

April 27, 2009 in General

I have been doing some research on text analysis, in particular on extracting sentiment in the absence of context (a la Twitter). Among all the theory, I came across the following joke which I think really captures the complexity of any natural language processing (not that it prevents us from trying):

A linguistics professor was lecturing her class one day. “In English,” she said, “A double negative forms a positive. In some languages, though, such as Russian, a double negative is still a negative. However, there is no language wherein a double positive can form a negative.”

A voice from the back of the room piped up, “Yeah, right.”

{ 0 comments }

How (not) to design a stress test

April 25, 2009 in Finance, News

Of course the big news of Friday afternoon was the preliminary release of the metrics used in the bank stress tests. Unfortunately it didn’t turn out to be much news at all; few hard figures were revealed and those that were came largely within expectations. The NYTimes published an excellent copy of the document right here.

One interesting thing which was confirmed was the fact that the baseline scenario is actually quite generous. It assumes, for example, that the unemployment rate in 2008 will be 8.4%. Of course, we are presently at 8.5% and despite all the talk of the “second derivative slowing,” the first derivative remains steeply negative, implying the unemployment rate will continue to rise. But this does not mean the baseline case is worthless, as some analysts are suggesting (though, to be fair, it should make us severely question the pass/fail result). Instead, it provides an interesting opportunity to analyze the proficiency of the Treasury’s investigation — if the baseline case tracks reality (or, in this case, understates it), then we can expect the banks’ trajectory to mirror the expectation described in the report. There is speculation that the Treasury will lowball the test and let every bank pass; however if the test looks like reality and banks incur very real losses, confidence in the government’s investigative apparatus will be sorely and irreversible undermined.

Another concerning point is the use of “common industry vendor models and developed proprietary models” to generate loss estimates. These are the same risk models which (depending how you prefer to look at it) absolutely failed to account for the losses the banks have incurred thus far or are being used to arbitrarily mark assets far above their market valuations (courtesy the suspension of FAS 157). Moreover, these models are used in a conjunction with “monte carlo simulation” that most likely implies the use of Gaussian assumptions.

But what more can we really expect from just 150 professionals asked to do – in one month – the job that couldn’t be done by thousands at the various banks.

{ 0 comments }

The Modeling Problem

April 16, 2009

Mike at Rortybomb has a post on the pros and cons of MBAs. Among his cons is the following:
They just wrote a post about MBA students being owned by their models. The general idea underneath this is that these models are too complicated to understand, and taught to be something that is true rather than [...]

0 comments Read the full post →

It’s been tried

March 17, 2009

Today’s Dilbert is excellent:

0 comments Read the full post →

Or perhaps his aim

February 24, 2009

All this talk of models and their interpretation reminds me of a saying:
Sometimes the problem is not the arrow, but the Indian.

0 comments Read the full post →