Fascinating… far from being a psychedelic tour of the imagination, one graduate student argues that Alice in Wonderland is actually a satire of Victorian mathematics:
Yet Dodgson [Lewis Carroll] most likely had real models for the strange happenings in Wonderland, too. He was a tutor in mathematics at Christ Church, Oxford, and Alice’s search for a beautiful garden can be neatly interpreted as a mishmash of satire directed at the advances taking place in Dodgson’s field.
In the mid-19th century, mathematics was rapidly blossoming into what it is today: a finely honed language for describing the conceptual relations between things. Dodgson found the radical new math illogical and lacking in intellectual rigor. In “Alice,” he attacked some of the new ideas as nonsense — using a technique familiar from Euclid’s proofs, reductio ad absurdum, where the validity of an idea is tested by taking its premises to their logical extreme.
The Sortino ratio has emerged as a popular risk measure when evaluating investments. It is a modifcation of the Sharpe ratio, a workhorse indicator of mean/variance economics.
The Sharpe ratio is constructed like this:
where 
is the expected return, 
is a benchmark hurdle, and 
is the standard deviation of the returns. If you buy into a Gaussian mean/variance paradigm, then the Sharpe ratio tells you how many units of excess return you receive per unit of risk you take.
The Sortino ratio is constructed similarly:
Here, 
is the downside deviation, or the standard deviation of returns below the benchmark. The intuition of using this statistic is that people do not penalize investments for positive volatility (i.e. unpredictable but beneficial returns); they only care about negative volatility.
And here lies the rub: it’s very easy to calculate a misleading Sortino ratio. The popular method – you’ll see it floating around the web – is to take any positive (or above-benchmark) return, change it to a zero, and calculate a standard deviation as one normally would, across all returns.
To me, that’s not right. You are artificially introducing a steady stream of zeros into your calculation, depressing the volatility calculation. A more proper way is to throw out any positive returns, and calculate the standard deviation of the negative returns (it should not be surprising that this method complies with the intuition for using the Sortino in the first place).
So the next time you’re presented with a Sortino ratio, take care to understand whether it includes zeros or not – if it does, the denominator is necessarily biased toward zero, and the ratio is overstated.
A beautiful article in the NYTimes contrasts abstract mathematics with the chilling reality of the Mexican drug cartel wars:
I was born in Mexico City, in a world that seems less and less familiar to me. I live now in the opposite corner of the continent. I am training to be a political scientist at Harvard. My passion has remained the afflictions of my homeland, but at Harvard I have found new ways to address them, to use mathematical models — matrices, vectors, equations, regressions — to understand the Mexican drug crisis.
The cartel wars are extremely violent, and the gangs are responsible for reprehensible kidnappings and deaths. They rank among the most deadly periods of organized crime in human history. The author’s goal isn’t to explain how she can analyze the wars from up in an ivory tower; it’s to describe how her mindset and toolkit inform her understanding of the world in any situation.
The article captured me because it never mentions what the author actually models. Instead, it presents her frightened thoughts and her efforts to calm herself by looking at the world through a mathematical lens. But it’s not what you think; there are no emotionally-distant mathematicians here. The author communicates her fascination with tying reality to abstract models, expecting and preempting the protest that reality is too complex and math too simple:
In this violent world, with the man in the blue Chevy whispering at me behind the window, math is my shield. Speaking up about drugs is in these parts a dangerous game. But not if you speak in the language of sigma and conditional expectations. Math protects me from the immediacy of the violence, and it protects me from them.
The beauty of my method lies in its simplicity. With mathematics I’m able to codify and simplify reality to make it manageable and, more important, malleable. I represent each possible individual as an equation in which each term symbolizes tastes, goals, profession and abilities. All people get portrayed: Policemen, politicians, citizens and drug cartels start living in this mathematical world as planes and hyperplanes and, as in real life, they interact and affect one another, sometimes colluding, sometimes colliding, sometimes neither.
I then use optimization to predict the form of interaction that will be the most probable to emerge and remain over time. Math starts speaking. It tells me, for example, under what conditions the outcome would be a drug war; when would the government prefer to cooperate with cartels; or when cruel intra-cartel purges will become the norm.
There is a part of every modeler’s mind which is constantly teasing out variables from constants. The statisticians among us may take a frequentist view, and wonder what would happen if a scene played itself out a million times; the programmers will deduce the underlying algorithms from the fuzzy result; the pure mathematicians will see manifolds everywhere:
In this abstract microcosmos, reality can be frozen or just slightly changed. I move and look at my hyperplanes from different angles. Let’s change the penalty code. No, let’s increase patrolling. Or reduce wages. Allow less contact between policemen and dealers. Assume the police force is corrupt. Assume it is not. I solve the equations and there it is. My answers come as Greek letters and probabilities.
But we all admit:
I know, I know, this is weird.
Ultimately, “free will” becomes the clarion of the independent. At least, it’s the best response to this explanation:
It may seem strange to examine this shadowy world with equations. But mathematics is transforming the social sciences. In the same way that physicists can predict the movement of atoms in space, we can use mathematics to model how individuals and groups will make decisions and interact in a society.
But free will has a (somewhat tentative) analogue in Heisenberg’s uncertainty principle, and with that philosophy and math (or theology and physics) are combined — but there’s been plenty of pop-sci written on that topic.
I found this brief article remarkable in how it was able to demonstrate the overlay mathematical thought on an extremely “human” subject without ever needing to explain either one.
(Via Drew Conway)
Today’s date represents a binary string.
So did yesterday’s. So will November 1’s. This is not news.
But today’s is a palindrome. This is slightly more newsworthy.
Geeks, rejoice.
December 20, 2009 in Math
The NYT recently ran an article on the math behind the recent and controversial mammogram advisory change. Unsurprisingly, it is heavily centered on a Bayesian argument. Of course, the key point here is not that the statistics dictated the change, but that budgets and political agendas dictated an acceptable level, which the statistics subsequently informed:
Let’s suppose 100,000 screenings for this cancer are conducted. Of these, how many are positive? On average, 500 of these 100,000 people (0.5 percent of 100,000) will have cancer, and so, since 95 percent of these 500 people will test positive, we will have, on average, 475 positive tests (.95 x 500). Of the 99,500 people without cancer, 1 percent will test positive for a total of 995 false-positive tests (.01 x 99,500 = 995). Thus of the total of 1,470 positive tests (995 + 475 = 1,470), most of them (995) will be false positives, and so the probability of having this cancer given that you tested positive for it is only 475/1,470, or about 32 percent! This is to be contrasted with the probability that you will test positive given that you have the cancer, which by assumption is 95 percent.
David Spiegelhalter is the Professor of the Public Understanding of Risk at Cambridge University. He has recently produced the following video to encourage better practices in the casual perception of risky behaviors:
I think it’s a brilliant video and would love to have been one of Professor Spegelhalter’s students. I firmly believe that the study of risk and statistics more generally suffers more than anything from a particularly awful and dare I say boring curriculum, not to mention one which many teachers choose to render in terms beyond the grasp of many students. Efforts like this go a long way toward alleviating that obstacle and I applaud the professor for his work.
December 11, 2009 in Math
Via Spontaneous Symmetry, a fascinating story about parallel processing and the power of blogging:
Normally, when [a mathematician] seeks a proof, he locks himself in a room with a chalkboard for long periods of time. He may consult his peers at his university, he may read books, he may look through papers, but the majority of thinking takes place within one brain. It’s serial. Gowers had a better idea. Instead of retreating to a dark room, he posted a section on his blog asking for help with the proof. Anyone from around the world could contribute to the idea by posting a comment. He hoped, in this fashion, to link together the brains of people from all around the world. Gowers eventually received hundreds of comments and, over the course of a few weeks, using the ideas in these comments, he was able to piece together a simple proof.
Though SS aptly notes:
I’m afraid to ask how many inane “comments” the poor mathematician had to wade through between each substantive remark.
I’ve previously covered the danger of attributing meaning to a forecast which is obviously based on little or no information. In that case, it was the manufacturing survey, which one might dismiss as a more obscure measure. Recently, however, Ken Houghton has written a pair of posts on inflation forecasts that bring me back to that argument.
In his first, he presents a study that seems to show that, indeed, inflation expectations tend to assume that the future will look just like the present:
Again, this does not surprise me, as the futre expectation of a random walk is its present value. In the second post, the time series of inflation vs expectations is presented:
With the additional dimension of time, I can see a simple heuristic for inflation expectations: consumers think that inflation will stay at roughly the same level that it is on any given day, with some slight reversion to the Fed target, unless inflation is currently below the target, in which case they think it will rapidly bounce back to – or above – that level.
You can see the inflationary spikes in 2006 echoed in the 2007 forecast; the sharp 2008 increase and subsequent fall are mirrored in the 2009 forecast for the time they remain above the target, at which point they halt their slide.
These charts tell me two things. First, that consumers have very little insight into future inflation levels, to the point that they are unwilling to even choose a simple number like 3% and prefer instead to say that the future level will be similar to today’s. Second, that consumers have blind faith in the Fed’s ability to keep inflation at or above its target level – even in the face of evidence against that power.
November 17, 2009 in Math
Silicon Alley Insider is running a series of posts called “15 _______ questions that will make you feel stupid.” The blank has been filled twice with “Google interview” and most recently with “management consultant interview.” I particularly enjoyed one of the Google questions:
If the probability of observing a car in 30 minutes on a highway is 0.95, what is the probability of observing a car in 10 minutes (assuming constant default probability)?
I have no idea how the word “default” snuck in there – I’m guessing whoever wrote this had a need to relate things back to dangerous CDS! – but the question is a good one. However, the answers posted on the site are absolutely horrendous. One ardent commentator wrote:
“observing a car in 30 minutes on a highway”
If 30 min = 95%, then 100% probability = 30/0.95 => 31.5 min
(ie, the max interval between 2 cars could be 31.5 min)
Probability in 10 min = 10/31.5
You have to wonder if, by his logic, there’s really a 110% chance of seeing a car in 34.7 minutes?
The correct answer is below…
- The probability of observing no cars in 30 minutes is 1-95%, or 5%
- The probability of observing no cars in 10 minutes, p, must agree with the statement p^3 = 5%, since three consecutive carless 10 minute periods will pass with 5% probability.
- Therefore, p = 36.8%
- And the probability of observing a car in 10 minutes is 1-p, or 63.2%.
This morning, I was excited to see two of my interests collide as Nathan from FlowingData posted a tutorial for creating a choropleth: a map that uses color to convey values (I didn’t know that’s what they’re called either). He used county-level unemployment statistics to generate the following image:
However, the process appears quite intense, involving some python scripts and mucking around inside an SVG file. I half-heartedly wondered if there wasn’t a simpler way to create the image. And just then, along came David from Revolutions to throw down the gauntlet: could anyone come up with a way to replicate Nathan’s map in R?
David’s post pointed me toward R’s maps package, and off I went to start downloading the tools…
It took some time to coerce the BLS data into a compatible form; R don’t understand the FIPS county identifiers, so I had to jump through some hoops to get the strings to match (BLS uses state abbreviations; R wants full names. BLS puts in the words “county”, “parish” or “borough”, R doesn’t expect those to be passed. The BLS has a “Miami-Dade” county in Florida; R recognizes only “Dade”. Etc.) Ultimately, I used the following code to format the strings:
With the data in the correct format, I aligned a color vector with R’s list of counties and plotted the result:
It came out like this:
Not too bad, I think. It’s a little rough around the edges and a couple of counties are missing – I assume they are the ones with odd naming conventions (you’ll notice I manually adjusted Miami-Dade in my code). Also, I’m not sure how to bring Hawaii and Alaska into the picture. Moreover, the image doesn’t look too good in R itself. For example, I had given up on getting the county borders to show up as faint lines (I could only get them to be completely opaque) – imagine my surprise when I exported the chart and could see the borders just fine!
In any case, I wasn’t satisfied with this result. I’ve been experimenting with ggplot2 and remembered it had some mapping functions, so off I went to recreate the image with yet another library. Ggplot2 is an excellent general-purpose graphics library; the maps package feels positively last-gen after playing with ggplot2. It’s much more extensible and has many more parameters to experiment with – hard to believe it’s not the standard graphics package that ships with R (which itself is another last-gen experience).
Anyway, I kept the data formatted as above – which may have added an extra line or two to the ggplot2 code, but makes it simpler to jump back and forth – and used the following script to draw a new version of the map:
And the resulting image:
Again, a couple drawbacks: Alaska and Hawaii are nowhere to be seen and the borders are slightly aliased. The aliasing does make a difference, especially when compared to the maps output, but the ease with which I put together the latter graph and the frustration I experienced with the maps package, in my mind, more than erase that perceived shortcoming.
On the whole, I’d still take Nathan’s map over these as a finished product. However, I don’t think R can be beat for ease of use and all-in-one packageability – if I wanted, I could run regressions on the data, overlay my chart with more colors or new metrics, explode out certain counties or states… the possibilities are endless. With just a couple lines of code, I could overlay states the voted for Obama in blue, or highlight counties starting with the letter “C”. The static SVG method doesn’t allow any of that flexibility. Also, I’m completely confident that if I had any experience with these mapping packages – rather than using them for the first time tonight – I could mimic Nathan’s image perfectly.
The ggplot2 package, in particular, is fantastically powerful. I really wish I had discovered it sooner. As a matter of fact, Josh Reich runs a monthly R meetup for R users in the New York area and the next topic happens to be ggplot2 – it’ll be my first time attending, so I can’t really say what to expect, but I’m definitely looking forward to it.