Fascinating… far from being a psychedelic tour of the imagination, one graduate student argues that Alice in Wonderland is actually a satire of Victorian mathematics:
Yet Dodgson [Lewis Carroll] most likely had real models for the strange happenings in Wonderland, too. He was a tutor in mathematics at Christ Church, Oxford, and Alice’s search for a beautiful garden can be neatly interpreted as a mishmash of satire directed at the advances taking place in Dodgson’s field.
In the mid-19th century, mathematics was rapidly blossoming into what it is today: a finely honed language for describing the conceptual relations between things. Dodgson found the radical new math illogical and lacking in intellectual rigor. In “Alice,” he attacked some of the new ideas as nonsense — using a technique familiar from Euclid’s proofs, reductio ad absurdum, where the validity of an idea is tested by taking its premises to their logical extreme.
A beautiful article in the NYTimes contrasts abstract mathematics with the chilling reality of the Mexican drug cartel wars:
I was born in Mexico City, in a world that seems less and less familiar to me. I live now in the opposite corner of the continent. I am training to be a political scientist at Harvard. My passion has remained the afflictions of my homeland, but at Harvard I have found new ways to address them, to use mathematical models — matrices, vectors, equations, regressions — to understand the Mexican drug crisis.
The cartel wars are extremely violent, and the gangs are responsible for reprehensible kidnappings and deaths. They rank among the most deadly periods of organized crime in human history. The author’s goal isn’t to explain how she can analyze the wars from up in an ivory tower; it’s to describe how her mindset and toolkit inform her understanding of the world in any situation.
The article captured me because it never mentions what the author actually models. Instead, it presents her frightened thoughts and her efforts to calm herself by looking at the world through a mathematical lens. But it’s not what you think; there are no emotionally-distant mathematicians here. The author communicates her fascination with tying reality to abstract models, expecting and preempting the protest that reality is too complex and math too simple:
In this violent world, with the man in the blue Chevy whispering at me behind the window, math is my shield. Speaking up about drugs is in these parts a dangerous game. But not if you speak in the language of sigma and conditional expectations. Math protects me from the immediacy of the violence, and it protects me from them.
The beauty of my method lies in its simplicity. With mathematics I’m able to codify and simplify reality to make it manageable and, more important, malleable. I represent each possible individual as an equation in which each term symbolizes tastes, goals, profession and abilities. All people get portrayed: Policemen, politicians, citizens and drug cartels start living in this mathematical world as planes and hyperplanes and, as in real life, they interact and affect one another, sometimes colluding, sometimes colliding, sometimes neither.
I then use optimization to predict the form of interaction that will be the most probable to emerge and remain over time. Math starts speaking. It tells me, for example, under what conditions the outcome would be a drug war; when would the government prefer to cooperate with cartels; or when cruel intra-cartel purges will become the norm.
There is a part of every modeler’s mind which is constantly teasing out variables from constants. The statisticians among us may take a frequentist view, and wonder what would happen if a scene played itself out a million times; the programmers will deduce the underlying algorithms from the fuzzy result; the pure mathematicians will see manifolds everywhere:
In this abstract microcosmos, reality can be frozen or just slightly changed. I move and look at my hyperplanes from different angles. Let’s change the penalty code. No, let’s increase patrolling. Or reduce wages. Allow less contact between policemen and dealers. Assume the police force is corrupt. Assume it is not. I solve the equations and there it is. My answers come as Greek letters and probabilities.
But we all admit:
I know, I know, this is weird.
Ultimately, “free will” becomes the clarion of the independent. At least, it’s the best response to this explanation:
It may seem strange to examine this shadowy world with equations. But mathematics is transforming the social sciences. In the same way that physicists can predict the movement of atoms in space, we can use mathematics to model how individuals and groups will make decisions and interact in a society.
But free will has a (somewhat tentative) analogue in Heisenberg’s uncertainty principle, and with that philosophy and math (or theology and physics) are combined — but there’s been plenty of pop-sci written on that topic.
I found this brief article remarkable in how it was able to demonstrate the overlay mathematical thought on an extremely “human” subject without ever needing to explain either one.
(Via Drew Conway)
December 23, 2009 in Data
Walmart is running ads right now which claim that shoppers who spend more than $100 per week at the supermarket would save $650 a year by purchasing their groceries at the giant retailer instead.
That’s quite a jumble of conditionals and varying metrics: you have to first meet the requirements of shopping at a supermarket and spending over $100 per week; the savings are then presented in a completely different timeframe of one year. That works out to $12.50 a week, or a still sizable 12.5% discount.
Why not present it as 12.5%? The simple answer is that “$700″ is a substantial figure, and the marketing folks wanted to make people feel like they were saving more; conversely, $5200 a year on groceries sounds like a lot – better restate that as $100 per week. Depressingly, it occurs to me that many Americans may not know what to do with percentages.
Another key point is found in the wording of the ad – why target shoppers who spend more than $100 a week? If Walmart’s prices are really lower, then all shoppers should reap a benefit, not just the high rollers. Since I do not think Walmart is price discriminating (offering discounts only to people spending more than $100), I have to conclude that they restricted their dataset to increase the dollar value of the average person’s savings. If every shopper saved 12.5%, then the average annual dollar savings per person might be, say, $250. But if we consider only people who spend more than $100, the average dollar savings jumps to $650 even though the percent savings remains 12.5%. I would guess that $100 was chosen as a cutoff because a) it’s a round, friendly number which b) creates a relatively high average dollar savings while c) remaining low enough to be in reach of many American families. This, of course, is further evidence that Americans don’t understand percentages well (or at least, that marketers think they can fools us by avoiding them).
Note also that all of my calculations use the stated minimum figure of $100 vs the average figure of $650 to get the 12.5% discount. That’s not a real discount – someone spending $100 wouldn’t get $650 in savings, as that is the average of all the people spending more than $100. That person would realize a smaller dollar savings, and the real discount rate must therefore be less than 12.5%.
December 20, 2009 in Math
The NYT recently ran an article on the math behind the recent and controversial mammogram advisory change. Unsurprisingly, it is heavily centered on a Bayesian argument. Of course, the key point here is not that the statistics dictated the change, but that budgets and political agendas dictated an acceptable level, which the statistics subsequently informed:
Let’s suppose 100,000 screenings for this cancer are conducted. Of these, how many are positive? On average, 500 of these 100,000 people (0.5 percent of 100,000) will have cancer, and so, since 95 percent of these 500 people will test positive, we will have, on average, 475 positive tests (.95 x 500). Of the 99,500 people without cancer, 1 percent will test positive for a total of 995 false-positive tests (.01 x 99,500 = 995). Thus of the total of 1,470 positive tests (995 + 475 = 1,470), most of them (995) will be false positives, and so the probability of having this cancer given that you tested positive for it is only 475/1,470, or about 32 percent! This is to be contrasted with the probability that you will test positive given that you have the cancer, which by assumption is 95 percent.
December 11, 2009 in Math
Via Spontaneous Symmetry, a fascinating story about parallel processing and the power of blogging:
Normally, when [a mathematician] seeks a proof, he locks himself in a room with a chalkboard for long periods of time. He may consult his peers at his university, he may read books, he may look through papers, but the majority of thinking takes place within one brain. It’s serial. Gowers had a better idea. Instead of retreating to a dark room, he posted a section on his blog asking for help with the proof. Anyone from around the world could contribute to the idea by posting a comment. He hoped, in this fashion, to link together the brains of people from all around the world. Gowers eventually received hundreds of comments and, over the course of a few weeks, using the ideas in these comments, he was able to piece together a simple proof.
Though SS aptly notes:
I’m afraid to ask how many inane “comments” the poor mathematician had to wade through between each substantive remark.
November 11, 2009 in Math
Via Spontaneous Symmetry, it appears that some people are a bit rusty on their math. The town of Truro, MA recently voted on a proposed zoning measure which required a two-thirds approval to pass. Out of 206 people, 136 voted in favor – just shy of the required two-thirds. Or was it?
The exact count of the vote — 136 to 70 —had town officials hitting their calculators yesterday. The zoning measure needed a two-thirds vote to pass. A calculation by town accountant Trudy Brazil indicated that 136 votes are two-thirds of 206 total votes, said Town Clerk Cynthia Slade.
Brazil said she used the calculation of .66 multiplied by 206 to obtain the number.
But using .6666 — a more accurate version of two-thirds — the affirmative vote needed to be 137 instead of 136, according to an anonymous caller to town hall and to the Times.
Slade said that she called several of her colleagues to see how they calculate a two-thirds vote, and the answer varied widely. In Provincetown, Town Clerk Doug Johnstone uses .66. But Johnstone said he’d never had a close vote where it might matter.
I don’t understand what the ambiguity is here. Two-thirds of 206 is unquestionably 137.33… which is quite plainly more than 136. If a two-thirds vote is required, then the bill did not pass. Moreover, using a factor of 0.66 is simply wrong. It’s not as if it even saves time – typing “*.66″ takes as many keystrokes as “*2/3″!
What more can we say: math is hard!
Wall Street & Technology briefly discusses some survey results and concludes: “Wall Street’s Quants Feel Misunderstood.” There’s the obligatory quote from Dr. Wilmott:
“These numbers are alarming,” said Dr. Wilmott. “They indicate that even with the events of the past year, financial institutions are still not taking the importance of financial education seriously, especially as it pertains to improving relationships and understanding between quants and their managers.”
There’s some “alarming” statistics: since last year, quants feel that 86% of their managers have the same or less understanding of their quantitative roles.
But looking at the actual survey results, it’s not quite so bad. In fact, there’s a good deal of exaggeration, depending on how you frame the data. Only 4.5% feel that their managers have less understanding since last year – meaning a full 82% felt that the level of understanding is roughly unchanged. So this largely becomes a question of whether or not the present level of understanding is satisfactory. Most people’s knee-jerk reaction (and that of the original article) will unequivocably be, “Of course it’s not!” However, is that because managers haven’t kept up with quantitative advances, or because quants have run far ahead of their supervisors (and of where they need to be)?
I think it’s a little of both. Certainly, when Things Were OK, supervisors were less incentivized to follow the activities of the mathematicians under them. As long as the numbers danced (higher and higher), it didn’t really matter what they were. Meanwhile, each quant is incentively to pursue ever-more obscure models to squeak out minute bits of alpha. In the end, we wind up with quants doing overly-complex work for managers with too-relaxed supervisory roles. The question isn’t “Does your manager understand what you do?” as much as it is “Do YOU understand why you do what you do?”
The problem here is not that quants ran amuck and screwed up the system (see the replies to question #2), it’s that no one even knew what they were doing in the first place. The article is putting a normative spin on the survey results, but it’s silly to believe that if supervisors understood what quants were doing, everything would be fine. Just the same, if quants only worked within the limits of their supervisors’ knowledge, disaster would result as well (what’s the point of roles, anyway?). What is missing – and what surveys like this fail to address – is the need for proper communication of goals, objectives, methods and ideas. Yes, it might be hard for a mathematician to boil his ideas down to simple English or a supervisor to pick up some mathematical tenets, but the resulting clarity will be well worth the effort in either case.
So in the end, is it bad that quants feel like most of their managers only somewhat understand what they do? It’s hard to say. If the quants are doing their job “properly”, then yes. If supervisors are slacking off, then yes. But if quants are running ahead with inappropriate methods, then although the answer is still yes, the solution isn’t necessarily to educate the supervisors – it’s to teach them how to reign in the quants. Alternatively, it’s to teach the quants a little about their real business objectives.
Increasingly, I’ve noted in my discussions with statisticians and practitioners a reliance on Bayesian methods. Bayesian statistics rely on an understanding of the uncertainty of a hypothesis. For example, Bayesian hypotheses are literally updated as new information becomes available. Bayesian analyses will also rely heavily on conditional probabilities, or the understanding of likelihoods that depend on the occurrence of related events. One of the biggest Bayesian proponents is Professor Andrew Gelman, who maintains an excellent blog and is involved in fivethirtyeight.com.
In some ways, Bayesian methods have become a bit fad-like and, as with many fads (I’m looking at you, VaR), there should be concern that they will be applied blindly, without thought. Like anything else, it’s possible to do Bayesian statistics wrong – and even extremely wrong – but when wielded correctly, they make for an excellent investigative resource.
New Scientist has an article on the use – and misuse – of probability in criminal cases. Naturally, it focuses on Bayesian statistics. The key point the article makes is that while it’s important to consider the odds of something happening, it is just as critical to account for the odds of it happening by chance. That may seem contradictory (isn’t an event’s likelihood, by definition, the probability it happens by chance?) so let’s use a classic example, lifted from the article:
You have just tested positive for a disease that affects 1 in every 10,000 people. The test is 99% accurate. On the surface, that sounds like a sound diagnosis, and most people would say they are 99% confident that they do, in fact, have the disease. But consider the following: if every one of the 10,000 people took the same test, then 1 of them would yield a true positive and 99 more would exhibit false positives just by chance. Therefore, among people who have tested positive, there is only a 1% chance of actually having the disease – not the 99% likelihood we naively assumed before!
How does that work – wasn’t there only a 1% chance of the test being wrong? Well, yes – but if you think about it, that 1% chance of error is much larger than the 0.01% chance of having the disease in the first place and the test result must be placed in that context. For the more spatial readers, here is a picture from New Scientist:
The false positive problem is a classic textbook example of how Bayesian reasoning (that is, accounting for the ways in which chance can manifest itself) can affect a seemingly obvious result. It’s a very important consideration which could be overlooked without care. And besides, it makes for interesting pop sci articles.
A bit of out-of-context math from a recent Bloomberg article on AIG:
The Federal Reserve Bank of New York, the regional Fed office with special responsibility for Wall Street, opened an $85 billion credit line for New York-based AIG. That bought it 77.9 percent of AIG and effective control of the insurer.
The government’s commitment to AIG through credit facilities and investments would eventually add up to $182.3 billion.
So, to review: $85B was a 77.9% stake, so $182.3B works out to… 167.1%?
(Obviously, it doesn’t work like that.)
But wait, there’s more.
September 23, 2009 in Math
Carl Bialik has written about lottery coincidences in his WSJ print column and on The Numbers Guy blog, inspired of course by the recent consecutive draws in the Bulgarian lottery. Addressing my recent confusion, he sheds a little light on why likelihood estimates varied so much:
The probability of Bulgaria’s repeated winning numbers became a subject of some disagreement. A Bulgarian mathematician estimated the probability at 1 in 4.2 million, a figure that was widely reported. Clio Cresswell, a mathematician at the University of Sydney in Australia, came up with 1 in 14 million. Many others arrived at 1 in 5.2 million.
One explanation for the wide range is that Bulgaria has multiple lotteries. Dr. Cresswell’s calculations relied on a different Bulgarian lottery with numbers ranging from 1 to 49. Mr. Smith and others made their calculations assuming the possible numbers went up to 42, the correct range for this particular lottery. As for the 1-in-4.2 million estimate, the Bulgarian mathematician didn’t respond to requests for comment.
The blog post in particular is full of really interesting links – I especially enjoyed Professor Leonard Stefanski’s account (pdf) of trying to reconcile accurate statistics with the media’s desire for sensationalism.