A number of posts about:

Math

An interview with Mandelbrot

October 1, 2009

The FT has posted a lengthy video interview with the brilliant mathematician Benoit Mandelbrot, whose book The (Mis)behavior of Markets first inspired me to enter finance and risk management in particular. I do find  that some of John Auther's questions mar an otherwise interesting (but extremely high-level) overview of Mandelbrot's thoughts on finance. Right from [...]

0 comments Read the whole post →

Suspicious poll distributions

September 25, 2009

I've covered Benford's method for first-digit fraud analysis before, and now Nate Silver has applied a similar method to polling results. He looked at the last digit of various polls (i.e. a 48% McCain, 49% Obama, 3% undecided poll would be recorded as an 8 and a 9) and compiled histograms of their frequencies. Following [...]

0 comments Read the whole post →

Lottery math is not so easy

September 23, 2009

Carl Bialik has written about lottery coincidences in his WSJ print column and on The Numbers Guy blog, inspired of course by the recent consecutive draws in the Bulgarian lottery. Addressing my recent confusion, he sheds a little light on why likelihood estimates varied so much: The probability of Bulgaria's repeated winning numbers became a [...]

0 comments Read the whole post →

Adventures in probability

September 17, 2009

Calculating the probability of the Bulgarian lottery drawing the exact same numbers in consecutive weeks.

5 comments Read the whole post →

Enlightenment

September 11, 2009

A relatively new program has been devised, with the blessing of the Dalai Lama, to instruct Tibetan monks and nuns in science and math. These students have little or no formal education in that area, but are adept learners and take to the material quickly and with interest. My favorite quote from the NYT's report: [...]

0 comments Read the whole post →

Junk Maths

September 10, 2009

Via Andrew Gelman, I've learned that the BBC has a radio programme (as they would say write) called More or Less which is dedicated to statistics. The first bit of the most recent one is called "Junk Maths" (and again, I wish I could have taken a class called "maths") with the following synopsis: Spurious [...]

0 comments Read the whole post →

Search forecasts

September 8, 2009

Google Insights recently rolled out a new feature: 12 month search forecasts. The forecast comes from a relatively simple decomposition of the search volume into trend, seasonal and residual components. The model's out-of-sample performance is tested on the most recent 12 month period; if that prediction proves accurate, then the model is accepted. Here's what [...]

0 comments Read the whole post →

Tanning in perspective

September 2, 2009

Information is Beautiful tipped me off to this poster from GOOD: Skeptic that I am, I immediately questioned the headline as propaganda. The Sun very well may produce that much energy, but how much of it reaches the Earth - in other words, how much of it can actually be harnessed? This makes for a [...]

1 comment Read the whole post →

Modelling interactions

August 18, 2009

Andrew Gelman's latest post highlights the importance of interactions. He includes this breakdown of where people fall depending on political party, ideology, and income: Consider the income dimension. Among liberals, the income curve is flat no matter whether the person is a Democrat, Independent or Republican. For conservatives, however, income has a large effect - [...]

0 comments Read the whole post →

R style guide

August 13, 2009

Google has posted a style guide for R which is being used throughout the organization. It's mostly in line with what I learned once upon a time, but it's nice to see such an authoritative body coming out with a set of standards. Universal coding benefits everyone, and R is growing so rapidly that some [...]

0 comments Read the whole post →

Deconstructing the Gaussian copula, part III

August 11, 2009

The intuition behind copula models: dependence, correlation, single factors and more.

0 comments Read the whole post →

Deconstructing the Gaussian copula, part II and a half

August 11, 2009

An aside on static recovery assumptions in CDO pricing.

2 comments Read the whole post →

Statistics: desired and feared?

August 10, 2009

My former department chair, Xiao-Li Meng, has published an excellent article on the emergent role of statistics and the challenge of teaching the science to non-statisticians. He addresses the negative perception of the field, often ingrained by a poor high school experience and summed up in a dismissive scoff that "the best speaker in statistics" [...]

0 comments Read the whole post →

Bell curves in action

August 6, 2009

An exhibit at MOMA invites visitors to mark their heights on a wall. A normal distribution results: Well, not quite. The distribution is actually slightly negatively skewed by the confounding presence of children, who are obviously shorter than adults - you can see this in the great number of names well below the central band which [...]

1 comment Read the whole post →

A deathly serious game

August 6, 2009

The FT provides a tool for simulating the spread of an infectious disease. Though they caution that the model is not based on any complex algorithm, simplicity should not be mistaken for error. Even a simple routing scheme like this one can capture many of the dynamics of the underlying process, as it doesn't take [...]

0 comments Read the whole post →

Photo finish: the Netflix prize

July 28, 2009

A month ago, the million dollar Netflix prize was finally won by a coalition of leading teams called Bellkor's Pragmatic Chaos, who blended their respective methods into a super-algorithm that finally crossed the 10% improvement barrier. ...or was it? The 10% mark sent the competition into a final, 30-day countdown, during which time other teams could [...]

0 comments Read the whole post →

A response to randomness

July 12, 2009

In response to my post on the WSJ's recent randomness article, B emailed me the following (reproduced here with permission): The quoted WSJ article writes "We find false meaning in the patterns of randomness for good reason: we are animals built to do just that… Many studies illustrate how this basic aspect of human nature [...]

0 comments Read the whole post →

Deconstructing the Gaussian copula, part II

July 9, 2009

A math-free introduction to CDO pricing.

9 comments Read the whole post →

Log differencing Top Chef

July 9, 2009

On the most recent Top Chef, one of the chefs received just half a star (out of five) for his initial dish - impressive because the dish hadn't finished cooking and wasn't even entirely served. More interesting perhaps was his response to getting half a star: "I was sure I'd get nothing... but that's 50% [...]

0 comments Read the whole post →

The subtleties of randomness

July 7, 2009

The WSJ has printed one of the best "fooled by randomness" pieces I've seen in quite a while, titled "The Triumph of the Random." This one uses streaks in sports as a central metaphor, with DiMaggio's 56-game hitting streak as exhibit A. It presents an immediate disclaimer: Recent academic studies have questioned whether DiMaggio’s streak [...]

1 comment Read the whole post →

How did I miss this?

July 1, 2009

In a post called "So Long and Thanks for All the F-Tests", Freakonomics writes about a new book called Mostly Harmless Econometrics: An Empiricist’s Companion, which they describe as: ...the rare book that captures the feeling of how to go about trying to attack an empirical question; and it does this by working through two [...]

1 comment Read the whole post →

On teaching math

June 29, 2009

Arthur Benjamin gives a short (3 minute) TED talk on the problems with how math is taught to high school students in America. He notes that the current curriculum is a sequence beginning with arithmatic and leading to the ultimate goal of calculus. But calculus isn't something most people use once they graduate - how [...]

0 comments Read the whole post →

Inferred ratings and modelling teacher comments

June 24, 2009

Another aspect of my conversation dealt with inferred ratings, a problem I've crossed before in other areas. There are two primary cases in which this arises: censored data and self-selection bias. In the first case of censored data, a problem is caused by the ratings system not eliciting useful responses. An example is a system [...]

0 comments Read the whole post →

Personalized Yelp ratings

June 24, 2009

I had a great conversation last night which at one point verged into the pros and cons of various ratings systems. In particular, we discussed the "star+comment" system used by Yelp, in which between 1 and 5 stars can be assigned in addition to a text comment of arbitrary length. Yelp does some clever things [...]

1 comment Read the whole post →

Misreading misleading charts: entrepreneur edition

June 18, 2009

Paul Kedrosky writes about a study on the rate of entrepreneurship among various age groups, which includes the following piece of junk (ch)art: Why is this chart 3D? It contains information in only two spatial dimensions (time and rate), with a third dimension coded by color. To make the chart itself is a purely superfluous [...]

0 comments Read the whole post →

Wilmott's stages of derivatives

June 18, 2009

Wilmott adapts the Kubler-Ross stages of grief to describe derivatives. An excellent read. Confused disbelief: I'm a great believer in education playing a bigger role in derivatives in future. But not the sort of education that we've got at the moment. I understand Warren Buffett when he says "The more symbols they could work into their writing [...]

0 comments Read the whole post →

When worlds collide

June 17, 2009

I just learned from Andrew Gelman that Mandelbrot wrote a paper on taxonomies... in 1955.

0 comments Read the whole post →

Random forecasts (with echoes!)

June 15, 2009

And speaking of forecasts, I'm reminded today of one of my favorite forecasting errors: the echo. This morning, the manufacturing survey missed the forecasted amount, and many pundits commented that it contributed heavily to the market's fall. Here is a plot of the manufacturing survey level as reported each month in red (prior to any [...]

0 comments Read the whole post →

Copulas in squash

June 15, 2009

Ball marks on a squash court form an interesting scatterplot.

0 comments Read the whole post →

The science of... traffic jams

June 15, 2009

Paul Kedrosky has shared a video demonstrating how traffic jams self-propogate. New Scientist refers to such incidents as "shockwave traffic jams." (This fascinates me to the point that I once wrote a paper on it.)

1 comment Read the whole post →

← Previous Entries

Next Entries →