Posts tagged as:

graph

Overcharting: airfare edition

November 28, 2009 in Data

Nate Silver writes about the dropping cost of air fares – yes, you read that correctly – over at Five Thirty Eight. His writing, as always, is excellent – I only want to point out a chart he uses and how it can be dangerous to draw conclusions at a glance (or, if you prefer, how similar charts can be used to mislead people).

Here’s the chart in question, showing the cumulative percent change in inflation-adjusted air fares since 1995:

At a glance, the chart is convincing: fares are off about 15% since 1995. But how meaningful is that number?

The chart exhibits a very noisy pattern. Just a year ago, Nate could have written an article about fares being unchanged over more than a decade, and he could have noted a steady rise in price following 9/11! It should be clear that the point in time at which the measurement is made is extremely important.

Additionally, the reference or base year matters a lot as well, from a perceptual standpoint. If the y-axis were zeroed on 1996 or 2004, a very different chart would result. Sure, the shape would be the same, but the present chart is almost entirely in negative territory; a different base year would put more points in positive territory. This makes me wonder if 1995 wasn’t just another spike like 1996, 2001, 2006 and 2008. I believe the dataset only goes back to 1995, so this is far from an accusation of cherrypicking data, but it’s possible that a 1994 base would reveal a very different story – either higher or lower.

Finally, people frequently make the mistake with charts like these of observing the gap area (the grey vertical bars) and attributing meaning to it across its entire length. In this case, that means looking at the two lines and making a statement like “the top 25 airports continued to outpace the rest of the airports in the last decade.” In reality, however, the two groups are almost exactly the same from 2003 to 2009. There is a one-time structural break following 9/11 and lasting about a year or two, during which time the top 25 markets experienced greater price drops than the rest. After that, the price changes are in lockstep. If both time series were zeroed on 2003, the lines would move in tandem following that date. I see this mistake frequently in interpreting the difference between two stocks – a divergence in prices, no matter how stable, always seems to imply a persistent difference even if the split was a one-time event.

My thoughts here have absolutely nothing to do with Nate’s post – please read it as I haven’t covered his reasoning at all – I merely want to take advantage of his graph to demonstrate these potential pitfalls. How’s that for some Saturday afternoon reading material?

{ 0 comments }

Pie chart fail

November 27, 2009 in Data

Via FlowingData, I found this amusing pie chart from a local Fox News broadcast:

The survey plainly allowed people to give more than one answer, resulting in responses that were not mutually exclusive. It’s tiresome but bears repeating: pie charts are only suited to data which adds up to 100% (and then, only if there are a few responses). This data obviously makes no sense in a pie format.

Full video of the broadcast embedded below:

{ 0 comments }

Radial clustering

September 14, 2009 in Data

Finally, a radial visualization which serves a purpose rather than just looking cool. Getting Genetics Done has a tutorial on using clustering functions in R. In it, they show how this this analysis:

is much better represented like this:

There’s nothing wrong with making a chart which looks good – in fact it’s encouraged - so long as the visual niceties enhance the message of the graphic. Radial graphics are all the rage these days, but they rarely help with information communication (and in many cases they detract!). It’s nice to see a truly constructive application of the technique.

(via Revolutions)

{ 0 comments }

How to fix a broken pie chart

September 8, 2009 in Data

Datavisualization.ch has a helpful step-by-step on how to turn this (from a Mashable post):

into this:

Of course, the motivation is worth more than the mechanics.

{ 0 comments }

Twitterverse demographics

August 28, 2009 in Internet

I spoke too soon – another post from ReadWriteWeb manages to frustrate yet again. In an article claiming that teenage use of Twitter is on the rise, they present this chart:

Let’s do what RWW did not and actually think about what this graph is showing. For each age group, their use of Twitter is plotted over time, relative to their use of the internet as a whole. In other words, this is a visualization of the relative composition of the Twitterverse. If all age groups used Twitter similarly to their overall internet consumption, then all the lines would be at 100.

I do find it amusing that RWW has a almost cliched “statistics can be misleading” section in its article, which fails to note the single most important caveat (unsurprisingly, given their misinterpretation of the chart): increased participation by any one age group must be offset by decreasing participation by another. So the rise in the “12-24″ line is equally and exactly offset by declines in the adult groups. Kind of a different headline, isn’t it: “Adults Abandon Twitter!” And yet, it’s based on the exact same information.

At this time we should note that just two days ago, the Times ran an article called “Who’s Driving Twitter’s Popularity? Not Teens.”

The key here is that we don’t know whether teens are using Twitter more or adults are using it less. All we know is that if you look at the Twitter userbase, teenagers form a greater percent of the community than they used to – even though the absolute number of teenage Twitterers could be static or even dropping (if adult use was falling off at a greater rate).

What’s much more interesting is that for the first time, teens are using Twitter disproportionately – they are a larger demographic of the Twitterverse than the internet generally. But again this gives us no context, and that fact could arise from their increased participation or adult accounts going stagnant.

It’s interesting and informative to note that young people are a steadily growing percentage of the Twitterverse. It is a mistake to make assumptions about their number from the graph, however.

I fully expect an article from RWW examining the “massive rise” in “2-11″ Tweets – who are these tweeting toddlers? What do they tweet about? And most importantly, how can your marketing strategy take advantage of this trend?

Update: I am not surprised to learn that this graph comes from Silicon Alley Insider’s Chart of the Day column. I cringe at the thought of that site’s influence.

{ 1 comment }

Monday light reading: Stevens’ power law

August 24, 2009 in Data

A post on Junk Charts sent me reading about Stevens’ power law, which supplies a quantification of a problem I’ve discussed before: the danger of representing single-dimensional data with two-dimensional graphics.

Stevens’ law measures the amount by which humans over- or under-perceive a stimulus, relative to its actual intensity. For example, the coefficient for “visual length” is 1, meaning that humans accurately gauge the true difference between lines of various lengths. However, the coefficient for “visual area” is just 0.7, meaning we underestimate differences in area by 30%!

This follows from the arguments laid out previously – area increases with the square of the one dimensional metric; therefore, as we look to that single measurement’s representation in a two-dimensional graph (say, the radius of a circle), we fail to account for the compounding effect of squaring it as it grows. This leads to an underperception of relative differences in area. Using a single-dimensional metric, like pure length in a bar chart, is much more appealing because our perception of variation will scale linearly with the actual measurements.

{ 0 comments }

The 100 users of Twitter

July 31, 2009 in Internet

An interesting visualization of Twitter as 100 people is a good take on a popular infographic meme, but reveals a few inconvenient truths about these sorts of images.

Firstly, although I am (not so) secretly pleased to see this illustration of Twitter’s non-inclusive communicative nature let’s not forget that Twitter, like so many other social phenomena, follows a power law distribution. We’ve all heard a lot about the “long tail” – here it is in action.

Second, “100 people” visualizations, or any display of percentages, need to have exclusive categories to work well; otherwise they may not add up to 100%. In this case, are the “5 loud mouths” really different people from the 5 with more than 100 followers? And couldn’t one of the users with many followers also be a lazy account? These overlaps create issues in the discrete presentation of demographic groups. If the groups really are exclusive – which in this case would have to be by chance rather than by design – then the graphic works well.

Finally, more of a nitpick than anything else: if Twitter were a community of 100 people, then how could anyone have more than 100 followers? Obviously, the number refers to the true Twitter population, but it’s incongruous with the graphic. One option is to scale 100 people down to this sub-community, but then the figure would lose its impact, for the scaled version of 100 users would be just .0024 (based on a true population of 4,200,000 Twitter users). A second option is to abandon the “100 people” metaphor and go with a percentage-based pie chart, but that would ruin the appeal of the infographic.

For a truly excellent set of “100 people” visualizations, see Toby Ng’s collection.

{ 0 comments }

Misreading misleading charts: entrepreneur edition

June 18, 2009 in Math

Paul Kedrosky writes about a study on the rate of entrepreneurship among various age groups, which includes the following piece of junk (ch)art:

Why is this chart 3D? It contains information in only two spatial dimensions (time and rate), with a third dimension coded by color. To make the chart itself is a purely superfluous move – in fact, it’s worse, because it distorts the graph.

One of the points of the study is that the rate of entrepreneurial activity is much lower among young people than older people. Thus, in the chart above, the blue line should be lower than the other lines. However, the forced perspective of the 3D chart makes the blue line appear even lower than it really is. For example, if all age groups had rates of 0.27% in 1996, then the blue line would print physically lower on the page than every other line despite having the same value.

Adding an extra dimension when it is unnecessary is a serious charting mistake. Adding an extra dimension beyond what your medium (in this case, a 2D screen) is capable of displaying requires serious justification. In this case, it adds nothing and even detracts from one’s ability to read the graph.

Of course, it does so in a way that enhances the study’s point (that blue line is REALLY low!). Perhaps the third dimension wasn’t added solely for visual effect, but for suggestive purposes as well?

{ 0 comments }

Misreading misleading charts: VIX edition

June 16, 2009 in Finance

Get your tin foil hats back out! Zero Hedge can’t seem to keep their manipulation theories under control (I addressed one here in one of TGR’s most popular posts) and today’s example is to egregious to pass up.

In this post, Zero Hedge reviews ground breaking “research” from Innovative Quant Solutions “on the very relevant topic of whether the VIX has any predictive power left at all, and just how gamed of an indicator has it lately become.” The brief is titled “Is the VIX Too Low?” and consists of six graphs, or appendices.

The VIX is a “volatility index;” its value corresponds to the annual volatility implied by options on the S&P 500 index. Without going into the details of its construction, its number should be interpreted as the mairket expectation of volatility over the next 30 days. Today, for example, the VIX moved back above 32, indicating that the expected annual volatility of the S&P 500 index over the next month is currently 32%. I should stress that the IQS analysis was performed on the 500 individual stocks which comprise the S&P 500, rather than the index itself.

Ultimately, for those who choose not to read ahead, I’ll demonstrate that IQS’s conclusions are based on flawed comparisons of a short term forward looking measure (VIX) and a long term historical reading (S&P annual realized vol), using merely graphs (no statistics) that have two axes (thereby preventing comparisons).

Here we go: Appendices A-C show the historical realized return and volatility of the S&P 500 components, and are of little analytical interest. Appendix D, however, is where the fun begins. Appendix D states:

We overlay S&P 500 of Equal-Weighted Returns (A) with the VIX. The pattern is consistent with intuition. Higher volatility of returns for higher VIX. The VIX has been declining while volatility of returns remains high. Is the VIX high enough for the latest data point?

Their question (“is the VIX high enough?”) is very difficult to answer by looking at the accompanying chart because it overlays returns and volatility on two axes. The relationship between returns and volatility is that between draws from a distribution and the distribution’s second moment; though there is a clear mathematical relationship, it is not one that can be inferred from a graph. Most obviously, returns can be negative and the VIX is always positive. Moreover, they are plotted on two axes! Adding a second axis is the easiest way to distort a chart – for example, the majority of the VIX points are plotted “below” the returns. That’s a choice of IQS, perhaps to enhance their argument that the VIX is too low. Either way, with two axes you can not make “too high” or “too low” observations because the height on the chart is arbitrary depending on the chosen scale.

Appendix E states:

We overlay S&P 500 Cross Sectional Volatility of Monthly Returns (B) with the VIX. Again, the pattern is consistent with intuition. The two lines seem to track each other quite well.

This chart has major problems as well. Specifically, it still has two axes and their very-different scales mean that the time series aren’t tracking at all. Yes, they are correlated, but if the VIX were really a volatility index, wouldn’t you expect it to at least have the same values as the volatility itself? For example, the last VIX point is about 29; the last standard deviation observation is about 12. The fact that they overlay is encouraging but hardly conclusive.

And now Appendix F, the prime evidence for the paper’s claim:

We overlay S&P 500 Volatility of Time-Series of Monthly Returns (C) with the VIX. The lines have tracked reasonably well over time. However, the latest data points suggest that either the VIX is too low or that recent realized volatility in returns will dampen sharply.

This chart shows incomparable time series. Specifically, it lines up the implied vol over the next 30 days with the realized vol over the last 12 months, a ridiculous pairing. Also, their two axes are coming back to haunt them: the last VIX observation is about 29; the last realized observation is about 36. If you just look at the graph, however, you might think the difference was much greater. This is a perfect example of why having two axes is extremely misleading.

So, what can we conclude from these junk charts? Nothing. Any message is lost in poor presentation and unclear comparisons. In reality, the VIX tracks 30-day vol quite well (if slightly overstating it); this is a well-known result. In “normal” times, the VIX even has some predictive power for S&P vol. In abnormal times, like 2008, the VIX only rises in lockstep with backward looking vol (no predictive power).

I have such a bad taste in my mouth from plodding through “research” like this. Tin foil hats may be removed.

{ 0 comments }

Truth in advertising?

June 15, 2009 in Economics

I find this graph very interesting, not just because of any implied political statements, but for how it highlights the absurdity of economic forecasting and the potentially misguided trust we place in such numbers.

The blue lines were circulated by Obama’s economic team when they were pitching the stimulus bill in order to illustrate its beneficial impact on national unemployment. The red line is the realized unemployment rate to date.

There are two ways to read it, depending on your objective:

Obama’s economic team was overly optimistic, underestimated the severity of the crisis, and the stimulus plan has failed to help as advertised.
Obama’s economic team was overly optimistic, underestimated the severity of the crisis, but things would have been much worse without the stimulus.

Ultimately, the question is whether the level or the shape of the graph is more important. Personally, I find it surprising that (as with the bank stress tests), a situation which was markedly better than a worst case scenario was used to demonstrate the effects of the stimulus. Nonetheless, the fact that this graph was used for demonstration purposes makes it difficult to fault simply because it was plotted 1% too low.

Perhaps it never should have been circulated in the first place. This raises a very touchy point in forecasting: an expectation is almost never perfectly realized. Unless an audience comprehends that fact, then putting a forecast out there can only lead to critique. In a simple example, if I calculate a distribution of outcomes and know it to be the correct distribution with high certainty, then my forecast will be the mean or expected value. But what are the chances that the mean is actually the realized outcome? To be sure, higher than any other single observation, but relatively small nonetheless. This speaks to the importance of confidence intervals and margins of error; my guess, however, is that the margins of error on this graph (however that might be measured) would have included the “improvement” line, making the difference not statistically significant.

More pointedly, however, the stimulus was supposed to “save or create” 4mm jobs. This means that the area between the two curves equals 4mm, but the implied difference here seems much larger to me.

{ 0 comments }

Illustrating the importance of data visualization

June 12, 2009

Andrew Gelman discusses research on attitudes toward gay marriage, by state, and notes this graph in particular, which shows the change in opinion over the last 15 years:

Critically, he points out that the states which experienced the greatest change in attitude were the ones that already were most receptive. A naive analysis of the data [...]

0 comments Read the full post →

Critiquing the Crimson

June 9, 2009

The Harvard Crimson has published its annual senior survey, which is making headlines in part because very few seniors are going into finance. Selected results were presented in an interesting visualization (the image below links to a full size pdf):

Now that my brother has graduated after successfully steering the Crimson’s business operations to one of [...]

0 comments Read the full post →

Breaking down labor mobility

June 7, 2009

Great graphic from the NYT (click to zoom):
(via LL)

0 comments Read the full post →

Dead shoots?

May 22, 2009

Happily, I’ve only used the term “green shoots” one time in the brief history of TGR, and then only sarcastically in the title of this cartoon (which I stand by, as this post should make evident).
The term has always struck me as ridiculous, and not solely because it was first uttered at a time when [...]

2 comments Read the full post →

On graphing horse races

May 8, 2009

In response to Andrew Gelman’s call for interesting visualizations of the Kentucky Derby, Megan Pledger created the following graph:
I think it’s especially interesting because the data is fictional, based on a few simple rules to simulate horse behavior (that’s right – this is just like a single realization of a Monte Carlo process!). Andrew has [...]

0 comments Read the full post →

Presidents vs Pirates

April 15, 2009

Two excellent graphs making the rounds – the first showing Obama the Pirate-Slayer:

And the second with historical context (that being the First and Second Barbary Wars):

0 comments Read the full post →

The graph is half full

March 31, 2009

The Big Picture has a post which borrows two graphs from Credit Suisse that are meant to illustrate the performance of the S&P 500 in the 100 days following a “major trough.” I re-borrow them here:

It looks like the top graph represents a collection of bear market bottoms, which are easily identifiable by the characteristic [...]

0 comments Read the full post →