I apologize to all my readers for the outage we experienced this weekend. I had to do some server surgery including a very frightening "erase everything" operation that I hoped -- but couldn't be entirely sure -- would be reversed with a TGR backup. This was all compounded by a bit of travelling that left me unable to address the issues until this morning.

The good news is, we seem to be up and running again. I'm sorry again for the inconvenience.

A nocturnal timelapse of the view from the ISS, featuring lighting and lightning:
 

Click here to view the embedded video.

This is probably the best button to press     ↑    

Here's a provocative opinion ("The Elusive Big Idea") in the NYT by Neal Gabler, who laments that "we are living in an increasingly post-idea world — a world in which big, thought-provoking ideas that can’t instantly be monetized are of so little intrinsic value that fewer people are generating them and fewer outlets are disseminating them, the Internet notwithstanding. Bold ideas are almost passé."

Nabler's argument, while extreme, bears some weight. While I disagree that there are no modern "celebrity intellectuals" (and anyway I think it's always a losing game to compare contemporary to historical fame; the historical winners demonstrate not only massive selection bias but also their celebrity has had sufficient time to bake into our hindsight-aided view of pop culture), I completely agree -- and frequently write in these pages -- that the modern focus on sensational information as opposed to true knowledge is tragic. I do not at all believe, however, that this implies that the quest for knowledge is dead.

Nonetheless, I do find it very interesting that Nabler offers an argument that doesn't wind up immediately concluding that "people are just dumber now:"

The real cause [of the post-idea world] may be information itself. It may seem counterintuitive that at a time when we know more than we have ever known, we think about it less.

We live in the much vaunted Age of Information. Courtesy of the Internet, we seem to have immediate access to anything that anyone could ever want to know. We are certainly the most informed generation in history, at least quantitatively. There are trillions upon trillions of bytes out there in the ether — so much to gather and to think about.

And that’s just the point. In the past, we collected information not simply to know things. That was only the beginning. We also collected information to convert it into something larger than facts and ultimately more useful — into ideas that made sense of the information. We sought not just to apprehend the world but to truly comprehend it, which is the primary function of ideas. Great ideas explain the world and one another to us.

This leads to two hypotheses: first, we are inundated by so much information that the act of thinking about it becomes exhausting. It is impossible to apply the same caliber of reasoning to all of the varied tidbits we are surrounded by, so we choose the path of least resistance and apply a constant level -- near zero -- to it all. Moreover, modern society incentivizes recency. As Nabler says, "We prefer knowing to thinking because knowing has more immediate value" and it's no surprise that the post-idea world coincides with the popularity of social networks.

A second culprit is a lack of motivation. Only recently has society emerged into a relatively "foolproof" state, where it isn't necessary to retain massive amounts of information, because the answers are just a phone call/tweet/web search away. This is actually an amazing achievement. But paradoxically, it implies that Wikipedia leads to the downfall of thinking. Why waste precious neurons remembering convoluted formulas or arguments when, with a few minutes notice, they can be reviewed on demand? The greatest ideas have always come from questioning the current state of knowledge, from testing its limits and finding that they give. If we don't question, we can't answer.

We rely on information curators to bring us the news, just as we have come to rely on highly -- even absurdly -- specialized products to take care of every other detail we no longer wish to be bothered by. We tell ourselves that these talking heads rose through a meritocracy and are qualified to tell us the news because they are the best at doing so. We ignore their true incentive -- the biggest audience for advertisers, the least effort for producers -- and believe the frequent sensationalist nonsense because the alternative -- thinking critically -- is hard.

Nabler concludes that the age of thinkers is dead, and that we are "information narcissists" accompanied by a "general media [that] have learned to service our narcissism." I don't think the situation is nearly so dire, nor (and this may surprise you) do I feel like this justifies an attack on social media. I lament that society no longer aspires toward intellectualism even as I cheer the reasons for which it is no longer necessary. I would choose this state over its alternative. The tragedy is not at all a lack of great thinkers; we have many of them, despite any claims to the contrary. The problem is simply that the more knowledge we have, the less we are incentivized to acquire more.

There will always be people who are dissatisfied, who seek to push the envelope and find a better explanation or method.  There have been times in history when such people have been praised, and times when they have been suppressed: Galileo certainly never got to enjoy his celebrity. It would be nice to think that we are more welcoming of such ideas today, but that's hardly a prerequisite for having them. I think there's a big gap between a non-intellectual society and an anti-intellectual one. I'm no fan of the former, but I won't be sounding any alarms until I am convinced we are moving toward the latter (Kansas Board of Ed, we're watching you!)

I'm sure that Nabler's argument could have been made equally convincingly 100, 200 or 500 years ago. The greatest reward goes to the ones who ignore it.

To see some of today's great thinkers in action, check out TED, an organization I've featured many times on this blog. TED invites prominent people from every discipline to give "TED Talks", in which they have 18 minutes to describe their work or research. As a result, the information is generally only an abstract, but it serves as a wonderful and entertaining introduction to some of the great work being done today. True, not every TED talk is interesting or inspiring, and some of them fall in the "pure entertainment" category, but frequently the caliber of presenter and presentation is extremely high.

Also, this seems an appropriate place for this SMBC comic from last week:

I wrote a post yesterday referencing Forbes' coverage of R, which I was psyched about. But as I was reading, one quote in the post stuck in my head. I couldn't get past how bad I thought it made statisticians look, and ultimately that became the focus of my post. I was pretty off base with the whole thing. Nonetheless, Steve McNally, the author of the post, wrote me a comment in which he apologized for "giving quants a bad name", as I had put it. I began to reply in line but decided to write this apology as a separate post instead.

Steve, there's no need to be apologetic. Instead, let me apologize to you. You set out to shine the spotlight on a relatively obscure corner of the world and somehow I ended up biting the hand that feeds - that was really stupid of me. Furthermore, I should have noted more clearly that the offending line was not your words at all, but lifted directly from a summary of a presentation (also not in any way at fault, as we will see).

Finally, it wasn't my intent to criticize at all, but to lament how difficult it is to write about statistics in a manner that "normal people" can appreciate. Obviously, I failed to convey that message in every way -- and in doing so, I also failed the mission of TGR. Somewhere around here it says (or used to say) that TGR is about democratizing information -- taking complicated topics and breaking them down to make them more understandable. That objective really began to take shape after I saw the overwhelming response to a series on the Gaussian copula, and it has been my hope since to take ideas and present them in as clear and informative a manner as possible.

Anyway, Steve wrote a piece about R. He did a really great job. He provided examples, explanations and justifications. He linked to everything. And I jumped on it for a line that wasn't even his.

I argued that the line misrepresented statisticians by not providing a context for the analysis. I still think that's true in an absolute sense, but certainly not in the context of the article and even more certainly not in any way that actually matters. Steve was just trying to give an example of a large company like Facebook that used R. He was not writing about the analysis itself. In fact, I assume he found the result interesting or he wouldn't have chosen to write about it - fully undermining my argument in the first place. The original author of the quote, Michael Driscoll of the Bay Area R Users Group, did present the context (albeit briefly, given that his audience was the aforementioned Users Group) because he actually was writing about the result, and therefore the context mattered.

No, the only person who screwed up here is me -- I should have supplied the context that I thought the line was missing AND recognized that the context didn't actually belong in that post (and this is the super important part) in a manner that could not be construed as an attack on Steve's work.

In the meantime, I have committed a cardinal sin of blogging and removed the post. I made that decision because in retrospect I can't really stand by what I wrote, and it benefits no one to have it up.

I hope Steve - and my readers -- will accept my apology. I missed an opportunity to provide supplemental information that some may have found beneficial. I will try my best to make up for it.

-J

Made me chuckle:

(via SMBC)

LOST is back tonight! And what better way to prepare than an interactive timeline from the excellent NYT graphics team? A good infographic should communicate otherwise-complex ideas in a simple and intuitive manner... oh, never mind, LOST is back and that's really what matters. Check out the timeline here!

Every single street in Boston is one-way, the wrong way. At least, that's what I've believed since I lived there - no matter where you want to go, the roads that appear to form the most direct route will inevitably carry traffic only in the opposite direction. And somehow that remains true when you try to drive back!

Until today, this was just a thought I kept to myself, except to commiserate with other Boston drivers. But now, Andy Woodruff at Cartogrammar (last seen on TGR mapping the colors of Harvard Square) has exposed this bizarre phenomenon to the world. Andy has mapped out a selection of Boston routes which look short and simple as the crow flies but end up being circuitous nightmares thanks to the city's bizarre traffic patterns.

I immediately recognized the first (above) as the loop of Cambridge Common in Harvard Square - anyone familiar with the rest?

(Via Cartogrammar)

The WSJ has crunched the numbers and concluded that:

In nearly 200 years of recorded stock-market history, no calendar decade has seen such a dismal performance as the 2000s.

Investors would have been better off investing in pretty much anything else, from bonds to gold or even just stuffing money under a mattress. Since the end of 1999, stocks traded on the New York Stock Exchange have lost an average of 0.5% a year thanks to the twin bear markets this decade.

I think the authors should have stopped right there, scratched their heads and wondered how the words "twin bear markets" and a tiny loss of just 0.5% a year could appear in the same  sentence. But they did not, they plowed on with such statements as, "From 2000 through November 2009, investors would have been far better off owning bonds."

The answer, of course, was a bubble-fueled bull market of low volatility and cheap credit coupled with a decade of unprecedented volatility in general. If we assume for a moment that instead of being a horde of mindless idiots, investors are shrewd market timers, they could have returned 100% just by buying the S&P 500 basket from 2003 to 2007. They would have had not one but two ("twin") opportunities to reap the same profit shorting the market in 2001 and 2008.

This is why the exercise of boiling arbitrary time periods down to a first order statistic - the average return - is such a meaningless exercise if performed without context. It fails to capture the full texture of the decade - the ups, the downs, the in-betweens. The richness and texture of the last 10 years are to some extent by the S&P's chart. It's not useful, instructive, or even a proper comparison to compress the decade. Show me an investor who purchased the index at the start of the decade and hasn't traded since, and I'll show you someone who actually might find long term bonds a more suitable alternative.

The second order measure is the real story of the decade: in 2008, the VIX hit 80!! And as late as January 2007, it was under 10!

But enough with the decade retrospectives, with the sensationalist headlines comparing this decade's average return to that of the 1930's (here's a hint: Black Friday was in 1929). You want a story? Focus on the twin bear markets - look at how amazing it was that you could have two in such a short span and still only lose 0.5% a year on average!

But the next time I find someone who only invests on January 1 of years ending in zeros, I'll be sure to pass this along.

Aaron Swartz makes some good points about Google, but I just can't keep reading once he invokes a tactic I can't stand (and which sadly seems to be gaining currency):

If Microsoft had Google’s market share in search, is there any doubt that they’d be systematically demoting or even banning their competitors in the search results?

Via Daring Fireball

I saw this ad for the new Cadillac lineup, in particular the CTS coupe, on Sunday night and was left with the feeling that I had seen something almost-but-not-quite-like it before. The sand, the straight-line speed, the radio-quality voices, the ambient nothingness... and finally, I remembered.

Here's the Cadillac ad:

Click here to view the embedded video.

And here's the classic BMW spot that preceded it by 8 years:

Click here to view the embedded video.

Jalopnik's QOTD: What's the worst state to drive across?

Posted as a public service following this announcement (click to zoom):

via Daily Fill.

The NYTimes Magazine has an article about cooking or, more accurately, how that act has is vanishing from American homes. It addresses the causes of that transformation, from pragmatic time constraints and demographic shifts to the technological (this excerpt stuck strongly in my mind):

“Here’s an analogy,” Balzer said. “A hundred years ago, chicken for dinner meant going out and catching, killing, plucking and gutting a chicken. Do you know anybody who still does that? It would be considered crazy! Well, that’s exactly how cooking will seem to your grandchildren: something people used to do when they had no other choice. Get over it.”

Leave it to TGR to find the futurist quote in a largely nostalgic essay.

Much of the article is dedicated to America's fascination with televised cooking, beginning with Julia Childs half a century ago (indeed, the timing of the article is suspiciously fortuitous for Nora Ephron's new movie). As a recent initiate into that sphere, I appreciated the analysis all the more.

Is cooking doomed to fall by the wayside, a casualty of the march of progress? I hope not; there is such reward in a good homecooked meal - even (especially?) when someone else prepares it.

This article on the newly-announced Ferrari 458 is so incredibly bad - in content and in tone - that I am forced to conclude that Ferrari paid the author for the publicity. Moreover, what's it doing on a technology site? The complete sum of knowledge regarding this vehicle is four photographs and a vague press release; it won't be fully revealed until the Frankfurt Motor Show in September - and the article doesn't mention any of that anyway.

Witness:

People who actually own a Ferrari are, to a man, the kind of self-obsessed weasels you'd cross the road to avoid. And they'd be walking because the Ferrari will be under a dust sheet, broken or being serviced.

And if that didn't scare you off:

A Ferrari may be an aspirational vehicle, but then so are BMWs, Mercedes and Jaguars costing a fraction of the price and offering practical, reliable, motoring enjoyment. These cars are bought by people who know about driving rather than simply having more money than sense.

Ah, the old "Why buy a steak at Luger's when you could get one at Friday's?" argument. And here I thought the whole idea behind choice in consumption was to accomodate differing preferences. I really don't want to enter this flame war, so just one more quick thought: is the author absolutely out of his mind?

And yes, I will spare you a soliloquy on the automotive wonder that is shared by all - and exclusive to none - of these manufacturers.

Love this brief observation from Andrew Gelman:

Original title of article: "Estimating turnout, vote intention, and issue attitudes in subsets of the population"

New title: "Who votes? How did they vote? And what were they thinking?"