Eric Fischer has updated the Geotagger's World Atlas (previously covered on TGR here) by overlaying an analysis of photographers on the geo-located picutures. The result is even more stunning, capturing the different behaviors of locals (blue) and tourists (red):

He drew conclusions by examining other photos by the same photographer. If they had taken photos in the same city more than a month apart, then they were probably a resident of that city; otherwise, they were most likely a tourist.

This is a great example of intelligent data science, putting a second-order correlation on full display. The original dataset would not have yielded this insight. It took a transformation and recombination of the data before the inference could be drawn. I hope the count-the-words-in-the-tweets crowd is paying attention. Brilliant work.

Eric Fischer posts the Geotaggers' World Atlas - a collection of urban networks revealed by the location of pictures taken along their routes. The geographic data comes from Flickr and was clustered and plotted to reveal various city grids. A fairly straightforward mashup of data and geography coupled with a clean visualization... I love this stuff.

In a report that should surprise no one, the NYTimes finds that convenience yields higher revenues for NYC taxis. Indeed, the credit card machines which are now mandatory for every cab have coincided with revenues rising 13% from last year, bucking a national trend of -15%. Tips have risen from 10% of the fare pre-cards to 22% with the new system - and considering that only 28% of rides are paid for with plastic, that means the credit card tips are significantly higher.

Much of that may be attributed to the choices which are presented to riders when the ride ends, starting at $2, $3 and $4 and graduating to 20%, 25% and 30% as the fare increases. For a short trip, the fixed dollar amounts can represent massive percentage tips (an informal NYT survey reported a 38% average, most of which came from people using the presets). Obviously, this beats the old system of rounding to the next dollar, when (in my cynical view) tips were given not to help the driver but so the passenger wouldn't have to deal with change. (Of course, use of the preset amounts suggests convenience remains a key factor.)

But all is not well: predictably, some taxi driver representatives are playing down the report, saying, for example "I know that's not true." And as we all know, anecdotal evidence from people with incentives to be biased is always more useful than TLC data (that's the all-powerful Taxi and Limousine Commission, for the non-NY'ers among you). But the choice quote - the one which inspired me to dedicate a few hundred words to this analysis of what has become a daily ritual for me - is this gem, which puts a bizzare sense of accounting on display:

“Because of credit cards we get customers, that’s true,” said Muhammed Hamid, 35, of Queens. “But if they give us cash, you can put the gas on that; you don’t have to wait three, four days.”

It's true that credit card payments take three days to clear. But why does that make a difference? If a taxi driver switched from all cash payments to all credit card payments and did exactly the same amount of business and received exactly the same amount of tips, there would be a period of three days during which he would realize no revenues - the days immediately following the transfer. From then on, there would be no difference; he would get cash every day, it would just be the cash from work done three days previously. Because of this rolling accumulation, there is absolutely no practical difference in revenues. Even if there were some bizarre quirk of the universe in which the credit card payments somehow lumped together and did not pay on a rolling basis, which there is not, the cash that is being received in the meantime can pay for gas.

I'm frankly at a loss to deconstruct this statement any further. And yet - if drivers get credit cards of their own and use those to pay gas, they can put off their gas payments for up to a month. It's not necessarily a good solution, but to try and tie this debate back to cashflow timing is very strange, especially in light of all the methods both sides have to adjust those timings and the fact that, as illustrated, it doesn't matter anyway.

Actually, that's not entirely true. It's probably much easier to hide cash payments from the IRS.

Last night, trying to catch a cab in a Manhattan rain, it occured to me that it really shouldn't be so surprising that so many of our systems fail exactly when we need them most. I was standing blocks from the nearest subway or bus and had no umbrella; I needed a cab to get home. But so did everyone else. Result: no cabs available for anyone. The real surprise is that we expect those systems to work at such times.

The number of available cabs is a (somewhat) efficiently determined quantity. At any moment, the supply of taxis matches the demand for private transportation. But when demand spikes, there is a scarcity of cabs. The trouble is that these demand spikes only occur infrequently - they are tail events. But more than that, they are wrong-way risk events. Not only is demand higher in terms of quantity demanded, but demand is also escalated in terms of how much people need cabs (think of this as the minimum price a person is willing to pay having gone up as well). Thus, the time I need a cab the most is the exact time it will be hardest to find.

It is in many ways a futile exercise to apply central measures to tail events. The real surprise is that we expect those systems to work.

Any system which is based on the center of the distribution will fail to give satisfactory results when tail outcomes appear, such as rainy nights or the United Nations convening. On the other hand, any system which is designed to operate smoothly 90% or 95% of the time must, by definition, be based on the center of the distribution. To deal properly with tail events, a system must either a) ignore the center or b) undergo a regime shift to deal with the tail (which is just a sneaky way of saying it should be two systems, one for the center and one for the tail). Taxis operate based on the center; therefore the system breaks down in the tail. However, tail events for cabs are generally forecastable - I would think that supply could adjust relatively easily and undergo the necessary regime change. If I drove a cab and I saw it was raining, you bet I'd be out looking for extra fares.

The BLS birth/death model which I discussed yesterday is another example. Designed to represent the manner in which businesses change over time, it must be built in accordance with normal (i.e. central) behavior. It therefore fails anytime the economy experiences sharp growth or contraction - tail events.

And how about value at risk? The analogy should be obvious by now. On the one hand, the distributional assumptions for VaR come from historically observed data. Thus, the system is based on the center. On the same hand, VaR doesn't tell us anything about the tail event - it merely defines what constitutes such an outcome. By the time you experience the tail, you're beyond VaR's ability to help you. So VaR, for all its comfort and reassurance, is another system which fails to account for the periods it will be needed most. Better risk measures deal exclusively with the tail, which is to say the outcomes they are being used to evaluate.

Anyway, I got a cab after a few minutes. It was my yellow swan.

When it rains it pours - via Jason Kottke, another take on a hypothetical Big Apple Burj Dubai:

The Burj Dubai is by far the tallest building in the world, despite being unfinished. However, I find it difficult to grasp just how massive it is. A recent Gizmodo post came close to capturing its immense height (see the image below) but still, a true sense of scale is absent.

The trouble is that I have no concept of relative height when I'm looking at those images; yes, the tower looms over other buildings that I know deep down would be considered immense in their own right, but they might as well be townhouses. They provide no context because I have nothing tangible to compare them to. Meanwhile, silhouette comparisons such as this one convince me of the Burj Dubai's height, but do little to impress any grand sense of scale:

What I need is a comparison that marries the abstraction of the silhouette with the concrete grounding of the actual photos. Once again quoting Tom Lehrer, I have a modest example here...

Turning to Google Earth, I mocked up the views from two popular Manhattan observation decks - the Empire State Building and the Rock (that's Rockefeller Center for the non-30 Rock fans New Yorkers among you). Then, I raised the viewpoint to 2,690 feet - the height of the Burj Dubai's hypothetical observation deck. The result is an impossible view of Manhattan which instantly captures the building's enormous scale by putting its height in a familiar context. If you are unfamiliar with the New York cityscape, then these examples may be as abstract as the actual Dubai pictures are to me; however this is an experiment well worth repeating in your own urban backyard.

Note: Clicking the following images will launch an image gallery in a lightbox. The first image will show the view from an existing New York observation deck. Clicking the right side of that image will load the next image, which shows the same view from the top of the Burj Dubai. Click outside the lightbox to close it. Note that all the images below will load, so you can click through all five viewpoints without leaving the lightbox.

Note also: The effect is much more dramatic in Google Earth, which supplies smooth transitions between the viewpoints - like taking an elevator up the spire. But I'm having trouble embedding the 3D view here, so I hope these images suffice...

First up, the view from the ESB looking north toward Central Park. The real view is impressive but the Burj Dubai can practically see upstate:

Next, a similar view - the ESB looking northeast into midtown and across the East River. The Burj Dubai view makes the surrounding buildings look tiny:

Another view familiar to tourists - the ESB looking south toward the Financial District. From the Burj Dubai, you could see clear across New York Harbor and out into the Atlantic:

Turning now to the Rock, here's a similar view to the south, including the Empire State Building and Chrysler Building. The Burj Dubai towers over these New York giants:

Finally, here's another view north, this time from the Top of the Rock. The difference is unbelievable:

I hope that these visual comparisons give some greater meaning to how incredibly tall the Burj Dubai is by supplying a familiar context for its height. In a final push for perspective, we are all familiar with this iconic view of downtown Manhattan:

Typically, a helicopter would be used to capture an image from such height. But in this case - you guessed it - all you'd have to do is take the elevator. Yurtle the Turtle had nothing on this!

A very nice graphic is making the rounds (though I believe it originated in a 2007 issue of Time Magazine) which shows Manhattan's population density by day and by night. The difference is striking:

Happily, the density bars mimic the placement of Manhattan's skyscrapers - this follows because obviously the tallest buildings support the highest population density. What's striking, however, is the breakdown - commercial buildings are indicated in the "day" graph and residential buildings in the "night" graph.

This means the building categories can be identified merely by time rather than any other dimension. Ordinarily, asking someone to pick out whether a building was primarily commercial or residential would probably involve a study of tenants or usage. Here we see that merely counting the daily/nightly inhabitants suffices: a clever use of data to avoid an otherwise grueling task.

Aside, the density heights have clearly been scaled for dramatic purposes. This means the two panes are not intuitively comparable; for example the daytime population seems much higher than the nighttime. While there's certainly more people working in the city than living in it, the exponential scaling makes it difficult to see just big a difference there is.

(via Kenny Herman)

The Central Park Zoo has opened a new snow leopard exhibit.

The NYTimes has a very interesting story on another side of New York.  As they put it, "There’s the tangible New York of concrete and smog, and there’s what the film historian James Sanders has called the 'mythic New York,' the dreamy celluloid landscape of a thousand crisscrossing fictions." The latter is the city with which millions of moviegoers have become familiar over the last century through the use (and overuse) of stock footage.

A towering office building gleams through the clouds. Fireworks crackle over the Hudson River. Dilapidated tenements throw chiseled shadows over a vacant courtyard on the Lower East Side. A wind-tossed calendar dissolves to the exterior of a brownstone. These are the movie equivalent of a comic book narrator saying, “Meanwhile” or “Later that day” at the top of the panel.

The cover of The Economist features a fantastic take on a classic New Yorker cover (click to zoom):

And here's the original, for comparison -- this is one of my favorite illustrations (again, click to zoom):

Via Strange Maps.

Four shots from the last big snow (click to enlarge):

Thought this photoessay was great, from today's NYT.

The resume of Captain Chesley B. "Sully" Sullenberger III of US Airways flight 1549 has been put online.  It is amazing. (PDF link)

US Airways flight 1549 crash landed in the Hudson this afternoon not far from our office. I think "crash landed" is a much better way of describing the event than the headlines you may have seen -- "Jet Crashes in Hudson River", "Jet Ditches in Hudson River", which sound considerably more violent. The pilot did a great job putting the plane down safely.

As it was all going down (pun most certainly not intended), I received this photo, somehow:

Crazy.

Also, I'm not sure if people realize how close this thing was to Manhattan. They are describing it as landing even with 57th street. From the NY Times (as it drifted downtown):