Revolutions

Analyzing Google's Winter Olympics Search Traffic with R

The Official Google Blog today includes an analysis of Google's search traffic related to the recently-concluded Winter Olympics, correlating various high-profile events with searches from particular countries. For example, traffic from the United States shows the expected diurnal cycle but with promintent peaks for the opening ceremony and the hockey matches featuring the USA team:

It's not specifically stated in the post, but it's a good bet that R was used to create this analysis. It's well known that R is in widespread use at Google, but the telltale sign here is the Y axis label. Google's traditionally coy about giving exact figures for search numbers, and in this chart the search volumes have been suppressed. That's easy to do in R, by passing the yaxt="n" option ("Y axis type: none") to the plot command. Here's an example, using the built-in lynx data set:

> plot(lynx,yaxt="n")

That tends to leave the Y axis label a bit too far off the axis itself though (it leaves room for number labels that are no longer there), but you can move it into place with an explicit call to the mtext function to place the label 1 line of text from the axis (and suppressing the original axis label with ylab="" when calling plot):

> plot(lynx,yaxt="n",ylab="")
> mtext("Lynx pelts",side=2,line=1)

I'd bet good money that similar code was used for these plots. Now if only that could recoup my losses from the USA-Canada hockey final...

Official Google Blog: Searching for gold during the Games

Analyzing plays in the NBA

The Bastketball Geek had a couple of in-depth analysis of professional basketball games in the US in the last month. The first analyzed the various ways a play could end (shots, out-of-bounds, etc.), and was followed up by an analysis of the ways the subsequent play might begin. Both analyses are based on Bayesian Statistics (assuming a Dirichlet distribution) from the MCMCpack package in R.

Baseball Geek: Conditioning the Distribution of Play Ending Events Given How the Play Starts

Basketball player performance, home vs away

Self-proclaimed basketball geek Ryan has collected player statistics from the 2006/2007 2007/2008 and 2008/2009 NBA seasons, broken down by home and away games. He has data like the free-throw percentage for each player at home and away games, 3-point shots landed, offensive and defensive rebound percentages and so on. (I confess I'm hazy on the meaning of a couple of the acronyms.) He uses R to explore the relationship between the home and away games, to see if there's any home advantage for any of these stats. A good first start is the scatterplots of the home vs away data. For example, here's the chart for free throw percentage (each dot is a player):

Follow the link for charts of the other stats, and links to the source data and the R code. It's an interesting data set, and would lend itself well to other visualizations and analyses, should any basketball fan and R user be so inclined.

It's great to see more sports analysis being done with R, so much that we now have enough articles for a dedicated sports section!

Basketball Geek: Player Statistics at Home vs Away

Kentucky Derby bump-chart

Last week I opined that it would be nice to see a chart of the Kentucky Derby done in R. Little did I know the challenge had already been answered! (Thanks for letting me know in the comments.)

I'd heard about Mine That Bird's come-from-behind performance, but hadn't seen the race myself. You can really see him move ahead at the 3/4 mile mark!

Visit Learning R to see how this chart was created using the ggplot2 package.

Learning R: ggplot2 Bump Chart

Visualizing a horse-race

A reader on the Statistical Modeling blog has come up with an interesting application of ggplot2 in R: visualizing the placements over time of horses in a horse-race (click to enlarge):

The code for the chart is available at the link below. This is simulated race data, but it would be cool to see this chart for the recent upset in the Kentucky Derby.

Statistical Science: A horse-race graph!

Quote of the day

A respected Columbia professor disputes the findings that trailing at halftime makes a team more likely to win. Andrew Gelman's rebuttal is over my head, but seems to boil down to different ways to interpret the same data, and he includes this instant classic of a line: "they do some ridiculous overfitting with a 5th-degree polynomial. Don't even get me started on this sort of thing." Someone had to say it, and he said it.

ESPN (!): Wednesday Bullets

Comparing baseball pitcher styles with lattice graphics

Last night's Bay Area R User's Group meeting was on the topic of Building Web Dashboards, and focused on integrating analytics built using R into web-based applications.

Mike Driscoll showed his PitchFX application, which visualizes the performance of major league baseball pitchers. I was intrigued to learn that the MLB tracks a host of statistics about each pitch thrown by each pitcher in pro baseball games: the speed of the pitch, the type of pitch (fastball, curveball, changeup, etc.) and the X-Y location where the pitch enters the batter's box. MLB have made these data available to the public, and Mike has created a neat application for visualizing it.

With PitchFX you can choose any pitcher from any team and get an instant analysis of his pitching style from the 2008 season. For example, here's the analysis for the Red Sox's Clay Buchholz:

Buchholz uses only four different pitches. The top row is a lattice chart showing the distributions of speeds for each type of pitch: fastballs are, uh, fast (80-85 mph and up); changeups are slower; sliders come at a range of speeds. But it's when you combine the pitch speed with the location that things really get interesting: that's what the bottom chart shows. Each dot is a pitch, and its location matches the location the pitch landed in the batter's box. The redder the dot, the faster the pitch (likewise, the bluer the slower); the darker the dot, the more pitches in that location. (Mike used the colorspace package to generate the range of colors in the plot.) Here you can see some interesting characteristics of Buchholz's pitching. Fastballs are high and left. Sliders come in two varieties: high above the plate but slow, or midspeed and to the right. Curveballs go low and slow, but occasionally hit the strike zone. Changeups are fairly localized, but the speed is unpredictable.

Mike also presented the technical details of how he created the PitchFX app. The charts are constructed in R, but R itself runs as a module of the Apache web server, thanks to Rapache. When you choose an analysis in the PitchFX web application the request is sent to Rapache, which in turn draws the data from a MySQL database for analysis in R. This creates a chart which in turn is sent to the web browser for display on your screen. The caching functionality of Apache means that charts are not needlessly regenerated unless the underlying data change. [Update May 10: Mike helpfully provided an architecture diagram which makes things clearer, shown below.]

It's a really well-put together application, and makes it easy and fun to compare the styles of your favourite pitchers.

Also in the same meeting, John Oram showed an outstanding app also using Rapache for visualizing environmental data in the Bay Area. More on that, tomorrow.

Dataspora: PitchFX viewer

Word of the day

Andrew Gelman introduces me to a new (well, new to me) word today: Sabermetrician.  According to Wikipedia, Sabermetrics is the analysis of baseball through objective evidence, especially baseball statistics. "Saber" comes from Society for American Baseball Research. (Oddly, Wikipedia is silent on the etymology.)

Do other sports statisticians get a specialized job title?

R, March Madness, and the Statistics of Basketball

Back in February, Mark Cuban announced on his blog that the Dallas Mavericks use a "player lineup and evaluation system" based on statistical analysis (generating some controversy, as it turned out). A New York Times Magazine article a week later reveals that the Rockets have a similar system in place. Ars Technica suggests that the analyses were heavily supplemented by data from high-tech stats packages like the open source program "R". It wouldn't be at all surprising: R has been used to analyze March Madness bracketology before, so it makes sense that it would be used to analyze the performance of individual players, too. 

Statistics and the Superbowl

Apparently, there's some kind of football game going on here in the US this weekend.  Strangely though, the ball isn't round. The playing field isn't even oval.  No, this is American Football.

Football (of all stripes) and statistical analysis have a long history together, but in each case the focus appears to be different.  For UK football (soccer), much of the research appears to be around predicting the outcome of games (usually, in order to optimize betting strategies -- in fact, I have a friend in the UK who's been quite successful modelling games and betting on the outcomes). I don't know much about statistics and Australian Rules football, but a quick google search reveals a lot of work in identifying the causes of injuries. For American Football, though, I've found a couple of articles where statistical analysis is used to determine the optimal strategy for play.

Around the time of the 2008 Superbowl, Technology Review published a detailed and interesting article describing how the New England Patriots use the analysis of the results of different plays to select strategies. Jim Orlin follows up in his blog with more detail about the kinds of analyses employed.

Relatedly, on strategy for building teams, Malcolm Gladwell had a fascinating article in the New Yorker on the wisdom (or otherwise) of recruiting star quarterbacks from college football to play in the professional league. As it turns out, performance in college football isn't a great predictor of merit in the big leagues.

So to the folks in the US and Superbowl fans elsewhere, enjoy the game tomorrow.  I'll be watching too, if only for the commercials.