Revolutions

Frank Harrell to teach Regression Modeling Strategies short course

If you're using regression models but want really hone your regression-fu this short course on Regression Modeling Strategies by Frank Harrell looks really interesting. Frank is the author of the book Regression Modeling Strategies which is my go-to reference whenever I'm doing regression of any kind in R, so it's definitely worth a trip to Nashville to if you want to learn how to make regression really work. Registration closes March 1, and the 3-day course begins March 31.

By the way, if you haven't tried them already check out Frank's rms package (formerly the Design package, a package of regression tools to accompany the book) and also his Hmisc package. Hmisc has a bunch of useful utilities (PDF) to extend the R language in various ways -- the tools for creating attractive tables in LaTeX are especially useful. 

Department of Biostatistics, Vanderbilt University: 2010 Upcoming short courses in Regression Modeling Strategies

Package Update Roundup: Dec 2009 - Jan 2010

A special double edition of the Package Update Roundup this month! This is a list of new or updated packages that were released for R in December and January, as announced on the r-packages mailing list. To include other updates on this list, please email David Smith. For a complete list of all updates on CRAN, see the CRANberries archive for December 2009 and January 2010. Follow package name links for ratings and other information on crantastic.org.

dcemriS4, a package for medical image analysis, has been released to update package dcemri.

Deducer, a cross-platform data analysis GUI, has been updated.

doMPI, a backend for the foreach package for parallel computing, has been updated. 

exact2x2, a package for exact conditional tests and confidence Intervals for 2x2 tables, was updated to allow optionally give an exact McNemar's test together with odds ratio estimates and confidence intervals.

exactci, a new package that calculates exact tests and confidence intervals for binomial and Poisson tests, has been released.

ff and bit, two packages that help manage large datasets in R by mapping them to disk instead of storing them in memory, have been updated.

fortunes, a package delivering snippets of amusing R wisdom, has been updated (and now includes more than 250 fortunes!)

GGally, a package to draw plot matrices (like pairs does, but using ggplot2), was released.

ggplot2, the comprehensive high-level graphics system for R, has been updated adding several new features.

itertools, a new package of tools for manipulating iterators, has been released. 

mecdf, a package for multivariate ECDF based models, has been released.

micEcon, micEconAids, and miscTools, three packages for microeconomic analysis and microeconomic modelling have been released (these packages were previously combined into a single micEcon package)

mspath, a new package to fit multi-state path-dependent models in discrete time, has been released.

parcor, a new package of tools to estimate the matrix of partial correlations based on different regularized regression methods, has been released.

pgfSweave, a package that improves the process of including graphs in Sweave documents, has been released.

randomSurvivalForest, a package for ensemble survival analysis based on a random forest of trees, has been updated with new model controls.

randtoolbox, a package for random number generation and testing, has been updated with new features.

RobASt, a family of packages for robust asymptotic statistics, has been updated.

Rcpp, the package interfacing R with C++, has been updated to make integration between R and C++ even easier.

sp, a package that  provides class definitions for spatial data, and utilities for spatial data handling and manipulation, has been updated to improve performance.

sqldf, a package for accessing data in data frames using SQL, has been updated.

Registration open for R/Finance 2010

Registrations are now open for the R/Finance 2010 conference, to be help April 16-17 in Chicago. Last year's meeting was a great success, and this year's looks to be just as good, with some great keynotes lined up:

• Analysis of Integrated and Co-integrated Time Series with R (Bernhard Pfaff)
• Leverage Space Portfolio Model (Ralph Vince)
• Signal Extraction (Marc Wildi0
• Applied Econometrics with R (Achim Zeileis)

There's also a great series of pre-conference tutorials lined up:

• Rcpp / RInside: Extending and Embedding R with C++ for Fun and Profit. (Dirk Eddelbuettel)
• Trading with R: Idea to Execution in 50 Minutes with IBrokers and R (Jeff Ryan) 
• Complex Portfolio Optimization with Generalized Business Objectives: Advanced Portfolio Optimization Techniques (Peter Carl & Brian Peterson)
• GPU Programming with R: An Introduction To GPU Programming with R (Josh Buckner & Mark Seligman) 

The tutorials are a steal at $50 each and sell out fast. Register for the conference and tutorials now at the link below.

(REvolution Computing is a proud sponsor of R/Finance 2010.)

R/Finance 2010: Conference and Tutorial Registration

Because it's Friday: Evolution in song

With thanks to my favourite science blog Bad Astronomy, the late great Carl Sagan sings again, this time on the topic of evolution. He's joined by autotuned Richard David Attenborough and Jane Goodall. Great stuff.

Bad Astronomy: The Unbroken Thread

Mapping the Massachusetts election upset with R, ctd

Last week we looked at an analysis done in R by the good folks at Offensive Politics, looking at the political climate surrounding the recent Senate election in Massachusetts. There were some very insightful comments (thanks, Revolutions readers!) about the design of the charts, especially in the choice of color schemes used (the originals didn't use a neutral white midpoint to represent no change). Well, Offensive Politics have taken another look at the charts, and produced some new ones which are a great improvement. With the revisions, the chart comparing the increase in Republican vote percentage from 2008 to 2010 looks like this:

 Offensive Politics notes:

From this map we can see that from 2008 to 2010 the Republican vote percentage actually increased in every single township in MA. This is kind of shocking, especially when you consider that Democrats enjoy a 15 point registration advantage statewide.

Check the link below for all the updated charts, and of course the R code to create them.

Offensive Politics: Re-mapping Massachusetts Special election results

Advanced graphics in R

A lot of attention recently has gone to the more modern (and, dare I say, sexier) graphics systems in R, ggplot2 and lattice. But there's a lot of power in the base graphics system built into core R, especially when you want control over every aspect of how the graph is laid out. Ryan Rosario has put together some slides detailing some more advanced tricks capable with the base graphics system. For example:

• How to create custom axes (for example, an axis on the right, or an axis labelled by dollars or other custom quantity)
• How to overlay multiple scatterplots (and set the scaling of the axes appropriately)
• How to create a custom legend
• How to add gridlines to your graph
• How to specify colors
• And even how to create a movie from your graph

All the details are in Ryan's slides, and yes, the R code to create the example charts is also available.

Byte Mining: Advanced Graphics in R

In case you missed it: January roundup

In case you missed them, here are some articles from last month of particular interest to R users.

This post linked to slides and video from a 30-minute "Introduction to R" talk I gave on January 28, with links to many useful R resources.

This post brought news that R's creators Robert Gentleman and Ross Ihaka have jointly won the inaugural Statistical Computing and Graphics Award from the ASA.

This post welcomed Robert Gentleman to REvolution Computing's board of directors.

This post announced the print availability of the new book from O'Reilly, R in a Nutshell. It has a harpy eagle on the cover.

This post showed how to use calendar heatmaps to analyze traffic on a FriendFeed group.

This post showed that the number of R packages is growing exponentially. The evidence comes from a paper in the first edition of the R Journal, updated with more recent data here.

This post showed R being used to evaluate biases in survey questions.

This post linked to slides and video from talks by Jeff Horner and Jeroen Ooms on "R-Powered Web Apps".

This post linked to slides from the New York R User Group "R Rosetta Stone" meeting, with tips on coming to R from SAS, Matlab, SQL, Clojure or Python.

This post linked to Bob Muenchen's table of equivalent R packages to modules in SAS and SPSS.

This post linked to a review of new features in version 0.85 of the ggplot2 package.

This post called for presenters at the useR! 2010 conference, and announced a commercial applications track.

This post linked to a step-by-step tutorial on how to make a heat map with R.

This post looked at two analyses of a surprise upset in a US Senate election done with R (one done on 64-bit Windows with the REvolution R Enterprise build).

This post gave some examples from the new RgoogleMaps package of combining satellite maps with data in R. 

This post showed off a very creative use of ggplot2 to illustrate the history of colors in a box of Crayola crayons.

Several new local R user groups are starting up in the USA: Seattle, Philadelphia, and Washington, DC.  

Other non-R-specific posts in the past month covered data visualizations in the New York Times and the Wall Street Journal, what programmers ought to know about Statistics, and (on a lighter note) visualizing gravity wells.

The R Community Calendar has also been updated.

As always, thanks for the comments and please send any suggestions to me at david@revolution-computing.com. Don't forget you can follow the blog using an RSS reader like Google Reader, or by following me on Twitter (I'm @revodavid). You can find roundups of previous months here.

Survey: Share your thoughts about predictive models with Aberdeen Group

Analyst firm Aberdeen Group is conducting research into the use of predictive models in business with a 10-minute survey. It's focused mainly on businesses that are using (or plan to use) predictive models to forecast aspects of their business and the systems they have in place (or plan to put in place) to do so. If you're using predictive models at work, or plan to, consider adding your voice to the survey. Here's the description of the survey from Aberdeen:

"Top performing organizations are increasingly using predictive models to improve their business. This research study will determine what strategies and actions companies are taking to:

• Predict customer buying behavior and grow revenue.
• Improve the product development process to pioneer new products and capture market share.
• Predict risk and detect fraudulent behavior more rapidly.
• Improve operational efficiencies, such as supply chain performance or energy consumption.

Whether you use predictive techniques or not, take this 10 minute survey and you'll be able to compare your organizations performance against its peers, learn where predictive analytics and data mining might bring benefits, and how to get started.  Everyone that completes the survey will receive a complimentary copy of the final research report.  Individual responses will be kept strictly confidential and data will only be used in aggregate. We greatly appreciate your participation in this research effort and look forward to sharing our findings with you. "

Aberdeen Group: Predictive Analytics - Niche Technology, or Must-Have Competitive Advantage?

InfoWorld: SAS and SPSS rise to R opportunity

At InfoWorld's "Open Source" blog Salvio Rodrigues found R co-inventor Robert Gentleman's appointment to the REvolution Computing board "a great impetus for me to look at R again". He notes that both SAS and SPSS have recognized the opportunity presented by R:

I suspect that SPSS and SAS made their individual decisions based on three factors. First, they likely both realized that based on the penetration of SAS and SPSS in the statistical community, neither was going away anytime soon. Second, adding R support enabled both vendors to take advantage of the community of users building extensions and new statistical methods for R. Finally, both vendors likely realized that customers have different skills and analysis needs, and as such, R would be used in conjunction with SAS and SPSS's programming languages for statistical analysis.

By the way, Robert Gentleman's appointment was also noted by Greg Laden at scienceblogs.com.

InfoWorld Open Sources: SAS and IBM/SPSS rise to the open source R opportunity

R creators win prestigious Statistical Computing and Graphics Award

The American Statistical Association recently created a new, bi-annual award to to recognize an individual or team for innovation in computing, software, or graphics that has had a great impact on statistical practice or research. The committee has just announced the winner (or in this, joint winners) of the first award: Robert Gentleman and Ross Ihaka, for their work in initiating the R Project for Statistical Computing.

It really can't be overstated how well-deserved this award is. No other project has made world-class software available to so many people while also by encouraging so many to participate in advancing the art and science of statistical computing through the medium of an open-source project. While many, many people have made significant contributions to the R project over the years, it's undeniable that Robert and Ross were the ones that got it all started. Congratulations, R&R!

(For more information about the history of the R project, this New York Times article from 2009 is a great resource. You can also read a profile of Ross Ihaka in the New Zealand Herald. Disclosure: Robert Gentleman recently joined the board of REvolution Computing.)

ASA Sections on Statistical Computing and Statistical Graphics: Statistical Computing and Graphics Award