Econometric reflections from Analytics 2014

This post will violate the “what happens in Vegas stays in Vegas” rule, because last week I had the pleasure of attending and participating in the Analytics 2014 event there and want to share some of what I heard for those who couldn’t attend. I was joined by over 1,000 attendees and colleagues as we gathered to share best practices in the fields of statistics, econometrics, forecasting, text analytics, optimization, data mining, and more. My talk, demo theatre presentation, and exhibit hall hours gave me the opportunity to meet many interesting people, so here are some of my thoughts based on what I heard and learned.

Jan Chvosta and I presented, “Why Econometrics Should Be in Your Analytics Toolkit: Applications of Causal Inference” (presentation available here) to approximately 100 attendees and sincerely appreciated the feedback we received. Of note to me was just how many audience members approached us afterward and said that “causal interpretation” is what they strive for with their predictive modeling. From marketing mix models to CCAR stress testing to price elasticity estimates, I saw many nodding heads when we talked the importance of interpretation in these models. To twist the words of Nobel Laureate Robert Lucas, “once you start thinking about causality, it is hard to think about anything else.” It appears to me that there are still many people interested in the meaning of models in this world of “big data” and “machine learning.”

I was able to attend Michele Trovero’s talk on using SAS/ETS® tools to estimate linear and non-linear state-space models. Michele showed several example and benefits of the existing SSM procedure as well as giving a look ahead to the upcoming ESMX procedure. Once released, this procedure will allow new exponential smoothing models to be estimated in SAS as well as provide a statistical treatment to structural time series models based on exponential smoothing. Use this link to find his slides or listen to this interview to hear Michele describe some of these tools.

At the econometrics booth I was asked about many different topics. We talked about state-space models (PROC SSM), time series data management with PROC TIMEDATA and the use of multinomial discrete choice models for price elasticity measurement (PROC MDC). However, far and away the dominating topic was the new CCAR compliance regulations. For those unfamiliar with these regulations, it is a requirement that bank stress tests incorporate macroeconomic scenarios into the forecasts of their financial performance with the goal being to ensure bank solvency under adverse economic events. One of the difficulties of this problem is that the objective of the analysis is no longer strictly a predictive modeling problem. New policies dictate that certain variables must be included and that these effects should behave consistently with economic theory. No fewer than ten banks independently asked about modeling techniques to satisfy these regulations. It was quite interesting, because each bank had a different method of solving the problem. Some banks with access to micro data chose to model probabilities of default based on each asset. Other banks without access to this data have opted for a time-series based approach. For some time now, I have been working on formalizing these methods and I will present a paper on the subject both at the Conference on Statistical Practice in February as well as SAS Global Forum in April.

It was my pleasure to lead a roundtable discussion about statistical and econometric modeling in health insurance. It was a packed table and I apologize if you were turned away. Many of the topics discussed during our causality talk were echoed during the roundtable, most notably non-random assignment of certain interventions. In fact, one large health insurer spoke about the early returns of the Affordable Care Act with respect to substitution effects, or lack thereof, from emergency room usage to traditional clinics. This may suggest that the population now covered by new rules remains unlikely to shift their healthcare usage from high-cost emergency room visits to less costly outpatient facilities.

Finally I would like to thank the 15 or so attendees at my 7:30am demo theatre presentation on the new items in SAS/ETS®. These were brave and dedicated souls to be such early risers in Vegas! I always enjoy the chance to evangelize our new tools. We spent a great deal of time talking about PROC HPCDM, a tool for simulating aggregate loss distributions in insurance and banking. People were interested because there now is a very computationally efficient way to simulate aggregate losses subject to business rules about deductibles and limits. We also talked about new methods of estimating limited dependent models in the presence of endogenous regressors. There was an interesting question about tools for spatial econometric models, which isn't part of the current portfolio but will definitely be part of future presentations.

Ken Sanford being interviewed on the demo floor at Analytics 2014 (click on the photo for the interview)