This SAS tip comes from Clement A. Stone and Xiaowen Zhu, authors of Bayesian Analysis of Item Response Theory Models using SAS.
Item response theory (IRT) models are the models of choice for analyzing item responses from assessments in the educational, psychological, health, social, and behavioral sciences. SAS PROC MCMC can be used in all types of assessment applications to investigate how particular characteristics of items and how particular characteristics of persons affect item performance. Use of the SAS system for Bayesian analysis of IRT models has several significant advantages over other available programs: (1) It is commonly used by researchers across disciplines; (2) it provides a robust programming language that extends the capability of the program—in particular, the capability for model checking; and (3) it shows increased performance and efficiency through the use of parallel processing.
Our book Bayesian Analysis of Item Response Theory Models using SAS provides step-by-step instructions for using SAS PROC MCMC to analyze various IRT models. Working through the examples in the book or with some prior knowledge of IRT models and Bayesian methods, you can…
Estimate simple as well as complex IRT models using PROC MCMC. It is a straightforward task in PROC MCMC to implement Bayesian estimation of a variety of simple and more complex IRT models. All you need to do is express the response probability function or likelihood for your particular model, declare the model parameters, and specify prior probability distributions for these parameters. PROC MCMC may be particularly useful for applications investigating multidimensionality or heterogeneity in item responses due to, for example, differential item functioning, content related processes (shared context or word orientation), or response related processes (solution strategies, response styles, response sets).
Evaluate the estimation of the model. Because the Markov Chain Monte Carlo (MCMC) method is a simulation based approach, you should determine whether the simulated draws have converged to the target posterior distributions for model parameters. PROC MCMC includes a number of tools and statistics for evaluating the convergence of the sampling process in the posterior distributions for model parameters. These include history and autocorrelation plots as well as various diagnostic tests and statistics: Gelman-Rubin, Geweke, Heidelberger-Welch (stationary and half-width tests), Raferty Lewis, and effective sample size.
Compare competing models and evaluate model fit. In many applications, different models may be estimated that reflect competing theoretical perspectives or competing formulizations of the item and person characteristics that are modeled. PROC MCMC and the SAS system provide the tools for choosing among competing models. The Posterior Predictive Model Checking (PPMC) method is a commonly used Bayesian model checking tool and has proved useful for evaluating the fit of models. PPMC can be implemented using the robust programming language in the SAS system and a variety of different plots can also be obtained to display results.
In conclusion, PROC MCMC makes estimating and model checking of IRT models in a Bayesian paradigm more accessible to researchers, scale developers, and measurement practitioners.
We hope you find this blog informative and invite you to read a free chapter from the book here.
