Average Reviews:
(More customer reviews)The MIRT sub-field of psychometrics has for years labored in obscurity, due in no small part to the inability of its practitioners and students to understand each other and master each other's algorithms and models. Reckase, one of the field's leaders, takes a bold step in correcting the situation. Excellently researched, clearly written, logically presented, fair and balanced, Reckase summarizes the foundations of probabilistic unidimensional models and shows how they generalize, so that persons (test examinees) and items (test questions) can be represented as points (vectors) floating around in a multidimensional space.
This is not a book for the field practitioner or the casual researcher. It does not skip over the math, and the math is hard-core. Nonetheless, it is surprisingly readable. The reader will be pleased to find himself following the gist of Reckase's explanations without difficulty, even when the mathematical details are too much.
To appreciate this work, it is important to know why MIRT is important. Unfortunately, Reckase never tells us. We understand that MIRT is motivated by the fact that items and tests are complex, that they embody multiple dimensions, that therefore a multidimensional model is necessary. This hardly touches the surface. As the fantastic drama of the Netflix contest revealed (a recently resolved contest to win $1 m. for best predicting movie ratings), we live in a world of psychological profiling and prediction, a world populated by weird and incredible mathematical models that touch on every aspect of life -- from selecting food at Safeway, to renting movies, to profiling terrorists, to guiding teacher instructional decisions, to training computers to read and understand text and recognize the spoken word. None of that is in this book. The great divide between educational psychometrics and "data mining" or "knowledge discovery" has yet to be crossed. MIRT is the subfield within educational psychometrics that will ultimately bridge that divide.
On the theory side, Reckase does not conceal his differences with the "Rasch School" of psychometrics (of which I am a member) regarding the purpose of educational measurement and modeling, though he is obviously well-versed in Rasch models and presents them well, including their MIRT flavors. He sees the purpose of a model to be "descriptive" (to describe the data closely), whereas Rasch theorists see the purpose of a model to be "prescriptive" (to prescribe the conditions under which data yield true measures, i.e., measures that are most likely to reproduce across datasets regardless of person and item samples). The models that Reckase speaks about with the confidence of personal knowledge are "descriptive" in this sense.
Due perhaps to his preference for descriptive models, I found there were certain questions that Reckase did not seem to spend time on, questions that are huge for me:
1. How well do MIRT models handle small sample sizes?
2. How do they handle missing data, whether randomly or non-randomly missing?
3. To what degree are the person and item parameters invariant across samples? Can I cherry-pick my samples and get different parameters?
These are the sorts of questions Rasch people are always asking and where the Rasch model, properly used, has much to offer.
I also found myself looking in vain for discussion of Rasch's "specific objectivity" property as relates to MIRT, often called the "invariance" property. I learned that Reckase means something else entirely by the same word. In the Rasch world, "invariance" means that item and person parameters, and the resulting response probabilities, are invariant across samples, that persons will obtain the same relative measures regardless of what items they are administered so long as the items embody the same dimension. For Reckase, "invariance" means that the origin and orientation of the coordinate system can be moved without affecting the response probabilities. It's got nothing to do with samples. So, in the end, I still don't know what, if any, invariance properties the various MIRT models discussed in the book possess, defining "invariance" in the Rasch sense as invariance across person and item samples.
But those are my problems, not Reckase's. This book is a significant step forward in the maturation of an extraordinarily important, but little known, field.
Click Here to see more reviews about: Multidimensional Item Response Theory (Statistics for Social and Behavioral Sciences)
First thorough treatment of multidimensional item response theoryDescription of methods is supported by numerous practical examplesDescribes procedures for multidimensional computerized adaptive testing
Get 45% OFF
No comments:
Post a Comment