Understanding Education Indicators: A Practical Primer for Research and Policy

Citation

Forsyth, P.B. (2012), "Understanding Education Indicators: A Practical Primer for Research and Policy", Journal of Educational Administration, Vol. 50 No. 1, pp. 125-126. https://doi.org/10.1108/09578231211196096

Authors Planty and Carlson are a somewhat unusual writing team. Planty is a statistician at the US Department of Justice and Carlson is a political science PhD candidate at the University of Wisconsin‐Madison. They begin their primer by describing how increased pressures for school accountability have made indicators of progress in education critically important. From there, they set as their goal to develop the reader's level of understanding of the variety in both quality and credibility factors that can be found in existing research and policy reports, using the perspective of “education indicators.”

The approach taken in this book is an important one for education social scientists because it provides a unique perspective on research. The entire research process is described, explored, and understood through the notion of indicators and measurement. Even seasoned scholars will find that the education indicators approach taken here provides a cohesive way to think about the key challenges facing scientific research in the current politicized environment. For scholars, the book fosters humility, cautiousness, and a powerful reminder that no research is perfect.

The approach taken is equally important for practitioners because is fosters a respect for systematic, scientific research, along with a healthy skepticism, especially of those who use research to sell commercial solutions to school problems. The text is written is such a way as to make the subject vitally important and interesting. The many examples connect the subject matter directly to the topical and controversial dilemmas facing public education and policy‐making.

Graduate students in education, having studied this volume, will be more critical and skilled makers and consumers of educational policy and designers of research projects. Especially through the use of concrete examples drawn from current education writing, their understanding of the variety of factors that need to be kept in mind when doing and reading research reports, will be significantly more savvy.

The book, divided into five concise chapters, has as its purpose “to provide users with a framework and tools for understanding and evaluating the quality and value of education indicators” (Planty and Carlson, 2010, p. 18). In Chapter 1, authors define specifically what they mean by indicators: “Specifically, indicators combine statistics with purpose, meaning, and context to provide useful information about a condition or phenomenon of interest” (p. 4). They also describe three purposes of indicators, namely, “ to report the status of a particular topic or policy area,” “to monitor change over time for a particular topic or policy area,” and “to project future patterns” (p. 7). Most importantly, in this chapter the framework for understanding indicator quality is introduced.

Chapter 2, “Measurement,” takes the reader through the process of constructing measures for scientific use in education. The discussion moves from problem identification, through conceptualization and operationalization, and finally on to observation. Not a psychometric treatise, the authors clearly explain notions such as “operational definition,” and other terms that frustrate newcomers to the language of research and statisticians. The chapter contains one of the cleanest and most straightforward discussions of the twin notions of validity and reliability in measurement that can be found anywhere. These are portrayed, very usefully and effectively, as the criteria for examining the quality of measurement.

Chapter 3, “Sources of Information and Error,” is in a way intimidating to the scholar and at the same time exceptionally rich in insight for others. Authors summarize the chapter with a framework for understanding indicator error structure. It consists of a series of questions. They rightly claim that “understanding the error structure is critical to understanding the substantive value of any indicator (p. 75). This chapter provides a very clear description of error, how it enters and is embedded in data and how researchers can design and analyze in ways that minimize error structure effects.

The last two chapters of the book, Chapter 4, “Statistics and Data Presentation,” and Chapter 5, “The Misuse of Statistical Indicators,” were less interesting to the reviewer, but possibly of great interest to practitioners. Chapter 4 reviews common and simple statistical notions, the concept of standard error, and also focuses extensively on these issues as they relate to student assessment reports and performance indicators. An important argument underscored in the final chapter is that indicators, and all research for that matter, are the products of a string of important choices, that is, all research is the product of compromise.

This book is aptly titled, although its purposes can be thought of as going well beyond an introduction for newcomers to the social sciences and policy studies. Its approach to research through the lens of “indicators” provokes new insights of interest to experienced researchers. Yet, the focus on indicators also serves to develop in practitioners and policy‐makers a heightened sensitivity to the differences in trustworthiness and credibility to be found in research reports and the claims of commercially distributed school reform schemes.

A great virtue of the volume is the reliance on extended examples that make the concepts and essential arguments of the book vivid. Examples carry across topics, making comparison and deeper understanding of what is somewhat complex content, comprehensible.

This book is recommended highly as a quick refresher, from a unique perspective, of topics related to measurement in the social sciences. It can also serve as an auxiliary text for courses that introduce teachers, school administrators, and policymakers to research. This is an informative and surprisingly accessible read for all who have an interest in education and policy research.