Handbook on Measurement, Assessment, and Evaluation in Higher Education

“The Handbook on Measurement, Assessment and Evaluation in Higher Education is coming out at a precarious and important moment in the history of American higher education―and, as a result, will become a critical resource to higher education leaders and assessment and teaching and learning practitioners. The fields of assessment, evaluation, and educational measurement continue to mature and, as a result, if used wisely, can dramatically improve educational outcomes. At the same time, higher education institutions are under enormous pressure from outside constituents to demonstrate their value to their students and to taxpayers and members of the public. The analysis and advice offered in this handbook will be extremely valuable as educational leaders work to respond to calls for greater accountability and work to improve their institution’s ability to use assessment data to improve programs.”

-Debra Humphreys, Vice President for Communications and Public Affairs, Association of American Colleges and Universities

“This Handbook provides a comprehensive and practical synthesis of the methods used to measure and improve higher education at every level. Pressures for accountability and accreditation have forced colleges and universities to devote greater analytical attention to measuring goal attainment, evaluating academic programs and student services, and assessing student learning outcomes. Drawing upon an impressive array of contributors from diverse disciplinary perspectives, Secolsky and Denison have assembled chapters that, in the aggregate, represent the best of what we know about measuring, assessing, and evaluating our institutions, programs, and students. Thus, the volume deserves frequent reference and space on the book shelf of every researcher, campus administrator, faculty member, and higher education policy maker.”

-J. Fredericks Volkwein, Professor Emeritus of Education, Pennsylvania State University

“This is a timely and important book. At a moment in our history when public problems are commonly reduced to oversimplifications that make them harder, not easier, to resolve, this brave new text by Secolsky and Denison presents — in a highly accessible way — the full complement of factors in higher education that need to be considered in evaluating student learning outcomes. The authors have produced an intelligent, complete, authoritative, realistic and useful piece of work that may well be a model for evaluating complex social issues generally.”

-Eleanor Chelimsky, Former Director of Program Evaluation, U.S. Government Accountability Office

“A working knowledge of measuring collegiate learning and evaluating academic effectiveness is as indispensable for today’s university administrator as reading a budget. For the scholar of English literature or the chemist thrust into the role of Dean, accreditation site visitor, or assessment committee chair, this volume delivers the basics of this vast field in a manner that is easy to grasp and apply. Keep it handy.”

-Peter Ewell, Vice President of the National Center For Higher Education Management Systems

Heather Kelly, University of Delaware, (Areas of expertise: Institutional Research, Assessment, Research Studies

I recommend this book for publication. If designed as a text, would you use the book for your courses? Why or why not? This is my overall recommendation. I hope all of the reviewers’ observations and suggestions will be taken seriously and communicated to individual submitting proposal. I would use this book as a textbook for a course if it contained a detailed section on Assessment and a Case Study approach.

For which course(s) would it be most appropriate? (*We would appreciate your providing specific course names, numbers and college /university examples). University of Delaware – Measurement Applications in Education EDUC605, Educational Research Procedures EDUC607, Data Analysis for Educ Decision Making EDUC846, Qualitative Research in Educational Settings EDUC850, Critical and Interpretive Methods in Education Research EDUC852, Advanced Qualitative Research Methods EDUC858, Educational Measurement Theory EDUC865, Advanced Educational Measurement EDUC872

Who do you consider to be the primary audience for the book? Statisticians and Master’s and Doctoral programs with a FOCUS on Measurement and Evaluation in Higher Education and NOT necessarily Higher Education Administration. The secondary audience? Institutional Researchers.

Please give us an indication of the size of the market, if possible, and of organizations/institutions which would be interested in the book

Colleges and Universities, regional and national organizations such Association for Institutional Research (AIR) and Association for the Study of Higher Education (ASHE).

Do you find any of the features of the text particularly appealing? Yes, text that references questions and decisions that need to be made in higher education administration and how Naturalistic Evaluation can be used to support and facilitate the decision-making process are very appealing. Is the book based on any assumptions with which you agree? Disagree? Please explain. Agree with overall approach which is very similar to elements of qualitative research.

Is the material timely? What would be the book’s potential shelf life? Timely yes. Potential shelf life is long as statistical theories and methodologies remain constant while applications may change.

Section is missing. Are there any topics you feel that should be relocated, removed, or added? First section “Improving College Effectiveness…” should be relocated to the end of the book. The theory sections should appear first, followed by application sections. I believe the first section “Improving College Effectiveness…” is truly the most relevant section to those in higher education administration and especially Institutional Research. Chapter 1 should focus on COURSE Assessment versus Classroom Assessment. Chapter 9 does not belong in that section and would fit better in an Assessment Section. An Assessment Section could include chapters 9, 16, 22, 26, and 27. The final section “Reporting Results…” needs to include Assessment in the title and content. I think this last section is one of the more important components of the proposed book and should be more developed than a total of two chapters. This section should tie together the practical application of the previous sections and discuss how to communicate and use measurement, evaluation, and assessment results. This section is really the driving force and what will really lead to increased institutional effectiveness.

Does it demonstrate thorough and up to date knowledge and relevance to research and theory in this are? Yes. Does it reflect scholarship? Yes.

How would you summarize the main arguments and conclusions? The Editor needs to be sure that there is a balance between theory and application and all of the application is truly relevant to higher education administration.

Jan Arminio, Shippensburg University (areas of expertise: Qualitative research, assessment in student affairs)

I recommend this book for publication.

I do not teach doctoral students so I would not adapt this for a course but I would purchase it for my own professional development.

Who do you consider to be the primary audience for the book? The secondary audience?

Primary audience: Doctoral students as a textbook

Secondary audience: Assessment and institutional research professionals for professional development

I also appreciate the emphasis on application rather than the technical. I also like that it is higher education specific. I had to learn this through the context of k-12 and I have never worked a day in my life in k-12.

Do the author(s)/editor(s) appear well qualified to write/edit the book? The author of the proposal has not clarified his expertise in writing about qualitative methodologies. Too often I have read texts by authors who are very well versed in quantitative methodologies and then “add on” chapters on qualitative work in an attempt make it appear that the text is comprehensive. In what ways has he been formally trained in qualitative evaluation methodologies?

Also, I was hoping to read about his role in assessing learning outcomes. My experiences with staff in Institutional Research are that they collect data for state and federal reporting. Having more explicit information on his role in assessing outcomes and learning outcomes in particular would be helpful.

Regarding measurement and evaluation I would concur that the editor appears very qualified to embark upon this project.

In my doctoral program at the University of Maryland, College Park I took two courses in educational measurement. I think this text would have been a fine text for the first course in that series. Currently I teach a second research methods course for masters students and I believe this text would be too complicated to them. I also compared this proposal to the text my Department currently uses in our assessment course, and again this proposed text is far more complex than what we currently use.

As the accountability movement in higher education increases and as resources public resources for public education shrinks, the need for comprehensive assessment texts will certainly increase for assessment professionals and for preparing graduate students. The author of this proposal states that most institutions have an institutional research staff member. I believe that soon most institutions will also have an assessment staff member.

Charles Secolsky is Director of Institutional Research and Planning at County College of Morris, New Jersey.

D. Brian Denison is Institutional Research Analyst at Champlain Regional College, Quebec, Canada.