Education Commons^™

Section I- The Cognitive Psychometric Connection

The Influence of Cognitive Psychology on Testing

Section I- The Cognitive Psychometric Connection

Section Ii- Cognitive Approaches To Psychometric Issues: Applications

The Influence of Cognitive Psychology on Testing

Section II- COGNITIVE APPROACHES TO PSYCHOMETRIC ISSUES: APPLICATIONS

Section Iii- Methodological Issues

The Influence of Cognitive Psychology on Testing

Section III- METHODOLOGICAL ISSUES

Title Page And Contents- The Influence Of Cognitive Psychology On Testing, Jane Close Conoley, Royce R. Ronning, John A. Glover, Joseph C. Witt

The Influence of Cognitive Psychology on Testing

The Influence of Cognitive Psychology on Testing

Contents

Foreword ix

1. Introduction: The Implications of Cognitive Psychology for Testing...........1

PART I: THE COGNITIVE-PSYCHOMETRIC CONNECTION

2. Science, Technology, and Intelligence...................11

3. Toward a Cognitive Theory for the Measurement of Achievement ...................41

4. The g Beyond Factor Analysis.........................87

PART II: COGNITIVE APPROACHES TO PSYCHOMETRIC ISSUES: APPLICATIONS

5. The Assessment of Cognitive Factors in Academic Abilities...................145

6. Theoretical Implications from Protocol Analysis on Testing and Measurement...191

PART III: METHODOLOGICAL ISSUES

7. Structure and Process in Cognitive Psychology Using Multidimensional Scaling and Related Techniques..................229

8. New Perspectives in the Analysis of Abilities.....................267

Author Index …

Foreword- The Influence Of Cognitive Psychology On Testing, James V. Mitchell Jr.

The Influence of Cognitive Psychology on Testing

For over 40 years Oscar K. Buros was Director of the Buros Institute of Mental Measurements and Editor of the Mental Measurements Yearbooks. He was a crusader, and he devoted his entire career to his crusade. He was a crusader for better tests and the more effective selection and use of tests, and he used the Mental Measurements Yearbooks as the principal instrument in this crusade. Buros passed away in 1978, and his widow, Luella Buros, worked tirelessly to find a new home for the Institute. As a result of her efforts the Institute was relocated at the University …

1. Introduction: The Implications Of Cognitive Psychology For Testing, Royce R. Ronning, Jane C. Conoley, John G. Glover

The Influence of Cognitive Psychology on Testing

The 1985 Buros-Nebraska Symposium was developed to address the broad issue of the influence of cognitive psychology on testing and measurement. In the planning process, four topics were formulated that we asked contributors to address. The following four issues provided the focus for the Symposium and hence for the present volume. We explore:

1. Cognitive psychology as a basis for questioning some of our assumptions about the nature of mental abilities;
2. The influence of cognitive psychology on test development;
3. Cognitive psychology influences on test validity;
4. Cognitive psychology as a means to provide a linkage between testing and …

2. Science, Technology, And Intelligence, Earl Hunt

The Influence of Cognitive Psychology on Testing

The intelligence test has been cited as psychology's most important technological contribution to society. Whether this is good or ill can be debated (Eysenck, 1979; Gould, 1981; Herrnstein, 1971; Kamin, 1974). Certain facts are not really subject to debate . Psychologists can and have developed "standardized interviews" that, on a population basis, provide a cost effective technique for personnel classification in industrial, military, and some government settings. However, the tests are very far from perfect indicators. Validity coefficients between tests and performance ratings typically range in the .3 to .5 range (i.e. , from 10 to 25% of the variance …

6. Theoretical Implications From Protocol Analysis On Testing And Measurement, K. Anders Ericsson

The Influence of Cognitive Psychology on Testing

One of the goals of psychology has always been to describe, understand, and measure individual differences. The diversity of human behavior makes it particularly challenging to seek to identify general and stable underlying elements that correspond to systematic individual differences . A major problem in the efforts to identify such elements is that the elements cannot be observed directly. The primary method has been to use the current psychological theory to develop procedures to measure such hypothetical elements. In this chapter I present a new theoretic framework, based on verbal reports from subjects, for identifying and measuring individual differences. I …

7. Structure And Process In Cognitive Psychology Using Multidimensional Scaling And Related Techniques, Edward J. Shoben, Brian H. Ross

The Influence of Cognitive Psychology on Testing

INTRODUCTION

The goal of cognitive psychology is to provide a general understanding of human cognitive processes through the development of general, formal models of cognition. Although it is clearly true that some areas (such as memory) have been more highly developed than others, it is undeniable that cognitive psychology has witnessed a proliferation of models in the past decade. Perhaps researchers are finding it increasingly difficult to discriminate among competing memory models because the constraints are so weak. One possibility that will be explored in this chapter is the prospect of using multidimensional scaling (MDS) and related procedures as a …

3. Toward A Cognitive Theory For The Measu Rement Of Achievement, Robert Glaser, Alan Lesgold, Susanne Lajoie

The Influence of Cognitive Psychology on Testing

INTRODUCTION

Given the demands for higher levels of learning in our schools and the press for education in the skilled trades, the professions, and the sciences, we must develop more powerful and specific methods for assessing achievement. We need forms of assessment that educators can use to improve educational practice and to diagnose individual progress by monitoring the outcomes of learning and training. Compared to the well-developed technology for aptitude measurement and selection testing, however, the measurement of achievement and diagnosis of learning problems is underdeveloped. This is because the correlational models that support prediction are insufficient for the task …

4. The G Beyond Factor Analysis, Arthur R. Jensen

The Influence of Cognitive Psychology on Testing

The problem of g, essentially , concerns two very fundamental questions: (1) Why are scores on various mental ability tests positively correlated? and (2) Why do people differ in performance on such tests?

SOME DEFINITIONS

To insure that we are talking the same language, we must review a few definitions. Clarity, explicitness, and avoidance of excess meaning or connotative overtones are virtues of a definition. Aside from these properties, a definition per se affords nothing to argue about. It has nothing to do with truth or reality; it is a formality needed for communication.

A mental ability test consists …

5. The Assessment Of Cognitive Factors In Academic Abilities, Stephen L. Benton, Kenneth A. Kiewra

The Influence of Cognitive Psychology on Testing

Nearly 30 years ago, Lee Cronbach (1957) distinguished between the two disciplines of correlational psychology, which investigated naturally occurring individual variance in behavior, and experimental psychology, which examined the effectiveness of certain treatments on behavior. Essentially, correlational psychology examined individual differences using factor analytic techniques; whereas experimental psychology attempted to eliminate individual differences using appropriate interventions. Cronbach believed that these two disciplines should join together to promote aptitude-treatment interaction (A Tl) research that would identify effective treatments for certain types of individuals. With this combined approach, different treatments could be prescribed for skilled and less skilled individuals.

The A Tl …

Subject Index- The Influence Of Cognitive Psychology On Testing

The Influence of Cognitive Psychology on Testing

Subject Index- 10 pages

A-Z

A

Ability, see also specific type classes of, Cattell-Horn model and, 15 definition of, 268- 274 exceptional, acquired skill vs., 216-221 fluid and crystallized, correlation of, 96- 97 measurement of, 87-88, 267- 283 cognitive ability factors and, 280- 282 definition s and, 268-274 person characteristic function and, 275-279 task difficulty and, 279-280
Academic abilities, 145-183 cognitive strategies and, 150-152 control processes and, 149-150 declarative knowledge and, 146- 148 factors in, 146 in mathematics, 175-179 metacognition and, 152- 153 procedural knowledge and, 148-149 in reading, 153-162, see also Reading in science, 179- 182 in writing, 163-175, …

8. New Perspectives In The Analysis Of Abilities, John B. Carroll

The Influence of Cognitive Psychology on Testing

INTRODUCTION

One can understandably be skeptical when a "new perspective" is offered on a topic that has been under scientific examination for a very long time. I am not sure that I have any truly new perspectives , but I entertain the notion that my perspectives have the kind of novelty that will last long enough to permit taking a fresh look at some very old problems and getting new insights into their solution . I'm concerned with several such problems: First, what is an " ability"? How can an ability be defined? This is a problem that I believe …

Author Index- The Influence Of Cognitive Psychology On Testing

The Influence of Cognitive Psychology on Testing

Author Index- 8 pages

A-Z

A

Abrahamsen, A. A., 234, 236, 237, 262, 265
Ackerman, P. L., 29, 37
Agrawal, N., 103, 136
Ahern, S., 121, 136
Ananda, S. M., 135, 136
Anastasi, A., 2, 8, 176, 184
Anderson, J. A., 19,38, 256, 263
Anderson, J. H., 44, 83
Anderson, J. R., 31, 36, 37, 44, 82, 154, 184, 195, 223, 230, 256, 263
Anderson, R., 54, 82
Arabie, P., 230, 231, 232, 241, 242, 245, 246, 247, 251, 253, 263, 264, 266
Arnkoff, D. B., 172, 184
Arnold, J. B., 234, 253, 262, 263
Atkinson, R. c., 135, 136, …

Complete Work- The Influence Of Cognitive Psychology On Testing, Royce R. Ronning, John A. Glover, Jane C. Conoley, Joseph C. Witt

The Influence of Cognitive Psychology on Testing

The Influence of Cognitive Psychology on Testing

Contents

Foreword ........................ix

1. Introduction: The Implications of Cognitive Psychology for Testing...........1

PART I: THE COGNITIVE-PSYCHOMETRIC CONNECTION

2. Science, Technology, and Intelligence...................11

3. Toward a Cognitive Theory for the Measurement of Achievement ...................41

4. The g Beyond Factor Analysis.........................87

PART II: COGNITIVE APPROACHES TO PSYCHOMETRIC ISSUES: APPLICATIONS

5. The Assessment of Cognitive Factors in Academic Abilities...................145

6. Theoretical Implications from Protocol Analysis on Testing and Measurement...191

PART III: METHODOLOGICAL ISSUES

7. Structure and Process in Cognitive Psychology Using Multidimensional Scaling and Related Techniques..................229

8. New Perspectives in the Analysis of Abilities.....................267

Author Index …

Title Pages And Table Of Contents- Social And Technical Issues In Testing Implications For Test Construction And Usage, Stephen N. Elliott, James V. Mitchell Jr., Barbara S. Plake

Social and Technical Issues in Testing: Implications for Test Construction and Usage

Contents

Preface

1. Filling the Gaps Between Test Outcomes and Usage: An Introduction Barbara S. Plake Purpose of the Volume Overview of the Chapters Conclusions References

PART I SOCIAL AND TECHNICAL INFLUENCES

2. Struggles and Possibilities: The Use of Tests in Decision Making Ellis Batten Page A Double Standard The Value of Testing Ideological and Scientific Issues Decision Making Test Scores and Deeper Values Production Functions and Causal Research Conclusions References

3. What Cognitive Psychology Can (and Cannot) Do for Test Development Robert J. Sternberg What is Cognitive Psychology? Cognitive Psychology and Test Construction Cognitive Psychology and Test Validation Cognitive …

Preface: Social And Technical Issues In Testing: Implications For Test Construction And Usage, Stephen N. Elliott, James V. Mitchell, Barbara S. Plake

Social and Technical Issues in Testing: Implications for Test Construction and Usage

Tests are constructed and used to facilitate assessment and understanding of human beings in all their multifaceted complexity. Hence, testing by its very nature is both a scientific and a social endeavor.

The interplay between testing and society has resulted in both praise and criticism from concerned citizens, psychologists, educators, and numerous other professional and consumer groups. For over 40 years, Oscar K. Buros, as Director of The Institute of Mental Measurements and Editor of the Mental Measurements Yearbooks. contributed immensely to this interplay between testing practices and societal issues. On March 19, 1978, Oscar Buros died. Luella Buros …

Author Index- Social And Technical Issues In Testing: Implications For Test Construction And Usage

Social and Technical Issues in Testing: Implications for Test Construction and Usage

Author Index pp. 173-178

A-Z (6 pages)

Numbers in italics indicate pages with complete bibliographic information.

A
Acker, S. R., 65 , 86
Ackoff, R. L. , 21, 35
Adair, F. L., 122, 125
Adkins, D. c. , 148 , 153
Aisner, D. J ., 32, 35
Alderman, D. L. , 148 , 153
Anastas i, A., 9 1, 99, 107, 129 , 130, 132, 134, 139, 164 , 168 , 169
Anderson, G. E. , Jr. , 2 1, 35
Anderson , 1. R. , 46 , 57
Anderson, R. C. , 142, 153, 157, 169 …

1. Filling The Gaps Between Test Outcomes And Usage: An Introduction, Barbara S. Plake

Social and Technical Issues in Testing: Implications for Test Construction and Usage

Why do we have tests? What useful purposes do they serve? How can test results be used to make decisions? How can a test be proved to provide accurate and usable information? Questions such as these have been posed recently by a concerned public who have become more aware of and concerned about testing, test quality, and appropriate test usage. Their questions are challenging, legitimate queries that can and should be addressed by members of the measurement community .

Some of the questions being asked by the public are value laden, providing topics for many thoughtful but heated debates. For …

7. Aptitude And Achievement Tests: The Curious Case Of The Indestructible Strawperson, Anne Anastasi

Social and Technical Issues in Testing: Implications for Test Construction and Usage

In a talk I gave at the 1979 ETS Invitational Conference, I remarked that, if I were suddenly endowed with the appropriate occult powers, I should choose to eliminate certain words from the psychometric vocabulary. Among them were the words aptitude and achievement (Anastasi, 1980). These terms have led to nearly as much confusion, misinterpretation, and misuse of tests as has the more notorious term intelligence. Having been asked once more to discuss the same general topic in 1982, it occurred to me that I might consider why the myths that surround these terms are so persistent-and persistent they …

8. Achievement Test Items: Current Issues, Robert L. Ebel

Social and Technical Issues in Testing: Implications for Test Construction and Usage

The writer of achievement test items is confronted with two major problems, as Lindquist pointed out nearly half a century ago (Lindquist, 1936, p. 17). The first of these is the problem of what to measure. The second is how to measure it. The solution proposed for the first problem is to focus primarily on testing for knowledge and only secondarily on testing for abilities. Cognitive abilities, it is reasonable to believe, depend entirely on knowledge. Although the term knowledge, as commonly used, includes both information and understanding, the most useful kind of knowledge, the kind that will occupy our …

9. Abilities And Knowledge In Educational Achievement Testing: The Assessment Of Dynamic Cognitive Structures, Samuel Messick

Social and Technical Issues in Testing: Implications for Test Construction and Usage

This chapter confronts the question of what role cognitive abilities play or ought to play in educational achievement testing, which raises the prior question of what educational achievement tests are or ought to be. I begin by considering the nature of educational achievement as a construct in an attempt to circumscribe what achievement tests ought to be rather than by examining extant achievement tests that may be variously off target. Similar consideration is accorded cognitive ability as a construct. This distinction between constructs and the imperfect, variously contaminated tests that are purported to measure them is a critical recurrent theme …

6. Testing And The Oscar Buros Lament: From Knowledge To Implementation To Use, James V. Mitchell Jr.

Social and Technical Issues in Testing: Implications for Test Construction and Usage

The field of measurement can be conceptualized as having three different but interrelated aspects. First of all, it is a science or a body of knowledge concerned with the development of theory and methodology and with the identification and confirmation of generalizations governing interrelationships among variables appropriate to its content. Measurement theory and its application to measurement problems are important contributors here. Second, it is an applied science or technology concerned with the development of products that represent a useful application of such a science or body of knowledge. For the field of measurement, test development and validation are important …

4. The Status Of Test Validation Research, Lyle F. Schoenfeldt

Social and Technical Issues in Testing: Implications for Test Construction and Usage

More than any other area, validation research is where the "rubber meets the road" in test construction and test usage. The very term validation implies the assessment or measurement of individuals and the relationship of this assessment to some criterion of performance. The success of a test validation effort, or the lack thereof, has implications for the value of the assessment and for the utility of the procedures.

In today's environment, whether the validation is intended for employee selection, educational decisions, or personal counseling, there is an increasing probability that the outcomes of research will have legal implications. In the …

3. What Cognitive Psychology Can (And Can Not) Do For Test Development, Robert J. Sternberg

Social and Technical Issues in Testing: Implications for Test Construction and Usage

Whenever research is launched under a new paradigm for studying an old set of mental phenomena, researchers joining the new armada of explorers hope, at best, to discover new uncharted mental territories and, at worst, to provide new mental maps of previously charted territories that amend errors of the old maps. This has been I believe, the experience of cognitive psychologists studying mental abilities. Although they may not have revolutionized our map of the mind (yet), neither have they left the old maps standing. What is critical is that at least the flaws and incompleteness’s of the new methods are …

5. Social And Legal Influences On Test Development And Usage, Donald N. Bersoff

Social and Technical Issues in Testing: Implications for Test Construction and Usage

It was the Chinese over 3000 years ago, not the Americans in this century, who first used large-scale psychological testing (Dubois, 1966). But, as with many other technological developments, it was the United States that enthusiastically adopted the method (Haney, 1981). By now it is highly probable that every person in our country has been affected in some way by the administration of tests. Testing has become the means by which major decisions about people's lives are made in industry, education, hospitals, mental health clinics, and the civil service.

Tests themselves, by and large, are facially neutral. They do not …

2. Struggles And Possibilities: The Use Of Tests In Decision Making, Ellis Batten Page

Social and Technical Issues in Testing: Implications for Test Construction and Usage

What a happy occasion it is to celebrate, as we do in this volume, the establishment of a national Buros Institute of Mental Measurements, located on the campus of the University of Nebraska, in Lincoln. What a culmination of many plans, hopes, and dreams! On such an occasion, we can take a quiet pride in our profession and in the life and accomplishments of one of our colleagues and friends, Oscar Krisen Buros, who with Luella Buros is leaving to us, and our posterity, an institution of integrity to foster the science and practice of testing.

How new all this …

Subject Index- Social And Technical Issues In Testing: Implications For Test Construction And Usage

Social and Technical Issues in Testing: Implications for Test Construction and Usage

Subject Index pp. 178-180

A-Z (3 pages)

A

Achievement, 129, 130, 155
Achievement-saturated intelligence tests , 53-54
Alternative choice items, 148- 149 Examples, 150, 151
American Civil Liberties Union, 94
American College Testing Programs (ACT), 18
Applied performance testing , 147
Aptitude, 129, 130, 167
AQ (Achievement or accomplishment quotient) , 5, 130, 131

...

W, Z

Waisman Center, 14
Zenotype, 19

Complete Work- Social And Technical Issues In Testing: Implications For Test Construction And Usage, Stephen N. Elliott, James V. Mitchell, Barbara S. Plake

Social and Technical Issues in Testing: Implications for Test Construction and Usage

SOCIAL AND TECHNICAL ISSUES IN TESTING Implications for Test Construction and Usage- Complete Work

Copyright © 1984 by Lawrence Erlbaum Associates, Inc. All rights reserved. No part of this book may be reproduced in any form, by photostat, microform, retrieval system, or an y other means without the prior written permission of the publisher.

Lawrence Erlbaum Associates, Inc., Publishers
365 Broadway
Hillsdale, New Jersey 07642

Library of Congress Cataloging in Publication Data
Main entry under title:

Social and technical issues in testing.

(Buros-Nebraska symposium on measurement & testing)
Includes bibliographies and indexes.