Education Commons^™

Dedication- Licensure Testing: Purposes, Procedures, And Practices, James C. Impara

Licensure Testing: Purposes, Procedures, and Practices

The impetus for this book and for my involvement in it are due principally to the efforts of three people: Jimmie Fortune, Howard Stoker, and Barbara Plake. Jimmie edited (and contributed to) an earlier book devoted to licensure testing (Fortune, J. C., & Associates, 1985, Understanding Testing in Occupational Licensing, Jossey Bass: San Francisco.). Shortly after that book was published he told me that he felt some important topics had not been included and that another book was needed. He encouraged me on several occasions to edit a new book that picked up where his left off and expand …

Preface- Licensure Testing: Purposes, Procedures, And Practices, James C. Impara

Licensure Testing: Purposes, Procedures, and Practices

This book represents a unique effort for the Buros Institute of Mental Measurements and the Buros-Nebraska Series on Measurement and Testing. All of the previous books in this series have been associated with a symposium sponsored by the Buros Institute of Mental Measurements and the University of Nebraska. This book is "free standing" in that it is an independent effort intended to fulfill a perceived need for a book, but without preceding the book with a symposium. There are few books devoted solely to the topic of licensure testing, but each state and the federal government is involved in this …

Section One: Purposes And Policy Issues

Licensure Testing: Purposes, Procedures, and Practices

No Content

Title And Contents- Licensure Testing: Purposes, Procedures, And Practices, James C. Impara, Linda L. Murphy

Licensure Testing: Purposes, Procedures, and Practices

Licensure Testing: Purposes, Procedures, And Practices

Contents

Dedication

Preface

SECTION ONE: Purposes and Policy Issues

1. What is Licensure?
Kara Schmitt

2. Legal and Professional Bases for Licensure Testing
William A. Mehrens

3. Policy Issues with Psychometric Implications
Michael Rosenfeld, Richard J. Tannenbaum, and Scott Wesley

SECTION TWO: Procedures

Overview of the Licensure Testing Process
James C. Impara

4. Practice Analysis: Building the Foundation for Validity
Joan Knapp and Lenora Knapp

5. Test Development: Systematic Item Writing and Test Construction
Anthony LaDuca, Steven Downing, and Thomas Henzel

6. Developing and Using Clinical Examinations
Jimmie Fortune and Ted Cromack

7. Basic …

Section Two: Overview Of The Procedures For Developing A Licensure Examination, James C. Impara

Licensure Testing: Purposes, Procedures, and Practices

There are a variety of strategies that may be employed in the development of a licensure examination. The following list of activities illustrates typical procedures. Depending on the needs and conditions of the particular occupation, certain variations in specific activities may take place or changes in the sequence may be appropriate. In addition to the procedures listed, many decisions will be made that may add activities. For example, the decision to use a computerized item bank or to enter into a computerized adaptive testing format will require procedures in addition to those described briefly below.

1. Conducting a job (or …

6. Developing And Using Clinical Examinations, Jimmie C. Fortune, Theodore R. Cromack

Licensure Testing: Purposes, Procedures, and Practices

INTRODUCTION

Generally, clinical examinations for licensing (sometimes called performance tests) involve the candidate completing one or more tasks (in licensing this is thought of as "services for a client") that have been selected from the supervised practice Gob analysis) of an occupation or profession. The clinical examination may exist in contexts (occupations) that do not require client interactions. Such contexts include building trades, automobile repair, accounting, etc. The tasks may range from fixing brakes, to preparing a body for burial, to wiring a house, or auditing a set of business interactions.

Other contexts require the candidate to perform services or …

7. Basic Psychometric Issues In Licensure Testing, Howard W. Stoker, James C. Impara

Licensure Testing: Purposes, Procedures, and Practices

INTRODUCTION

The number of people in the United States who carry some responsibility for the writing of examination questions and the construction of tests is unknown. In the Preface to The Construction and Use of Achievement Examinations, published by the American Council on Education in 1936, the authors indicated that the number probably exceeded a million. That number has certainly grown in the past 60 years. Questions are posed to students by teachers at all levels of education; the Armed Forces have people whose job it is to construct tests which are used in the promotion of personnel; over …

4. Practice Analysis: Building The Foundation For Validity, Joan E. Knapp, Lenora G. Knapp

Licensure Testing: Purposes, Procedures, and Practices

INTRODUCTION

A review of the literature associated with job analysis reveals two extremes of opinion as represented by the following provocative quotes:

Historically job analysis has been a relatively soporific area of industrial and organizational psychology, characterized by neither heated controversy nor prominent visibility in the research literature. (Harvey, 1991, p. 71)

Validation was once a priestly mystery, a ritual behind the scenes with the professional elite as witness and judge. Today it is a public spectacle combining the attraction of chess and mud wrestling. (Cronbach, 1988, p. 3)

Both our evaluation of practice analysis research and our professional experience …

2. Legal And Professional Bases For Licensure Testing, William A. Mehrens

Licensure Testing: Purposes, Procedures, and Practices

In this chapter the author presents the legal setting for licensure testing, I discusses the role of various professional standards and codes (i.e., the EEOC Uniform Guidelines, 1978, and the AERA/APA/NCME Standards, 1985), presents some of the pertinent rulings from several court decisions, and makes inferences about future changes in professional standards and their potential impact on licensure test development.

There necessarily is some minor overlap with the material in this chapter and some other chapters in this book. There is a brief discussion of the differences between licensure, certification. and employment testing and how those differences relate …

5. Systematic Item Writing And Test Construction, Anthony Laduca, Steven M. Downing, Thomas R. Henzel

Licensure Testing: Purposes, Procedures, and Practices

Standardized objective testing remains the most popular mode of licensure testing. Even where other types of tests are incorporated, it is often the case that they are provided as complimentary to standardized, multiple-choice (MC) tests. Moreover, scoring theories and standard-setting procedures have been developed over the years in the context of standardized MC testing. At the same time, critics have pointed to limitations of contemporary MC testing practices, including lack of fidelity to real-life challenges and emphasis on recall of factual minutiae. In our view, testing professionals should make conscientious attempts to modify test development procedures so as to address …

3. Policy Issues With Psychometric Implications, Michael Rosenfeld, Richard F. J. Tannenbaum, Scott Wesley

Licensure Testing: Purposes, Procedures, and Practices

Testing candidates with disabilities, testing repeaters, and coaching involve issues of fairness, the validity of the inferences made from test scores, and protection of the public. Licensing boards must develop policies to deal with each of these issues. It is interesting to note that although all three are of concern to licensing agencies, little of the research on these topics has been conducted in licensure settings. This chapter discusses the results of research conducted on each topic, considers the psychometric implications for policy of each, and suggests steps licensing boards can take when formulating policy.

TESTING CANDIDATES WITH DISABILITIES IN …

1. What Is Licensure, Kara Schmitt

Licensure Testing: Purposes, Procedures, and Practices

When most individuals hear the terms license and licensure, their first reaction is that these are easily understood and relatively simple words. Everyone knows what these terms mean. Or do they?

What is licensure? It is a multifaceted, complex governmental system of regulation with the stated purpose being public protection. According to Webster's dictionary (Guralnik, 1976), a license is defined as "a formal permission to do something: esp., authorization by law to do some specified thing (license to marry, practice medicine, hunt, etc.)." The term Licensure is then defined to mean "the act or practice of granting licenses, …

Section Three Emerging Practices

Licensure Testing: Purposes, Procedures, and Practices

No abstract provided.

8. Item Banking, Betty A. Bergstrom, Richard C. Gershon

Licensure Testing: Purposes, Procedures, and Practices

Item banks developed by licensure agencies range from a collection of items stored on index cards to highly sophisticated electronic databases. Regardless of the storage mechanism, most banks contain items that have been organized and referenced according to procedures established by the licensure agency. This chapter outlines useful practices for building and maintaining a computerized item bank. We address storage of item text, graphics, and statistical history. We deal with the creation of paper-and-pencil and computerized tests from an item bank and the use of Item Response Theory (lRT) to calibrate and equate item banks. New directions in item banking …

Subject Index

Licensure Testing: Purposes, Procedures, and Practices

Subject Index listed A-Z (8 pages)

A

a parameter: 299-300, 303, 305-306, 314
absolute standard procedure: 329
absolute standards: 222-223
accuracy of the inference: 150
Adaptive Mastery Testing (AMT): 298,305
advisory committee: 100-101 , 103-105,111- 112
AERA/APA/NCME Standards: 33,37-43, 53, 55 , 66, 68, 72, 84, 94, 11 4, 118, 122, 137-138,168, 170- 171,179,185, 236, 248, 254-255, 26 1, 281
all -in-one requirement: 155
alternate-choice: 120
alternate-choice multiple choice: 324
Americans with Disabilities Act of 1990 (ADA): 22-23, 43, 54-55, 60-6 1, 63, 65, 68-70, 297
amplified objective: 129,147
amplified objective method: 129
analysis of variance (ANaYA): 153
anchor items: …

12. Computerized Testing In Licensure, C. David Vale

Licensure Testing: Purposes, Procedures, and Practices

Computerized testing has come out of the laboratory and into the field. By rough estimates, over a million licensure and certification examinations are currently given by computer each year, and the number is rising. Computerized testing is not appropriate for every application, however. Computerized tests always result in significantly greater direct costs than paper-and-pencil tests. To justify their use, a computerized test must result in a net dollar saving. This means that something in the process of computerization must offer a cost reduction that more than offsets the direct cost of computerization. The purpose of this chapter is to identify …

Author Index

Licensure Testing: Purposes, Procedures, and Practices

Authors listed A-Z (8 pages)

A

Ader, H. 1. : 2 10,2 18
Ahluwalia, N. T.: 240, 25 1, 330, 344
Aitkin, M.: 262, 267, 282
AI-Dosary, I. M.: 28 1, 282
AI-Karni , A.: 27 1, 28 1, 282
Albanese, M. A.: 125, 138,324,341
Alexander, R. A.: 225, 250
Algin a, 1.: 256, 283
Allen, D. L.: 158, 164
American Counci l on Education: 167, 185
American Educational Research Association: 33,37,38,39,40, 41 , 42,43,53,55,66, 68,72,84,94, 11 4, 11 8, 122,137, 138, 168, 170, 17 1, 179, 185, 236, 248, 254, 255,26 1,28 1
American Psychological Association: 33, 37, 38,39,40, …

10. Establishing Passing Standards, Craig N. Mills

Licensure Testing: Purposes, Procedures, and Practices

INTRODUCTION

When tests are used to determine eligibility for a license, a passing standard or cut score must be established that divides the test scores into two categories: eligible for license or not. Standard setting has been widely researched and there are many reviews available (see, for example, Jaeger, 1989; Mills & Melican, 1988; Berk, 1986; Hambleton, 1980; Hambleton & Eignor, 1980; and Shepard, 1980a, 1980b), yet there is limited practical advice available for conducting standard setting studies and establishing standards. The one available resource (Livingston & Zieky, 1982) is somewhat dated. The purpose of this chapter is to provide …

9. Differential Item Functioning In Licensure Tests, Barbara S. Plake

Licensure Testing: Purposes, Procedures, and Practices

When test scores are used to make important decisions, as is typically the case with licensure tests, the validity of test score interpretations is extremely critical. The validity of the decision (e.g., pass or fail the licensure examination) relies heavily on the validity of the test score that is used in making the licensure decision. So, although validity is always a critical component in test score interpretation, it has increased importance when the score is used in high-stakes decision situations such as licensure testing.

Issues in validity for licensure tests have been addressed in Chapter 4 of this volume. The …

13. Future Psychometric Practices In Licensure Testing, Steven S. Nettles

Licensure Testing: Purposes, Procedures, and Practices

New technologies continue to emerge each year, and influence testing practices. In particular, in the last 10 years the personal computer has evolved from a curious and minimally useful tool to an indispensable partner in many certification and licensure testing programs. It is involved in every aspect- including candidate scheduling, test assembly, test administration, test scoring and analysis, and score reporting. Initially, it is used to determine the content to be included in the job analysis instrument, and later, to analyze the returned surveys. After the job analysis is completed and test specifications prepared, it can be used to bank …

11. Equating, Judy A. Shea, John J. Norcini

Licensure Testing: Purposes, Procedures, and Practices

INTRODUCTION

Testing programs nearly always need examinations that measure the same thing, but are composed of different questions (i .e., alternate forms of the same test). When different questions are used, however, there is no assurance that scores on the forms are equivalent; different sets of items might be easier or harder and, therefore, produce higher or lower scores. Equating is used to overcome this problem. Simply stated, it is the design and statistical procedure that permits scores on one form of a test to be comparable to scores on an alternate form.

A hypothetical example will help explain why …

Licensure Testing: Purposes, Procedures, And Practices [Complete Work], James C. Impara, Linda L. Murphy

Licensure Testing: Purposes, Procedures, and Practices

Licensure Testing: Purposes, Procedures, and Practices (374 pages)

Copyright © 1995 by Buros Institute of Mental Measurements All rights reserved. No part of this book may be reproduced in any form, by photostat, microfilm, retrieval system, or any other means, without the prior written permission of the publisher.

Buros Institute of Mental Measurements 135 Bancroft Hall University of Nebraska-Lincoln Lincoln, NE 68588-0348

The paper used in this publication meets the minimum requirements of American National Standard for Information Sciences-Permanence of Paper for Printed Library Materials, ANSI Z39.48-1984.

Section One: Family Assessment: History, Theory, And Applications

Family Assessment

In this section a more unified research effort in family assessment is advocated by Dr. Halverson. He urges the constructs most important in the study of families be identified by shifting from the study of isolated components of the family to a more global view of family functioning. There is a lack of attention to the nomological net of constructs. Multi-trait and multi-method analysis is recommended to produce useful information regarding the family.

Dr. Carlson continues this critique by highlighting the influential role of theory in the development and use of family assessment measures and methods. Carlson traces the development …

1. Measurement Beyond The Individual, Charles F. Halverson

Family Assessment

This chapter has several goals. First, I will briefly review the history of measurement as it applies to family assessment. This history has been recounted by many and is available in many recent publications, so I shall be fairly brief. Second, I will discuss family measurement in terms of important issues still facing the family measurement field-issues that are not, in my opinion, being well addressed at this time. And finally, I will attempt to weave these various threads into some speculations about the future directions that family measurement might (or maybe needs) to take.

I will confine this …

6. Assessing Marital Quality In Longitudinal And Life Course Studies, David R. Johnson

Family Assessment

INTRODUCTION

Family researchers have been developing measures to assess the quality of the marital relationship for over six decades (e.g., Hamilton, 1929). Indeed, the quality of the husband-wife relationship has been the focus of more research than any other single topic in the field of family study (Spanier & Lewis, 1980). Embedded in these studies are hundreds of varied scales and measures that were designed to assess some aspect of the quality of a marriage (Touliatos, Perlmutter, & Straus, 1990). Lack of consensus on what constitutes marital quality and the absence of any widely accepted and used instruments have contributed …

Family Assessment- Subject Index

Family Assessment

Subject Index (10 pages)

A-W

A

abandonment: 219-220
ABCX model of family stress: 274
academic ability: 206
achievement: 207, 210, 221,225, 227
acculturation: 107, 114, 123
adjusted mean difference: 228
adolescents: 215-216, 220-221
affect: 20, 31, 40, 69, 71, 75, 77, 82-84
affective responsiveness: 77, 86, 92
African Americans: 105-107,109,113- 114
age: 47,82 aggregation: 9,51,54
Americans with Disabilities Act (ADA): 262
analysis of covariance: 227
anger toward parents: 219-220
antisocial behavior: 235-255
Asian Americans: 105-109, 111, 113, 120
assessment formative: 246-248,253
global: 247 standardized: 262, 264-277
asymptomatic family functioning: 70
attitude: 244-245,247
autonomy: 215
average family functioning: 70

W

well-being: …

3. Assessing Family Health And Distress: An Intergenerational-Systemic Perspective, James H. Bray

Family Assessment

In the past several decades there has been a proliferation of interest and development of family systems theories. A unique aspect of a systems perspective is that human problems develop in and because of social interactions usually within the family, rather than solely from some internal process within an individual. A second innovation is the view that human behavior always occurs in a context, and that understanding the context is essential for understanding problem development and resolution. The empirical evaluation and validation of these perspectives has lagged behind theoretical and therapeutic developments. Further, research in this area has been hampered …

Family Assessment- Epilogue

Family Assessment

The previous chapters have illustrated in great depth the intricacies of family assessment. The meaning of family across cultures, the effects of emotional, physical, and mental challenges on family functioning, and the frameworks useful in defining important family constructs have all been explored.

Although there are many measures for the many constructs that have been created to capture the meaning of family interaction, most are rather exploratory or useful only with limited populations. Clinical judgement and research acumen are required to be sure valid assessments are accomplished. There are significant challenges left to meet in designing assessment programs to illuminate …

Family Assessment- Test Index

Family Assessment

Test Index (4 pages)

A

Adult-Adolescent Parenting Inventory: 78

B

Becker Adjective Checklist: 117
Behavior Rating Profile: 117
Building Houses: 6

C

California Test of Personality: 117
Child Behavior Checklist: 46-47, 265
Child Behavior Profile: 117
Child Behavior Questionnaire: 148
Child Report of Parent Behavior Inventory: 117
Child-Rearing Practices Report: 6
Children's Attitudes Toward Parental Separation Inventory: 217-218
Children's Emotional Reactions to the Divorce: 218
Child's Attitude toward Mother and Father Scales: 117
color-matching test: 5
Colorado Self-Report Measure of Family Functioning: 76, 77, 78
Conflict Tactics Scale: 76
Couples Interaction Scoring System: 5

...

T

Temperament Assessment Battery: 149 …

Family Assessment- Complete Work, Jane Close Conoley, Linda L. Murphy, Elaine Buterick Werth

Family Assessment

Assessing families suggests both interesting measurement issues

and significant clinical applications. This volume is a collection of

important papers to explore the topic in some depth.

Some of these papers were first given at the Buros-Nebraska

Symposium on Testing and Measurement. Others have been written

especially for this volume. All are outstanding examples of scholarship

in this very thorny area of psychological measurement beyond the

individual. We commissioned papers that examined the history of

measurement with families and to cover family issues that are of

particular interest to both clinicians and researchers. Overall, the authors have tackled very big issues. …