Education Commons^™

Factors That Influence The Difficulty Of Problem Solving Items, Dara Ramalingam, Ray Philpot

Ray Philpot

Computer-based assessment of problem solving allows problems of both static and interactive natures to be posed. Examples of static problems are scheduling and logic puzzles in which all relevant information is available to the solver at the outset. Problems of an interactive nature, on the other hand, require exploration of the situation to acquire additional knowledge needed to solve the problem. Examples include discovering how to use an unfamiliar mobile telephone or automatic vending machine. This study used data from the 2011 Field Trial of the PISA 2012 computer-based assessment of problem solving which comprised 34 static and 45 interactive …

Measuring The Outliers: An Introduction To Out-Of-Level Testing With High-Achieving Students, Karen Rambo-Hernandez, Russell Warne

Russell T Warne

Out-of-level testing is an underused strategy for addressing the needs of students who score in the extremes, and when used wisely, it could provide educators with a much more accurate picture of what students know. Out-of-level testing has been shown to be an effective assessment strategy with high-achieving students; however, out-of-level testing has not been shown to work well with low-achieving students. This article provides a brief history of out-of-level testing, along with guidelines for using it.

Special Education At The Core: Where Do The Common Core State Standards Leave Our Students With Disabilities?, Lisa Beymer

Lisa Beymer

The oversimplification with which the new Common Core State Standards approach the instruction of students with disabilities leaves much to be desired on the part of the classroom teacher, who is left to determine how the standards will be made accessible for all students. The new standards require increased skill and application across the subject areas. Those students with specific learning needs may require additional support, altered classroom instruction and an increase in staff-to-specialist collaboration. At the same time, the new standards also provide an opportunity for students with disabilities to access higher-level thinking skills that may prove advantageous for …

Getting Off The Back Burner: The Impact Of Testing Elementary Social Studies As Part Of A State-Mandated Accountability Program, Kenneth Vogler, Timothy Lintner, George Lipscomb, Herman Knopf, Tina Heafner, Tracy Rock

Timothy Lintner

No abstract provided.

Trialling Of Test Items; Is The Data A Reliable Predictor Of Final Test Performance?, Ross Hudson

Dr Ross Hudson

Trialling is seen as a necessary first step in producing a reliable valid assessment tool. However, how reliable is the trialling result in terms of predicting population test performance?

Pisa Reading Literacy Framework, Juliette Mendelovits

Juliette Mendelovits

No abstract provided.

Adaptive Testing For Psychological Assessment: How Many Items Are Enough To Run An Adaptive Testing Algorithm?, Michaela Wagner-Menghin, Geoff Masters

Prof Geoff Masters AO

Although the principles of adaptive testing were established in the psychometric literature many years ago (e.g., Weiss, 1977), and practice of adaptive testing is established in educational assessment, it is not yet widespread in psychological assessment. One obstacle to adaptive psychological testing is a lack of clarity about the necessary number of items to run an adaptive algorithm. The study explores the relationship between item bank size, test length and measurement precision. Simulated adaptive test runs (allowing a maximum of 30 items per person) out of an item bank with 10 items per ability level (covering .5 logits, 150 items …

Naplan And My School : Shedding Some Light On A Work In Progress, Geoff Masters

Prof Geoff Masters AO

Debate about NAPLAN and the My School website has generated plenty of heat. Geoff Masters casts some light on what is essentially a work In progress.

Computer Adaptive Testing : A Feasibility Study, Siek Khoo, Geoff Masters, Ray Adams

Prof Geoff Masters AO

The Australian National Office of Overseas Skills Recognition (NOOSR) commissioned ACER to investigate the feasibility of implementing computer adaptive testing (CAT) in NOOSR’s screening examinations for overseas trained professionals. The National Office administers screening examinations in seven professions (Dentistry, Dietetics, Occupational Therapy, Podiatry, Pharmacy, Physiotherapy and Veterinary Science) at venues throughout Australia and in up to fifty centres around the world. Surveys were conducted to gain an overview of the current methods and procedures used by NOOSR in the screening examinations. Issues related to the application of CAT to NOOSR’s screening examinations and the possible improvement of NOOSR’s assessment program …

History And Development Of Above-Level Testing Of The Gifted, Russell Warne

Russell T Warne

Above-level testing (also called out-of-level testing, off-grade testing, and off-level testing) is the practice of administering a test level that was designed for and normed on an older population to a gifted child. This comprehensive literature review traces the practice of above-level testing from the earliest days of gifted education through the present. It was found that there were five reasons frequently given for above-level testing: raising the test ceiling, increasing score variability and discrimination, improving reliability, the sound interpretations of above-level test data, and reducing regression toward the mean. Although all of these reasons were theoretically supported, the strength …

Which Form Of Assessment In A Chemistry Examination Best Describes Student Understanding?, Ross Hudson

Dr Ross Hudson

No abstract provided.

Does Question Type, Content And Gender Influence Student Understanding As Demonstrated In An Entrance Examination?, Ross Hudson, David Treagust

Dr Ross Hudson

The research inquires into the effectiveness of the two predominant forms of questions that are used on the State University Entrance examination for chemistry. These are multiple-choice questions and short-answer questions. This research examines the style of question but also the content type examined (recall and application questions) along with gender differences. The research involved an analysis of previous State University Examinations as well as class trial testing students of both genders on tests designed by the researcher. Rasch analysis of the class trial data was performed allowing comparison of question type and content performance as well as differential analysis …

Creating Powerful High Schools For Immigrant And English Language Learning Populations: Using Past And Present Ideas In Today's Schooling Paradigm, Reynaldo Reyes, Leena Her

Leena N. Her

The high-stakes testing climate and the growing immigrant and English language-learning population have changed the face of teaching and learning in today's high schools. In this chapter, the authors emphasize the impact of a new paradigm of schooling based on high-stakes testing on Asian and Latin American students, as they represent the largest combined immigrant and ELL student populations. They discuss the research on high schools that have worked with significant numbers of immigrant and ELL students, and what can be learned from the ideas and programs that such schools have implemented. The authors argue that high schools in today's …

Pisa : Frequently Answered Criticisms, Ray Adams

Prof Ray Adams

Studies such as PISA that attempt to compare outcomes across educational systems are expensive and difficult to implement. Further, the results of such studies are routinely criticized by educational commentators - particularly when the results are not consistent with their preconceived ideas about the relative merits and efficiencies of various educational practices and systems. This chapter discusses what is done to ameliorate the threats to the validity of PISA in five areas often targeted by commentators and reviewers as sources of invalidity in international comparisons. The five areas discussed are: (1) sampling - are the samples of students who undertake …

Patscience : Progressive Achievement Tests In Science, Ross Hudson, Ron Martin, Daniel Urbach, Stavroula Zoumboulis

Dr Ross Hudson

The ACER progressive achievement tests in science are for use in Australian schools to provide information to teachers about the level of achievement attained by their students in the concepts, skills and processes of science.

Best Start 2008 : Kindergarten Literacy Assessment, Department Of Education And Training, Nsw : Data Analysis Report, June 2008, Marion O. Meiers, Siek Toon Khoo

Marion Meiers

Language Proficiency And Testing For Migration Purposes: What Are The Practical Implications?, Sacha Develle

Dr Sacha DeVelle

No abstract provided.

Ameliorating Culturally Based Extreme Response Tendencies To Attitude Items, Maurice Walker

Maurice Walker

No abstract provided.

The Influence Of Equating Methodology On Reported Trends In Pisa, Eveline Gebhardt, Ray Adams

Prof Ray Adams

In 2005 PISA published trend indicators that compared the results of PISA 2000 and PISA 2003. This paper explores the extent to which the outcomes of these trend analyses are sensitive to the choice of test equating methodologies, the choice of regression models and the choice of linking items. To establish trends, PISA equated its 2000 and 2003 tests using a methodology based on Rasch Modelling that involved estimating linear transformations that mapped 2003 Rasch-scaled scores to the previously established PISA 2000 Rasch-scaled scores. This paper compares the outcomes of this approach with an alternative, which involves the joint Rasch …

The Impact Of Differential Investment Of Student Effort On The Outcomes Of International Studies, J Butler, Ray Adams

Prof Ray Adams

International comparative assessments of student achievement, such as Trends in Mathematics and Science (TIMSS) and Programme for International Student Achievement (PISA) are becoming increasingly important in the development of evidence-based education policy. The potentially far-reaching influence of such studies underscores the need for these assessments to be valid and reliable. In education, increasing recognition is being given to motivational factors which impact on student learning. This research considers a possible threat to the validity of such studies by investigating the influence the amount of effort invested by test-takers has on their outcomes. Reassuringly, it is found that the reported expenditure …

Assessing Second Language Writing: The Rater’S Perspective, Tom Lumley

Dr Tom Lumley

This study investigates the process of rating texts written by adult ESL learners. Four experienced raters provided think-aloud protocols describing the rating process for a set of 24 texts. The think-aloud data allowed analysis of the sequence of rating, raters' interpretations of the scoring categories, and difficulties raters faced. The study reveals the complexity of the rating process, whereby raters struggle to resolve a tension between the wordings (or rules) of the rating scale and their complex, initial, intuitive impression of the text. Rating requires training to provide reliable measurement. The study also demonstrates that caution is needed in interpreting …

Assessment Criteria In A Large-Scale Writing Test: What Do They Really Mean To The Raters?, Tom Lumley

Dr Tom Lumley

The process of rating written language performance is still not well understood, despite a body of work investigating this issue over the last decade or so (e.g., Cumming, 1990; Huot, 1990; Vaughan, 1991; Weigle, 1994a; Milanovic et al., 1996). The purpose of this study is to investigate the process by which raters of texts written by ESL learners make their scoring decisions using an analytic rating scale designed for multiple test forms. The context is the Special Test of English Proficiency (step), which is used by the Australian government to assist in immigration decisions. Four trained, experienced and reliable step …

The Effect Of Interlocutor And Assessment Mode Variables In Overseas Assessments Of Speaking Skills In Occupational Settings, T Mcnamara, Tom Lumley

Dr Tom Lumley

The increasing demand for performance assessment of speaking skills in second languages has led to logistic complications, for example, the delivery of tests in overseas locations. One solution to the problem has been to train native interlocutors to carry out a series of oral interactions with the candidate, with assessment from audiorecordings of the test session postponed and conducted centrally by a small team of trained raters. But these procedures raise questions about the effect of such facets of the assessment situation as interlocutor variables and the quality of the audiotape recording. This article examines these issues in the context …