Education Commons^™

A Review Of International Large-Scale Assessments In Education: Assessing Component Skills And Collecting Contextual Data, John Cresswell, Ursula Schwantner, Charlotte Waters

OECD Programme for International Student Assessment (PISA)

The OECD has initiated PISA for Development (PISA-D) in response to the rising need of developing countries to collect data about their education systems and the capacity of their student bodies. This report aims to compare and contrast approaches regarding the instruments that are used to collect data on (a) component skills and cognitive instruments, (b) contextual frameworks, and (c) the implementation of the different international assessments, as well as approaches to include children who are not at school, and the ways in which data are used. It then seeks to identify assessment practices in these three areas that will …

Comparison Of Piaac And Pisa Frameworks For Numeracy And Mathematical Literacy, Iddo Gal, David Tout

OECD Programme for International Student Assessment (PISA)

This paper describes key aspects of the frameworks for the assessment of adult numeracy and mathematical literacy in PIAAC and PISA, which are OECD two flagship programs for international comparative assessment of competencies. The paper examines commonalities and differences in how the constructs of adult numeracy and mathematical literacy were assessed in PIAAC and PISA, and sketches selected challenges associated with interpretation of results from these surveys.

A Framework For Predicting Item Difficulty In Reading Tests, Tom Lumley, Alla Routitsky, Juliette Mendelovits, Dara Ramalingam

OECD Programme for International Student Assessment (PISA)

Results on reading tests are typically reported on scales composed of levels, each giving a statement of student achievement or proficiency. The PISA reading scales provide broad descriptions of skill levels associated with reading items, intended to communicate to policy makers and teachers about the reading proficiency of students at different levels. However, the described scales are not explicitly tied to features that predict difficulty. Difficulty is thus treated as an empirical issue, using a post hoc solution, while a priori estimates of item difficulty have tended to be unreliable. Understanding features influencing the difficulty of reading tasks has the …

How Well Do Young People Deal With Contradictory And Unreliable Information On Line? What The Pisa Digital Reading Assessment Tells Us, Tom Lumley, Juliette Mendelovits

OECD Programme for International Student Assessment (PISA)

There is sometimes an assumption that young people, as ‘digital natives’, are able to use online information effectively, including selecting and negotiating digital texts that are not only relevant for what they need, but also are likely to provide reliable information. This paper examines the question of how well young people are in fact able to recognise whether information is likely to be trustworthy. While some small-scale work has been done in this area, this paper draws on data from the first large-scale international assessment of online reading, the Digital Reading Assessment (DRA) that was part of the Organisation for …

Print And Digital Reading In Pisa 2009 : Comparison And Contrast, Juliette Mendelovits, Dara Ramalingam, Tom Lumley

OECD Programme for International Student Assessment (PISA)

PISA was administered for the fourth time in 2009. Since in each administration, one of reading, maths or science is chosen as the major domain, the 2009 survey marked the first time that a domain (in this case, reading) was revisited as the major focus of the assessment. This allowed a full review of the framework for reading literacy and the inclusion of new elements to reflect the way that reading has changed since 2000 (OECD, 2009). One such change is the increasing prevalence of digital texts. The assessment of digital reading in the PISA 2009 cycle, undertaken by 19 …

Some Drivers Of Test Item Difficulty In Mathematics, Ross Turner

OECD Programme for International Student Assessment (PISA)

This paper is one of four contributions to the symposium session at the AERA’s 2012 Annual Meeting titled Exploring Reading and Mathematics Item Difficulty: Teaching and Learning Implications of PISA Survey Data. The author presents a rubric used to analyse mathematics test items developed for use in the OECD’s PISA survey. The rubric focuses on a set of mathematical competencies that are components of mathematical literacy. The work on which this report is based suggests that demand for activation of these competencies functions as a significant driver of item difficulty, which potentially has implications for the teaching and learning of …

Pisa 2009 Technical Report, John Cresswell (Ed)

OECD Programme for International Student Assessment (PISA)

PISA is methodologically highly complex, requiring intensive collaboration among many stakeholders. The successful implementation of PISA depends on the use, and sometimes further development, of state-of-the-art methodologies and technologies. The PISA 2009 Technical Report describes those methodologies, along with other features that have enabled PISA to provide high quality data to support policy formation and review. The descriptions are provided at a level that will enable review and, potentially, replication of the implemented procedures and technical solutions to problems. This report contains a description of the theoretical underpinning of the complex techniques used to create the PISA 2009 Database, which …

Effects Of Item Positions On Their Difficulty And Discrimination : A Study In Pisa Science Data Across Test Language And Countries, Luc T. Le

OECD Programme for International Student Assessment (PISA)

This study was based on a four-cluster rotation design of 13 linked test booklets from PISA 2006 science data. It investigated effects of item positions on their difficulty and discrimination parameter estimates obtained from one and two parameter IRT Partial Credit models. The analyses were done separately for 57 test language groups from 53 countries with a total of about 340,000 students.

The results revealed that for all of the test language groups the items tended to become more difficult when they were located later in the test. However, a high linear relationship between the item difficulty estimates by the …