Education Commons^™

Understanding Pisa And Its Impact On Policy Initiative: A Review Of The Evidence, Petra Lietz, Mollie Tobin, Dita Nugroho

Dr Petra Lietz

Assessing Maths Literacy: Learning From The Pisa Experience, Ross Turner

Ross Turner

A new book that draws on ACER’s work on one of the world’s largest international surveys is providing educators with important insights about how assessment can be used to improve learning outcomes.

Described Proficiency Scales And Learning Metrics, Ross Turner

Ross Turner

We see the process of educational measurement as one of defining dimensions of educational progression and locating learners on those dimensions. Such dimensions are variously referred to as proficiency scales or learning metrics. When we measure something using a scale we can express how much of a given attribute is possessed by that something. We use a scale of centimetres or inches to describe the length of an object. We use a scale of hours and minutes to describe an amount of time. This is also the case for educational measurement: when we measure proficiency in a subject area we …

Pisa In Brief : Highlights From The Full Australian Report : Challenges For Australian Education : Results From Pisa 2009 : The Pisa 2009 Assessment Of Students' Reading, Mathematical And Scientific Literacy., Sue Thomson, Lisa De Bortoli, Marina Nicholas, Kylie Hillman, Sarah Buckley

Kylie Hillman

No abstract provided.

Pisa In Brief : Highlights From The Full Australian Report : Challenges For Australian Education : Results From Pisa 2009 : The Pisa 2009 Assessment Of Students' Reading, Mathematical And Scientific Literacy., Sue Thomson, Lisa De Bortoli, Marina Nicholas, Kylie Hillman, Sarah Buckley

Kylie Hillman

No abstract provided.

Snapshots Issue 3: Equity And Effectiveness, Sue Thomson, Kylie Hillman

Kylie Hillman

A key message from the most recent OECD PISA report released by ACER in December 2013 is that high-performing countries tend to allocate resources more equitably across socioeconomically advantaged and disadvantaged schools. This issue of Snapshots looks to the data from the Progress in International Reading Literacy Study (PIRLS) and the Trends in International Mathematics and Science Study (TIMSS), to examine whether this is the case for Australia.

Pisa In Brief : Highlights From The Full Australian Report : Challenges For Australian Education : Results From Pisa 2009 : The Pisa 2009 Assessment Of Students' Reading, Mathematical And Scientific Literacy., Sue Thomson, Lisa De Bortoli, Marina Nicholas, Kylie Hillman, Sarah Buckley

Kylie Hillman

No abstract provided.

Snapshots Issue 4: Boys, Girls And Mathematics, Lisa De Bortoli

Lisa De Bortoli

The most recent OECD Programme for International Assessment (PISA) survey enables educators, policy makers and the wider community to compare Australian students with each other, as well as their counterparts across the world. PISA measures the extent to which 15-year-old students near the end of compulsory education have acquired the knowledge and skills that young adults need to meet the challenges of the future. This issue of Snapshots looks to the data from PISA on similarities and differences in student performance by sex to inform education policies, improve educational outcomes and equity.

Perspectives On Numeracy: Reflections From International Assessments, Dave Tout, Iddo Gal

David (Dave) Tout

This paper examines perspectives regarding the mathematical skills expected of adults and school graduates, comparing ideas developed as part of two major multinational comparative assessments of skills: the Programme for International Student Assessment and the Programme for International Assessment of Adult Competencies (also known as the OECD Survey of Adult Skills). The paper reflects on the conceptual and assessment frameworks developed for these two programmes, aiming to shed light on the commonalities and differences between the constructs of numeracy and mathematical literacy and to inform current debate about directions for developing mathematical skills in the 21st Century.

Location, Location: Student Achievement And Regionality, Petra Lietz

Dr Petra Lietz

The further schools are from larger cities, the more challenging it is for their students to excel, but some buck the trend. Petra Lietz explains.

Using Competencies To Explain Mathematical Item Demand: A Work In Progress, Ross Turner, Werner Blum, Morgens Niss

Ross Turner

Maths + Confidence = Higher Scores For Boys, Sue Thomson, Lisa De Bortoli

Lisa De Bortoli

Teenage boys outperform Australian girls in mathematics by the equivalent of about one-third of a school year, according to the latest PISA survey, as Sue Thomson and Lisa De Bortoli explain.

Comparison Of Piaac And Pisa Frameworks For Numeracy And Mathematical Literacy, Iddo Gal, David Tout

David (Dave) Tout

This paper describes key aspects of the frameworks for the assessment of adult numeracy and mathematical literacy in PIAAC and PISA, which are OECD two flagship programs for international comparative assessment of competencies. The paper examines commonalities and differences in how the constructs of adult numeracy and mathematical literacy were assessed in PIAAC and PISA, and sketches selected challenges associated with interpretation of results from these surveys.

Beyond Pisa : Improving Student Outcomes, Geoff Masters

Prof Geoff Masters AO

The latest PISA results indicate that we must focus on strategies to improve what happens in every classroom, every day of the school week if we are to improve student outcomes, says Geoff Masters.

Questioning The Standards Of Literacy And Numeracy, David Tout, Juliette Mendelovits

David (Dave) Tout

David Tout and Juliette Mendelovits examine why we receive such differing reports on the literacy and numeracy skills of young Australians.

Questioning The Standards Of Literacy And Numeracy, David Tout, Juliette Mendelovits

Juliette Mendelovits

David Tout and Juliette Mendelovits examine why we receive such differing reports on the literacy and numeracy skills of young Australians.

A Teacher's Guide To Pisa Mathematical Literacy, Sue Thomson, Kylie Hillman, Lisa De Bortoli

Kylie Hillman

This report focuses on mathematical literacy and is one of a series of three reports on Australian students' performance in PISA. The report provides an overview of the PISA mathematics framework and Australia's results in the PISA 2003 international assessment. Also included are mathematics items released for public viewing after the PISA 2003 assessment and examples of responses, marking guides and comparisons of results with other countries. The context behind achievement, e.g. attitudes, engagement and learning strategies, is also presented.

A Teacher's Guide To Pisa Scientific Literacy, Sue Thomson, Kylie Hillman, Lisa De Bortoli

Kylie Hillman

This report focuses on Australian students' performance on the PISA items that have been released in Scientific Literacy. It includes an overview of the PISA Scientific Literacy Framework, Australian students' results in the 2006 international assessment, examples of responses and marking guides and the context behind achievement, e.g. attitudes, engagement and motivation, and recognition of environmental issues.

A Framework For Predicting Item Difficulty In Reading Tests, Tom Lumley, Alla Routitsky, Juliette Mendelovits, Dara Ramalingam

Dr Alla Routitsky

Results on reading tests are typically reported on scales composed of levels, each giving a statement of student achievement or proficiency. The PISA reading scales provide broad descriptions of skill levels associated with reading items, intended to communicate to policy makers and teachers about the reading proficiency of students at different levels. However, the described scales are not explicitly tied to features that predict difficulty. Difficulty is thus treated as an empirical issue, using a post hoc solution, while a priori estimates of item difficulty have tended to be unreliable. Understanding features influencing the difficulty of reading tasks has the …

Large-Scale Group Score Assessments: Past, Present, And Future, B Naemi, E Gonzalez, J Bertling, A Betancourt, J Burrus, P Kyllonen, J Minsky, Petra Lietz, E Klieme, S Vieluf, J Lee, R Roberts

Dr Petra Lietz

he influence of large-scale group score assessments on research, policy, and practice in education has increased dramatically over the past few decades. The goal of this chapter is to provide an overview of the value and scope of this program of research. The chapter begins by providing an overview of the history of large-scale assessment. Next, it focuses on current research and development surrounding the National Assessment of Educational Progress (NAEP, a government-mandated assessment in the United States) and the Programme for International Student Assessment (PISA, a large-scale survey and assessment commission by the Organisation for Economic Co-operation and Development, …

Large-Scale Group Score Assessments: Past, Present, And Future, Petra Lietz

Dr Petra Lietz

The influence of large-scale group score assessments on research, policy, and practice in education has increased dramatically over the past few decades. The goal of this chapter is to provide an overview of the value and scope of this program of research. The chapter begins by providing an overview of the history of large-scale assessment. Next, it focuses on current research and development surrounding the National Assessment of Educational Progress (NAEP, a government-mandated assessment in the United States) and the Programme for International Student Assessment (PISA, a large-scale survey and assessment commission by the Organisation for Economic Co-operation and Development, …

Print And Digital Reading In Pisa 2009 : Comparison And Contrast, Juliette Mendelovits, Dara Ramalingam, Tom Lumley

Dr Tom Lumley

PISA was administered for the fourth time in 2009. Since in each administration, one of reading, maths or science is chosen as the major domain, the 2009 survey marked the first time that a domain (in this case, reading) was revisited as the major focus of the assessment. This allowed a full review of the framework for reading literacy and the inclusion of new elements to reflect the way that reading has changed since 2000 (OECD, 2009). One such change is the increasing prevalence of digital texts. The assessment of digital reading in the PISA 2009 cycle, undertaken by 19 …

Using Mathematical Competencies To Predict Item Difficulty In Pisa, Ross Turner, J Dossey, W Blum, M Niss

Ross Turner

On The Use Of Rotated Context Questionnaires In Conjunction With Multilevel Item Response Models, Petra Lietz

Dr Petra Lietz

While rotated test booklets have been employed in large-scale assessments to increase the content coverage of the assessments, rotation has not yet been applied to the context questionnaires administered to respondents. This paper describes the development of a methodology that uses rotated context questionnaires in conjunction with multilevel item response models and plausible values. In order to examine the impact of this methodology on the continuity of the results, PISA 2006 data for nine heterogeneous countries were rescaled after having been restructured to simulate the outcomes of the use of different rotated context questionnaire designs. Results revealed negligible differences when …

Some Drivers Of Test Item Difficulty In Mathematics : An Analysis Of The Competency Rubric, Ross Turner, Ray Adams

Ross Turner

This paper is concerned with the empirical validation of the competency rubric described in another paper presented at the same conference: Turner, Ross (April 2012). Some drivers of test item difficulty in mathematics. Paper presented at the Annual Meeting of the American Educational Research Association (AERA), Vancouver, 13-17 April 2012 http://research.acer.edu.au/pisa/4/

Using items developed for the PISA 2012 survey, and data collected as part of an extensive field trial of the PISA tasks conducted during 2011 in some 67 countries, the authors use multidimensional Rasch modelling and latent regression to examine the following three questions: 1. What is the …

An Exploratory Analysis Of The Talis And Pisa Link Data: An Investigation Of The Possible Relationships, Frances Eveleigh, Chris Freeman

Chris Freeman

This paper proposes to report a preliminary investigation of the field trial data of PISA combined with the TALIS data from the same pool of schools. It proposes exploratory analyses of the data through correlation, ANOVA and MANOVA, and multi-level modelling techniques to identify plausible relationships and explained variation that may be uncovered within the data. This investigation will inform the types of analyses that may be performed on the main study data that are being collected in mid to late 2012.

Some Drivers Of Test Item Difficulty In Mathematics, Ross Turner

Ross Turner

This paper is one of four contributions to the symposium session at the AERA’s 2012 Annual Meeting titled Exploring Reading and Mathematics Item Difficulty: Teaching and Learning Implications of PISA Survey Data. The author presents a rubric used to analyse mathematics test items developed for use in the OECD’s PISA survey. The rubric focuses on a set of mathematical competencies that are components of mathematical literacy. The work on which this report is based suggests that demand for activation of these competencies functions as a significant driver of item difficulty, which potentially has implications for the teaching and learning of …

A Framework For Predicting Item Difficulty In Reading Tests, Tom Lumley, Alla Routitsky, Juliette Mendelovits, Dara Ramalingam

Dr Tom Lumley

Results on reading tests are typically reported on scales composed of levels, each giving a statement of student achievement or proficiency. The PISA reading scales provide broad descriptions of skill levels associated with reading items, intended to communicate to policy makers and teachers about the reading proficiency of students at different levels. However, the described scales are not explicitly tied to features that predict difficulty. Difficulty is thus treated as an empirical issue, using a post hoc solution, while a priori estimates of item difficulty have tended to be unreliable. Understanding features influencing the difficulty of reading tasks has the …

Differential Performance Of Poor Readers In International Mathematics Tests By Item Formats, Luc Le

Dr Luc Tu Le

The important role of language, including reading, on student performance on assessments in school academic areas such as mathematics has been addressed in number of studies. Although positive relations between different aspects of reading and mathematics performance have been investigated, there have been few studies focusing on the interaction between the context of the mathematics item and student reading proficiency, and most of those there were related only to the verbal loading. Recently, a low correlation between verbal loading and item difficulty was found for PISA mathematics items. This study used PISA 2003 mathematics data with 41 countries with about …

Pisa In Brief : Highlights From The Full Australian Report : Challenges For Australian Education : Results From Pisa 2009 : The Pisa 2009 Assessment Of Students' Reading, Mathematical And Scientific Literacy., Sue Thomson, Lisa De Bortoli, Marina Nicholas, Kylie Hillman, Sarah Buckley

Lisa De Bortoli

No abstract provided.