Education Commons^™

Understanding Pisa And Its Impact On Policy Initiative: A Review Of The Evidence, Petra Lietz, Mollie Tobin, Dita Nugroho

Dr Petra Lietz

Using Large-Scale Assessments Of Students' Learning To Inform Education Policy: Insights From The Asia-Pacific Region, Mollie Tobin, Petra Lietz, Dita Nugroho, Ramya Vivekanandan, Tserennadmid Nyamkhuu

Dr Petra Lietz

Not much is known about the ways in which assessment data have actually been used in education policy to date. Understanding the role of assessments in informing system-level decision-making is a first step towards helping stakeholders improve the design and usefulness of assessments. Moreover, this understanding can help to further discussions about how assessment data can best be used to inform policy and practice and to evaluate the effectiveness of policy reforms. This paper presents results from a systematic review of 68 studies that examined the link between participation in large-scale assessment programs of students’ learning and education policy in …

Comparative Research In Education: Iea Studies, Petra Lietz

Dr Petra Lietz

During the second half of the twentieth century, the globalization movement transformed the nature of comparative research in education. The United Nations Organization and its agency UNESCO, promoting cooperation among nations through education, science, and culture, led this development through ‘Education for All’ and the integration of organizational, international, comparative, and global approaches to education. The conduct of educational research worldwide has been greatly influenced by the International Association for the Evaluation of Educational Achievement through the monitoring of educational outcomes and the introduction of systemic and modeling approaches to research in education leading to evidence-based making of educational policy.

Location, Location: Student Achievement And Regionality, Petra Lietz

Dr Petra Lietz

The further schools are from larger cities, the more challenging it is for their students to excel, but some buck the trend. Petra Lietz explains.

Towards A ‘Good Life’ For Children In Their Middle Years, Petra Lietz

Dr Petra Lietz

Petra Lietz describes a new research collaboration to design and conduct Australia’s first internationally comparable survey of student wellbeing, with a particular focus on disadvantaged students.

Informing Policy In Developing Countries, Petra Lietz, Mollie Tobin

Dr Petra Lietz

Petra Lietz and Mollie Tobin discuss the impact of large-scale assessment programs on education policy in developing countries.

Large-Scale Group Score Assessments: Past, Present, And Future, B Naemi, E Gonzalez, J Bertling, A Betancourt, J Burrus, P Kyllonen, J Minsky, Petra Lietz, E Klieme, S Vieluf, J Lee, R Roberts

Dr Petra Lietz

he influence of large-scale group score assessments on research, policy, and practice in education has increased dramatically over the past few decades. The goal of this chapter is to provide an overview of the value and scope of this program of research. The chapter begins by providing an overview of the history of large-scale assessment. Next, it focuses on current research and development surrounding the National Assessment of Educational Progress (NAEP, a government-mandated assessment in the United States) and the Programme for International Student Assessment (PISA, a large-scale survey and assessment commission by the Organisation for Economic Co-operation and Development, …

Large-Scale Group Score Assessments: Past, Present, And Future, Petra Lietz

Dr Petra Lietz

The influence of large-scale group score assessments on research, policy, and practice in education has increased dramatically over the past few decades. The goal of this chapter is to provide an overview of the value and scope of this program of research. The chapter begins by providing an overview of the history of large-scale assessment. Next, it focuses on current research and development surrounding the National Assessment of Educational Progress (NAEP, a government-mandated assessment in the United States) and the Programme for International Student Assessment (PISA, a large-scale survey and assessment commission by the Organisation for Economic Co-operation and Development, …

The Relationship Between School Resources And Grade 8 Mathematics Achievement: A Comparison Of Palestinian Authority, Israeli Hebrew And Israeli Arab Schools In Timss 2007, Yasin Afana, Petra Lietz, Mollie Tobin

Dr Petra Lietz

The relationship between school resources such as learning materials, computers, and student-teacher ratio and student performance has been shown to be important. In this article, school resources and their relationship with student achievement are compared for Israeli and Palestinian Authority (PA) schools using data from the International Association for the Evaluation of Educational Achievement’s (IEA) 2007 Trends in International Mathematics and Science Study (TIMSS) in the context of eff orts aimed at reducing the differences in resources in schools in Israel. Results of bivariate analyses show some differences regarding the availability of school resources between the three groups of schools …

On The Use Of Rotated Context Questionnaires In Conjunction With Multilevel Item Response Models, Petra Lietz

Dr Petra Lietz

While rotated test booklets have been employed in large-scale assessments to increase the content coverage of the assessments, rotation has not yet been applied to the context questionnaires administered to respondents. This paper describes the development of a methodology that uses rotated context questionnaires in conjunction with multilevel item response models and plausible values. In order to examine the impact of this methodology on the continuity of the results, PISA 2006 data for nine heterogeneous countries were rescaled after having been restructured to simulate the outcomes of the use of different rotated context questionnaire designs. Results revealed negligible differences when …

Quality Of Education In Madrasah : Main Study, Mohammad Ali, Julie Kos, Petra Lietz, Dita Nugroho, Furqon, Asmawi Zainul, Emi Emilia

Dr Petra Lietz

This report covers the main phase of the ‘Quality of Education in Madrasah’ (QEM) study. The study aimed to provide high quality research into various dimensions of quality of education in Madrasah in Indonesia. The focus of the study was on Year 9 students in Madrasah Tsanawiyah (MTs; Islamic Junior Secondary School). Quality of education was measured through: (1) level of student achievement, comparing three regions in the country and to some extent using international benchmarks, as well as the identification of variables that are linked to achievement; (2) students’ views of their schooling experience; and (3) the extent to …

The Impact Of Values And Learning Approaches On Student Achievement: Gender And Academic Discipline Influences, Liudmila Tarabashkina, Petra Lietz

Dr Petra Lietz

This paper presents results from a longitudinal study of sojourner students which was conducted at an international university in Germany from 2004 to 2007. The study followed a cohort of undergraduate students from the first week of their studies to their graduation. Participants completed three questionnaires: the Portrait Value Questionnaire (Schwartz et al., 2001), the Study Process Questionnaire (Biggs, 1987b), and the Student Background Questionnaire (Matthews, Lietz, & Darmawan, 2007). Structural Equation Modelling was used to examine how personal values influenced students' learning approaches and how these, in turn, were related to students' achievement. It was also examined how robust …

The Effects Of College Students’ Personal Values On Changes In Learning Approaches, Petra Lietz, Bobbie Matthews

Dr Petra Lietz

Many studies of changes in learning approaches have used data from different age groups at one point in time only (Gow and Kember, High Educ 19:307–322, 1990; Watkins and Hattie, Br J Educ Psychol 51:384–393, 1981) or have analyzed the effects of just two or three factors using single level analytical techniques (Cano, Br J Educ Psychol 75:203–221, 2005; Duckwall et al., Res High Educ 32(1):1–13, 1991; Jay and Love, NCSSSMST J 7(2):4–8, 2002; Loo, Educ Psychol, 17(1/2), 1997; Watkins and Hattie, Hum Learn 4:127–141, 1985; Zeegers, Br J Educ Psychol 71:115–132, 2001). This study employs multilevel modeling as a …

The Impact Of Values And Learning Approaches On Achievement: Do Gender And Academic Discipline Make A Difference., Petra Lietz, Liudmila Tarabashkina

Dr Petra Lietz

This paper presents results from a longitudinal study of sojourner students which was conducted at an international university in Germany from 2004 to 2007. The study followed one cohort of undergraduate students from the first week of their studies to graduation from their Bachelor of Science or Bachelor of Arts degree. Participants completed three questionnaires at the beginning of each year: the Portrait Value Questionnaire (PVQ) (Schwartz et al. 2001), the Study Process Questionnaire (SPQ) (Biggs 1987), and the Student Background Questionnaire (Matthews, Lietz & Darmawan 2007). Structural Equation Modelling (SEM) was used to examine how personal values influenced students’ …

I'M A Girl, You're A Boy. You Study Social Science, I Study Natural Science:, Petra Lietz, Ognyan Seizov

Dr Petra Lietz

In this multiple case study (n= 8) with process orientation (Yin, 2003) in-depth interviews were conducted annually over a three-year period with undergraduate students from eight countries at an international university. They had been purposely selected to reflect a mix of learning approaches, gender and discipline areas. Interviews probed students about their instructional experiences, such as evaluation of teaching, assessment preferences and preparation, adaptation of learning approaches and plans for the future. The systematic content analysis was aimed at comparing responses across learning approaches, gender and discipline areas. Results indicated commonalities across learning approaches, gender and discipline areas in terms …

Are Values More Important Than Learning Approaches? Factors Influencing Student Performance At An International University, Petra Lietz, Bobbie Matthews

Dr Petra Lietz

Educators frequently face the question regarding the extent to which they can actually influence student learning, particularly as learners are also influenced by a complex network of many different factors. Specifically, it is of interest to identify so called malleable variables - that is factors which can be influenced by educators compared with variables that educators are unable to change. In this study, data from a cohort of 207 students pursuing a Bachelor degree at a German international university where the language of instruction is English are examined to investigate the way in which various background factors influence student performance. …

Values And Learning Approaches Of Students At An International University, Bobbie Matthews, Petra Lietz, I Gusti Darmawan

Dr Petra Lietz

This study indicates that values are statistically significant precursors to approaches to learning in a cohort of predominantly Bulgarian, German and Romanian students studying at a German university where the language of instruction in all subject areas is English. Values have been measured with the Portrait Values Questionnaire, and approaches to learning have been assessed by the Study Process Questionnaire. The relationships between values and approaches to learning have been estimated by canonical correlation analysis. Results of the analysis suggest that values can be linked to learning approaches in a situation where students have left their home countries to undertake …

Some Problems In The Analysis Of Cross-National Survey Data, John Keeves, Kelvin Gregory, Petra Lietz, I Gusti Darmawan

Dr Petra Lietz

In this article three emergent problems in the analysis of cross-national survey data are raised in a context of 40 years of research and development in a field where persistent problems have arisen and where scholars across the world have sought solutions. Anomalous results have been found from secondary data analyses that would appear to stem from the procedures that have been employed during the past 15 years for the estimation of educational achievement. These estimation procedures are briefly explained and their relationships to the observed anomalies are discussed. The article concludes with a challenge to the use of Bayesian …

Issues In The Change In Gender Differences In Reading Achievement In Cross-National Research Studies Since 1992 : A Meta-Analytic View, Petra Lietz

Dr Petra Lietz

Results of a previous meta-analysis of gender differences in reading achievement at the secondary school level showed significant differences between major assessment programs. Thus, the gender gap in favour of girls was more pronounced for the assessment programs conducted by the National Assessment of Educational Programs in the United States (NAEP), for the more recent assessment programs in Australia and the Programme for the International Student Assessment (PISA) conducted by the OECD. In contrast, no such effect was found for earlier studies conducted by the International Association for the Evaluation of Educational Achievement (IEA), namely the International Reading Comprehension Study …

A Meta-Analysis Of Gender Differences In Reading Achievement At The Secondary School Level, Petra Lietz

Dr Petra Lietz

This study conducts a meta-analysis using hierarchical linear modeling (HLM) to address the questions of the extent of gender differences in reading, whether or not these differences apply similarly across English and non-English speaking countries and decrease with age. Female secondary students performed 0.19 standard deviation units above their male peers, regardless of age, language of instruction, and whether effect sizes were based on mean differences or correlation coefficients. However, differences emerged between assessment programs, with recent Australian Studies, the National Assessment of Educational Progress Studies (NAEP) and the Programme for International Student Assessment (PISA) reporting greater gender differences.

Factors Influencing Reading Achievement In Germany And Spain : Evidence From Pisa 2000, Petra Lietz, Dieter Kotte, Maria Lopez

Dr Petra Lietz

This article examines the factors fostering and inhibiting student achievement in Germany and Spain at the 15-year-old level in the OECD Programme on International Student Assessment (PISA) in 2000. Both countries performed significantly below the OECD average not only in Reading but also in Mathematics and Science on this occasion. Since the two countries are similar in average levels of student achievement, it is of interest to consider whether similar patterns exist in the ways in which factors operate to influence student achievement in these two countries. Preliminary analyses were carried out with PLSPATH and subsequent two-level models were analysed …

Manual And Automatic Estimates Of Growth And Gain Across Year Levels, Petra Lietz, Dieter Kotte

Dr Petra Lietz

Users of statistical software are frequently unaware of the calculations underlying the routines that they use. Indeed, users, particularly in the social sciences, are often somewhat adverse towards the underlying mathematics. Yet, in order to appreciate the thrust of certain routines, it is beneficial to understand the way in which a program arrives at a particular solution. Based on data from the Economic Literacy Study conducted at year 11 and 12 level across Queensland in 1998, this article renders explicit the steps involved in calculating growth and gain estimates in student performance. To this end, the first part of the …

On Decreasing Gender Differences And Attitudinal Changes: Factors Influencing Australian And English Pupils' Choice Of A Career In Science, Linda Miller, Petra Lietz, Dieter Kotte

Dr Petra Lietz

Encouraging the entry of women into science-related careers remains problematic in the UK and Australia. Although recent initiatives have resulted in increased participation of girls in science-related subjects in schools, there remains a significant gender divide in entry to scientific programmes of study at post-compulsory and university levels. In this study, datasets for 13-year-old pupils in Australia and England taken from the Third International Maths and Science Survey (TIMSS) were used to derive path models that identify factors influencing the extent to which science-related careers were viewed as desirable by Australian and English pupils. Results indicated that, in both countries, …

Levels Of Economic Literacy: From Items To Global Indicators, Petra Lietz, Dieter Kotte, Danielle Hebers

Dr Petra Lietz

Currently, Economics is an elective school subject in all States and Territories. In Queensland, only students in Year 11 and 12 have the possibility to study Economics. The major aim of the Economic Literacy Survey - Queensland 1998 was to assess Economic literacy levels of students at the senior secondary school throughout Queensland. The target group, thus, comprised students from both Year 11 and 12. The Economic Literacy Survey built on last year's Economic Literacy Pilot Study conducted in schools throughout Central Queensland. Piloting served to validate the various questionnaires for students, teachers and schools participating in the survey. Performance …

The Sources Of The Evidence, Petra Lietz

Dr Petra Lietz

In this chapter, the data collected from two large international studies conducted by the IEA, namely the 1970-71 Reading Comprehension Study and the 1990-91 Reading Literacy Study ample have been described. Information gathered in eight countries that participated in both studies at itudy. the 14-year-old level, namely Belgium (French), Finland, Hungary, Italy, The Netherlands, New Zealand, Sweden and the U.S., constitute the body of data under consideration. This international evidence is used in the present study in the search for general structural and causal models of reading comprehension that apply across countries. On the one hand, the two studies share …

International Comparative Research In Education, Petra Lietz

Dr Petra Lietz

Internationally comparative research in education which is aimed at investigating the relationships and processes that explain possible differences between educational systems has so far relied on the replication of the same analysis using different data sets. Patterns of results from these analyses have allowed for the examination of differences, uniquenesses or similarities of the way in which factors operated in various countries. However, it has not been possible to test the extent to which effects varied across countries. Recent technological and methodological advances have made it possible to move beyond replication to the systematic and rigorous testing of structural and …

In Search Of General Causal Models Of Reading Comprehension Across Countries Using Conjunct Analysis, Petra Lietz

Dr Petra Lietz

No abstract provided.

Learning And Writing Difficulties At The Tertiary Level: Their Impact On First Year Results, Petra Lietz

Dr Petra Lietz

No abstract provided.

In Search Of General Structural Models For Reading Comprehension Across Countries Using Simultaneous Analysis, Petra Lietz

Dr Petra Lietz

The previous chapter focused on the application of simultaneous analysis to investigate structural models of reading comprehension. Results of the simultaneous analysis of data sets from eight countries and on two occasions showed that it was consistent with the evidence to assume a general factor underlying the ability to read with comprehension at the 14-year-old level. Hence, it can be argued that reading comprehension beyond the initial stages involves a general ability that is similarly structured across cultures and across time. As a consequence, it is meaningful and of considerable interest to compare levels of reading achievement and examine the …

The Move Beyond Replication: What Are Its Benefits, Petra Lietz

Dr Petra Lietz

Results presented in this monograph have emphasized that analytical efforts should seek to apply techniques that are appropriate to the structure of the available data. Moreover, the application of simultaneous and conjunct analyses across countries has illustrated that the use of appropriate techniques does contribute important information to the understanding of educational phenomena that would not necessarily be revealed by the analysis of data within a particular country. In this way, the present study has shown that internationally comparative research can provide useful explanations of phenomena involving educational outcomes as well as the processes influencing such outcomes. In summary, the …