Education Commons^™

Creating An Instrument To Explore The Self-Efficacy Of Writing Instructors To Teach Apprehensive Writers, Kathleen Marie Romrell

Theses and Dissertations

Writing apprehension is a writing difficulty that can impair the writing experiences of students. While researchers have proposed many interventions that teachers can implement to assist their students who struggle with writing apprehension, there has not been an instrument that is designed to measure teacher self-efficacy to help these same students. This study created such an instrument, entitled the Self-efficacy to Teach Apprehensive Writers Scale (STAWS), using Tschannen-Moran and Woolfolk Hoy’s (2001) Teacher Sense of Efficacy Scale and Daly and Miller’s (1975) Writing Apprehension Test as model instruments. The new instrument was designed to measure writing instructor self-efficacy to teach …

Validity Evidence Based On Content: Controversies And Quantification, Agung Santoso, Alice Whita Savira, Robertus Landung Eko Prihatmoko

International Conference on Assessment and Learning (ICAL)

Validity is a crucial issue in test development because it represents the accuracy of test score interpretation in describing the measured attribute or construct. Among validity evidence that can be collected to support test score interpretation, the one based on test content incited controversies. The content-based validity evidence includes analyses of the relationship between test content and the construct the test purported to measure. Some authors opposed the use of such evidence in the validation process, arguing that the evidence can only provide hypotheses to be tested in the subsequent processes. Other authors argued that the content validation process provided …

Is It Necessary To Use Two-Way Items? Testing The Effects Of Using Favourable And Unfavourable Items, Sukaesi Marianti, Dera Kariema, Puri Intan Larasati, Dhia Ulfah Purwati, Nur Hasanah

International Conference on Assessment and Learning (ICAL)

The topic of ethnicity is a topic that is often discussed in research, especially psychology research. Considering that Indonesia is a country with many ethnicities, therefore, ethnicity is a very important and relevant topic to be discussed in many studies. In this regard, robust tests measuring ethnicity are needed to collect data and further the data can be validly interpreted. The purpose of this study was to psychometrically evaluate the effects of the method of using two-way items (favorable and unfavorable) on two scales, Ethnic Experience and Ethnic Identity. This evaluation will lead to an evaluation of possible new factors …

Preparing For Life In A Digital World: The Iea International Computer And Information Literacy Study 2018 International Report, Julian Fraillon, John Ainley, Wolfram Schulz, Tim Friedman, Daniel Duckworth

ICT - Digital Literacy

The second cycle of ICILS in 2018 (ICILS 2018) continued to investigate students’ computer and information literacy (CIL) and also investigated students’ computational thinking (CT). This dimension involves conceptualising problems (through algorithmic or systems thinking) and operationalising solutions (creating, implementing, and evaluating computer-based responses to problems). The inclusion of CT as an option in ICILS 2018 reflects recent interest by educators, researchers, and policymakers in the value of CT in schooling. ICILS 2018 studied how these components of digital competence related to each other and to the school and out-of-school contexts that support learning with and about computer technology. This …

International Computer And Information Literacy Study 2018: Assessment Framework, Julian Fraillon, John Ainley, Wolfram Schulz, Daniel Duckworth, Tim Friedman

Daniel Duckworth

The primary purpose of International Computer and Information Literacy Study (ICILS) 2018 is to assess systematically the capacities of students to use ICT productively for a range of different purposes, in ways that go beyond a basic use of ICT. ICILS 2018 includes authentic computer-based assessments that are administered to students in their eighth year of schooling. These generate data reflecting two dimensions of ICT-related capacities: computer and information literacy (CIL) and computational thinking (CT). ICILS 2018 investigates variations in CIL and CT between and within countries and the relationships between those constructs and student attributes (background characteristics and developed …

International Computer Andinformation Literacy Study 2018: Assessment Framework, Julian Fraillon, John Ainley, Wolfram Schulz, Daniel Duckworth, Tim Friedman

Dr Tim Friedman

The primary purpose of International Computer and Information Literacy Study (ICILS) 2018 is to assess systematically the capacities of students to use ICT productively for a range of different purposes, in ways that go beyond a basic use of ICT. ICILS 2018 includes authentic computer-based assessments that are administered to students in their eighth year of schooling. These generate data reflecting two dimensions of ICT-related capacities: computer and information literacy (CIL) and computational thinking (CT). ICILS 2018 investigates variations in CIL and CT between and within countries and the relationships between those constructs and student attributes (background characteristics and developed …

Iea International Computer And Information Literacy Study 2018 Assessment Framework, Julian Fraillon, John Ainley, Wolfram Schulz, Daniel Duckworth, Tim Friedman

Dr Wolfram Schulz

The primary purpose of International Computer and Information Literacy Study (ICILS) 2018 is to assess systematically the capacities of students to use ICT productively for a range of different purposes, in ways that go beyond a basic use of ICT. ICILS 2018 includes authentic computer-based assessments that are administered to students in their eighth year of schooling. These generate data reflecting two dimensions of ICT-related capacities: computer and information literacy (CIL) and computational thinking (CT). ICILS 2018 investigates variations in CIL and CT between and within countries and the relationships between those constructs and student attributes (background characteristics and developed …

Iea International Computer And Information Literacy Study 2018 Assessment Framework, Julian Fraillon, John Ainley, Wolfram Schulz, Daniel Duckworth, Tim Friedman

Julian Fraillon

The primary purpose of International Computer and Information Literacy Study (ICILS) 2018 is to assess systematically the capacities of students to use ICT productively for a range of different purposes, in ways that go beyond a basic use of ICT. ICILS 2018 includes authentic computer-based assessments that are administered to students in their eighth year of schooling. These generate data reflecting two dimensions of ICT-related capacities: computer and information literacy (CIL) and computational thinking (CT). ICILS 2018 investigates variations in CIL and CT between and within countries and the relationships between those constructs and student attributes (background characteristics and developed …

Iea International Computer And Information Literacy Study 2018 Assessment Framework, Julian Fraillon, John Ainley, Wolfram Schulz, Daniel Duckworth, Tim Friedman

ICT - Digital Literacy

The primary purpose of International Computer and Information Literacy Study (ICILS) 2018 is to assess systematically the capacities of students to use ICT productively for a range of different purposes, in ways that go beyond a basic use of ICT. ICILS 2018 includes authentic computer-based assessments that are administered to students in their eighth year of schooling. These generate data reflecting two dimensions of ICT-related capacities: computer and information literacy (CIL) and computational thinking (CT). ICILS 2018 investigates variations in CIL and CT between and within countries and the relationships between those constructs and student attributes (background characteristics and developed …

From Science Student To Scientist: Predictors And Outcomes Of Heterogeneous Science Identity Trajectories In College, Kristy A. Robinson, Tony Perez, Amy K. Nuttall, Cary J. Roseth, Lisa Linnenbrink-Garcia

STEMPS Faculty Publications

This 5-year longitudinal study investigates the development of science identity throughout college from an expectancy-value perspective. Specifically, heterogeneous developmental patterns of science identity across 4 years of college were examined using growth-mixture modeling. Gender, race/ethnicity, and competence beliefs (efficacy for science tasks, perceived competence in science) were modeled as antecedents, and participation in a science career after graduation was modeled as a distal outcome of these identity development trajectories. Three latent classes (High with Transitory Incline, Moderate-High and Stable, and Moderate-Low with Early Decline) were identified. Gender, race/ethnicity, and competence beliefs in the first year of college significantly predicted latent …

The Art Of Item Writing: Cross-Curricular Testing At Acer 1964-2014, Doug Mccurry, John Harding

ACER Research Monographs

This monograph is about the development of a distinctive style of cross-curricular testing that was developed at the Australian Council for Educational Research in the 1960s and 1970s, and is still in use in some testing programs presently. The impetus for this kind of test development was the decision of the Menzies government to award 10,000 scholarships each year for students to study at Years 11 and 12 through a test called The Commonwealth Secondary Scholarship Examination (CSSE). The testing program ran for 10 years, during which around one million candidates sat the test. This monograph attempts to demonstrate the …

Principles Of Good Practice In Learning Assessment, Australian Council For Educational Research (Acer), Unesco Institute For Statistics

Monitoring Learning

The Principles of Good Practice in Learning Assessment (GP-LA) are a central element of the international commitment to the management of Sustainable Development Goals (SDGs) 4 data quality for learning outcomes. The GP-LA is an independent articulation of good practices that accommodates the diversity of large-scale learning assessment activities being undertaken throughout the world. Within SDG 4 reporting processes it serves two purposes. First, it serves as the conceptual framework to evaluate the quality of large scale assessments and data from these assessments submitted for SDG 4 reporting. Second, the principles described in the GP-LA will support the diagnosis of …

Informed And Uninformed Naïve Assessment Constructors’ Strategies For Item Selection, Helenrose Fives, Nicole Barnes

Department of Educational Foundations Scholarship and Creative Works

We present a descriptive analysis of 53 naïve assessment constructors’ explanations for selecting test items to include on a summative assessment. We randomly assigned participants to an informed and uninformed condition (i.e., informed participants read an article describing a Table of Specifications). Through recursive thematic analyses of participants’ explanations, we identified 14 distinct strategies that coalesced into three families of strategies: Alignment, Item Evaluation, and Affective Evaluation. We describe the nature of the strategies and the degree to which participants used strategies with frequency and effect size analysis. Results can inform teacher education on assessment construction through explicit instruction in …

Conceptualizing And Validating The Character Virtues Index (Cvi), Mark Allen Liston

Dissertations

Can a valid, reliable measure of multi-dimensional adolescent character be developed? Goals were: 1) To construct a grid of trait lists by experts in Positive Psychology (PP), Character Education (CE), and Positive Youth Development (PYD) to create the Character Taxonomy as a conceptual basis; 2) to construct the Character Virtues Index (CVI) as a brief measure of the taxonomy’s traits; and 3) to validate CVI. The Character Taxonomy produced 18 traits that were hypothesized to cover the various dimensions of character. Two CVI field tests produced a reliable measure with 11 factors for a validation study. The validation study of …

Developing A Written Language Inventory For Deaf And Hard Of Hearing Students: A Systemic Functional Grammar Approach, Jennifer Renée Kilpatrick

Doctoral Dissertations

Deaf and hard of hearing (d/hh) students are extremely diverse in language development due to vast differences in residual hearing, response to hearing technologies, and exposure to American Sign Language. Writing is a struggle for these students who have delayed and limited access to English. Studies have found that d/hh students continue to lag behind their hearing peers in syntactic development. Unfortunately, current methods of writing assessment do not provide teachers with sufficient information regarding the syntactic development of d/hh students. This dissertation responds to the need for an assessment that is able to provide this information that is necessary …

How Do We Know If It Works? Measuring Outcomes In Bystander-Focused Abuse Prevention On Campuses, Alison C. Cares, Victoria L. Banyard, Mary M. Moynihan, Rebecca Warner

Sociology and Criminology Department Faculty Works

Objective: To address acknowledged limitations in the effectiveness of sexual and relationship abuse prevention strategies, practitioners have developed new tools that use a bystander framework (Lonsway et al, 2009). Evaluation of bystander-focused prevention requires measures, specific to the bystander approach, that assess changes over time in participants’ attitudes and behaviors. Few measures exist and more psychometric analyses are needed. We present analyses to begin to establish the psychometric properties of four new measures of bystander outcomes and their subscales. Method: We collected data from 948 first year college students on two campuses in the northeast United States. Items assessing attitudes …

Pat Science : Getting Good Assessment Down Pat, Geoff Masters

Prof Geoff Masters AO

A new school assessment resource in science from the Australian Council for Educational Research (ACER) has just been launched, following more than two years of development and trial testing. Called PAT Science, the new addition to the Progressive Achievement Tests series complements PAT Reading and PAT Maths, which are now widely used in Australian schools. PAT Science fills an identified need for better classroom information about the progress students make in their science learning between Year 3 and Year 10. By the time they reach 15 years of age, 13 percent of Australian students are classified as 'at risk' of …

Pisa 2009 Technical Report, John Cresswell (Ed)

OECD Programme for International Student Assessment (PISA)

PISA is methodologically highly complex, requiring intensive collaboration among many stakeholders. The successful implementation of PISA depends on the use, and sometimes further development, of state-of-the-art methodologies and technologies. The PISA 2009 Technical Report describes those methodologies, along with other features that have enabled PISA to provide high quality data to support policy formation and review. The descriptions are provided at a level that will enable review and, potentially, replication of the implemented procedures and technical solutions to problems. This report contains a description of the theoretical underpinning of the complex techniques used to create the PISA 2009 Database, which …

Alternative Scaling Models And Dependencies In Pisa, Greg Macaskill

OECD Programme for International Student Assessment (PISA)

This paper examines trends between PISA 2003 and PISA 2006 for the reading and mathematics scales for the OECD countries using both the standard one parameter (1PL) model that has been used in published OECD reports, and a generalised partial credit model, as implemented in ACER ConQuest, which estimates optimal values for each possible score on an item.

Best Start 2008 : Kindergarten Literacy Assessment, Department Of Education And Training, Nsw : Data Analysis Report, June 2008, Marion O. Meiers, Siek Toon Khoo

Marion Meiers

Development And Validation Of The Counterfactual Thinking For Negative Events Scale, Tarika Daftary Kapur, Mark S. Rye, Melissa B. Cahoon, Rahan S. Ali

Department of Justice Studies Faculty Scholarship and Creative Works

We examined the psychometric properties of the newly created Counterfactual Thinking for Negative Events Scale (CTNES) in two studies involving university undergraduates. In Study 1 (N = 634), factor analysis revealed four subscales that correspond with various types of counterfactual thinking: Nonreferent Downward, Other-Referent Upward, Self-Referent Upward, and Nonreferent Upward. The subscales were largely orthogonal and had adequate internal consistency and test–retest reliability. The CTNES subscales were positively correlated with a traditional method of assessing counterfactual thinking and were related as expected to contextual aspects of the negative event, negative affect, and cognitive style. In Study 2 (N …

Developing Tests And Questionnaires For A National Assessment Of Educational Achievement, Prue Anderson, George Morgan

Assessment and Reporting

This book addresses the design of two types of data collection instruments: student achievement tests and background questionnaires. Part 1 covers the development of an assessment framework and a test blueprint, item writing, pretesting, and final test layout. Part 2 delineates comparable stages and activities in the construction of background questionnaires, which are used to gather information from students, teachers, head teachers, or parents on variables that might help explain differences in student performance on the achievement test. Part 3 describes how to design a manual for test administration to help ensure that all students take the test under standardised …

Developing Tests And Questionnaires For A National Assessment Of Educational Achievement, Prue Anderson, George Morgan

Prue Anderson

The authors introduce readers to the activities involved in the development of achievement tests, including developing as assessment framework, writing multiple choice and constructed response type items, pretesting, producing test booklets, and handscoring items. A section on questionnaire construction features designing questionnaires, writing questions, coding responses, and linking questionnaire and test score data. The final section covers the development of a test administration manual, selecting test administrators, and contacting sampled schools. A companion CD contains examples of released items from national and international tests, sample questionnaires, and administrative manuals. [Back cover, ed]

Development And Validation Of The Information Literacy Assessment Scale For Education (Ilas-Ed), Penny Beile

Penny Beile

Development And Validation Of The Beile Test Of Information Literacy For Education (B-Tiled), Penny Beile

Penny Beile

Response To 'Cautions On Oecd's Recent Educational Survey (Pisa), Ray Adams

Prof Ray Adams

This paper refers to the work of S J Prais who questioned the outcomes of the Organisation for Economic Cooperation and Development's PISA survey of the reading, mathematics and science attainments of fifteen year-olds. Prais suggested that methodological flaws in Programme for International Student Assessment (PISA) had resulted in an apparent improvement in the attainment of British students, particularly when compared to their Swiss and German counterparts. This paper responds to Prais's criticisms, noting that when Prais's conjectures are tested with empirical data they are not supported. The author argues that many of Prais's criticisms are due to an incomplete …

Fair And Meaningful Measures? A Review Of Examination Procedures In The Nsw Higher School Certificate, Geoff N. Masters

Assessment and Reporting

In September 1997 the New South Wales government introduced a number of significant reforms to the Higher School Certificate (HSC). The broad objectives of the 'new' HSC, which was implemented for the first time in 2000-01, are to increase the rigour and quality of the HSC curriculum; ensure HSC marks fairly reflect the standards achieved by students; provide more informative reports of students' HSC achievements; and enhance the chances for more equitable educational outcomes. Although the new HSC incorporates reforms of both HSC curricula and methods of assessing and reporting student achievement, the focus of this review is on new …

Pisa 2000 Technical Report, Ray Adams (Ed), Margaret Wu (Ed)

Professor Ray Adams

The Programme for International Student Assessment (PISA) was launched by the OECD to provide internationally comparable evidence on student performance. This technical report is the product of a collaborative effort between the countries participating in PISA and the experts and the institutions working within the framework of the International PISA Consortium.

Better Than Beige : Designing Assessment Tasks To Enhance Learning And Measure Growth In The Early Years Of School, Prue Anderson, Marion Meiers

Prue Anderson

'I think of our school as beige. We tend to be the average of 'like' schools. Can we do better than that?' In order to promote growth in student learning and demonstrate achievement schools require rich assessment tasks that enhance the teaching and learning cycle and provide a valid and reliable measure of growth. This paper examines the five linked sets of assessment tasks that have been developed to progressively measure growth in literacy and numeracy in the first three years of school. The tasks were designed for the ACER Longitudinal Literacy and Numeracy Survey (LLANS). The tasks are based …

Conflicting Perspectives On The Role Of Test Preparation In Relation To Learning, Tom Lumley, Bernadette Stoneman

Dr Tom Lumley

Reports on the development of a learning package for a newly introduced test in Hong Kong, the Graduating Students' Language Proficiency Assessment (GSLPA)-English. Examines reactions of teachers and students to the learning package during piloting of the materials.