Open Access. Powered by Scholars. Published by Universities.®
- Publication Year
Articles 1 - 21 of 21
Full-Text Articles in Education
Toward A Unified Validation Framework In Mixed Methods Research, Amy B. Dellinger, Nancy L. Leech
Toward A Unified Validation Framework In Mixed Methods Research, Amy B. Dellinger, Nancy L. Leech
Nancy Leech
The primary purpose of this article is to further discussions of validity in mixed methods research by introducing a validation framework to guide thinking about validity in this area. To justify the use of this framework, the authors discuss traditional terminology and validity criteria for quantitative and qualitative research, as well as present recently published validity terminology for mixed methods research. The authors discuss the rationale for their framework and how it unifies thinking about validity in mixed methods research. Finally, they discuss how the framework can be used.
Group Of National Experts On The Ahelo Feasibility Study : Engineering Assessment Framework : Ahelo Feasibility Study, Hamish Coates, Sarah Richardson, Jacob Pearce, Julian Fraillon
Group Of National Experts On The Ahelo Feasibility Study : Engineering Assessment Framework : Ahelo Feasibility Study, Hamish Coates, Sarah Richardson, Jacob Pearce, Julian Fraillon
Julian Fraillon
This AHELO Engineering Assessment Framework describes and illustrates the domain of engineering competency that is used in the AHELO Feasibility Study. The framework gives an organisational structure for the domain in terms of engineering knowledge, processes and contexts; describes the types of assessment items that have been developed; and details how reporting will be carried out. - p.4
Assessing Vocational Competencies In Civil Engineering: Lessons From Ahelo For Future Practice, Jacob Pearce
Assessing Vocational Competencies In Civil Engineering: Lessons From Ahelo For Future Practice, Jacob Pearce
Dr Jacob Pearce
There has been much interest in the notion of vocational competencies of late. The pressing question, however, is how to measure vocational competencies. In order to contribute to this ‘how’, the results of the Civil Engineering strand of the recent Assessment of Higher Education Learning Outcomes (AHELO) Feasibility Study are discussed. The focus of the assessment instrument was explicitly on “above content” areas of the domain, and the assessment framework provides an example of a construct which articulates certain domain-specific competence components. This paper draws specific attention the Constructed Response Tasks (CRTs) in the instrument and discusses the results of …
Scaling Procedures For Icils Test Items, Eveline Gebhardt, Wolfram Schulz
Scaling Procedures For Icils Test Items, Eveline Gebhardt, Wolfram Schulz
Dr Wolfram Schulz
This chapter describes the procedures used to analyze and scale the ICILS test items that were administered to measure students’ computer and information literacy (CIL).
Measuring Learning Growth In A World Of Universal Education, Ross Turner
Measuring Learning Growth In A World Of Universal Education, Ross Turner
Ross Turner
ACER is leading the development of new ‘learning metrics’ – tools for measuring learning growth – as Ross Turner explains.
National Assessment Program : Ict Literacy 2008 : Year 6 And Year 10 : Technical Report, Chris Freeman, John Ainley, Julian Fraillon, Eveline Gebhardt, Kate O'Malley, Wolfram Schulz
National Assessment Program : Ict Literacy 2008 : Year 6 And Year 10 : Technical Report, Chris Freeman, John Ainley, Julian Fraillon, Eveline Gebhardt, Kate O'Malley, Wolfram Schulz
Dr Wolfram Schulz
This report describes the technical aspects of the National ICT Literacy Sample Assessment and summarises the main activities involved in the data collection, the data collection instruments and the analysis and reporting of the data. Chapter 2 summarises the development of the assessment domain and describes the process of item development and construction of the instruments. Chapter 3 reviews the sample design and describes the sampling process. Chapter 3 also describes the weighting procedures that were implemented to derive population estimates. Chapter 4 summarises the field administration and data management procedures, including quality control and the cleaning and coding of …
National Assessment Program : Ict Literacy 2008 : Year 6 And Year 10 : Technical Report, Chris Freeman, John Ainley, Julian Fraillon, Eveline Gebhardt, Kate O'Malley, Wolfram Schulz
National Assessment Program : Ict Literacy 2008 : Year 6 And Year 10 : Technical Report, Chris Freeman, John Ainley, Julian Fraillon, Eveline Gebhardt, Kate O'Malley, Wolfram Schulz
Julian Fraillon
This report describes the technical aspects of the National ICT Literacy Sample Assessment and summarises the main activities involved in the data collection, the data collection instruments and the analysis and reporting of the data. Chapter 2 summarises the development of the assessment domain and describes the process of item development and construction of the instruments. Chapter 3 reviews the sample design and describes the sampling process. Chapter 3 also describes the weighting procedures that were implemented to derive population estimates. Chapter 4 summarises the field administration and data management procedures, including quality control and the cleaning and coding of …
National Assessment Program : Ict Literacy 2008 : Year 6 And Year 10 : Technical Report, Chris Freeman, John Ainley, Julian Fraillon, Eveline Gebhardt, Kate O'Malley, Wolfram Schulz
National Assessment Program : Ict Literacy 2008 : Year 6 And Year 10 : Technical Report, Chris Freeman, John Ainley, Julian Fraillon, Eveline Gebhardt, Kate O'Malley, Wolfram Schulz
Dr John Ainley
This report describes the technical aspects of the National ICT Literacy Sample Assessment and summarises the main activities involved in the data collection, the data collection instruments and the analysis and reporting of the data. Chapter 2 summarises the development of the assessment domain and describes the process of item development and construction of the instruments. Chapter 3 reviews the sample design and describes the sampling process. Chapter 3 also describes the weighting procedures that were implemented to derive population estimates. Chapter 4 summarises the field administration and data management procedures, including quality control and the cleaning and coding of …
Mapping Student Achievement, Geoff Masters, Ray Adams, Jan Lokan
Mapping Student Achievement, Geoff Masters, Ray Adams, Jan Lokan
Prof Ray Adams
The analyses, publications and reports of quantitative data in education and the social sciences usually omit basic information on the construct measured. Probabilistic models for test data make it possible to delineate coherent and richly described measurement continua that facilitate interpretation of student achievement. The potential of conjoint measurement to bring about fundamental advances in educational testing practice lies in part in the opportunities it provides to build useful maps of learning domains and to use those maps in communicating student achievements. This chapter presents two applications of conjoint measurement aimed at constructing and describing achievement variables, developing insights into …
Extended Mantel-Haenszel Procedure For Dif Detection : A Note On Its Implementation In Acer Conquest, Xiaoxun Sun
Extended Mantel-Haenszel Procedure For Dif Detection : A Note On Its Implementation In Acer Conquest, Xiaoxun Sun
Xiaoxun Sun
Many methods have been developed for studying Differential Item Functioning (DIF). An approach developed by Holland and Thayer (1988) adapted the Mantel-Haenszel (MH) statistic (Mantel and Haenszel 1959). The author of this chapter has implemented MH procedure and its extension in an experimental version of ACER ConQuest which can handle DIF analysis for multiple focal groups and polytomous items. This chapter compares the extended MH procedure in DIF detection to the standardized item difficulty difference method based on the Rasch estimates for both dichotomous and polytomous items. The use of MH method in DIF analysis with multiple focal groups is …
National Assessment Program : Ict Literacy Technical Report, Eveline Gebhardt, Julian Fraillon, Wolfram Schulz, Kate O'Malley, Chris Freeman, Martin Murphy, John Lindsey
National Assessment Program : Ict Literacy Technical Report, Eveline Gebhardt, Julian Fraillon, Wolfram Schulz, Kate O'Malley, Chris Freeman, Martin Murphy, John Lindsey
Martin Murphy
This report is concerned with procedures, processes and technical aspects of the National Assessment Program - ICT Literacy 2011 Literacy (NAP - ICTL 2011) and should be read in conjunction with the Public Report from Literacy NAP - ICTL 2011 which focuses on results and interpretation of results from that assessment. The first cycle of the NAP - ICTL was held in 2005 and provided the baseline against which future performance would be compared. The second cycle of the program was conducted in 2008 and was the first cycle where trends in performance were able to be examined.
Group Of National Experts On The Ahelo Feasibility Study : Engineering Assessment Framework : Ahelo Feasibility Study, Hamish Coates, Sarah Richardson, Jacob Pearce, Julian Fraillon
Group Of National Experts On The Ahelo Feasibility Study : Engineering Assessment Framework : Ahelo Feasibility Study, Hamish Coates, Sarah Richardson, Jacob Pearce, Julian Fraillon
Dr Jacob Pearce
This AHELO Engineering Assessment Framework describes and illustrates the domain of engineering competency that is used in the AHELO Feasibility Study. The framework gives an organisational structure for the domain in terms of engineering knowledge, processes and contexts; describes the types of assessment items that have been developed; and details how reporting will be carried out. - p.4
Mapping Student Achievement, Geoff Masters, Ray Adams, Jan Lokan
Mapping Student Achievement, Geoff Masters, Ray Adams, Jan Lokan
Prof Geoff Masters AO
The analyses, publications and reports of quantitative data in education and the social sciences usually omit basic information on the construct measured. Probabilistic models for test data make it possible to delineate coherent and richly described measurement continua that facilitate interpretation of student achievement. The potential of conjoint measurement to bring about fundamental advances in educational testing practice lies in part in the opportunities it provides to build useful maps of learning domains and to use those maps in communicating student achievements. This chapter presents two applications of conjoint measurement aimed at constructing and describing achievement variables, developing insights into …
The Partial Credit Model And Null Categories, M Wilson, Geoff Masters
The Partial Credit Model And Null Categories, M Wilson, Geoff Masters
Prof Geoff Masters AO
A category where the frequency of responses is zero, either for sampling or structural reasons, will be called anull category. One approach for ordered polytomous item response models is to downcode the categories (i.e., reduce the score of each category above the null category by one), thus altering the relationship between the substantive framework and the scoring scheme for items with null categories. It is discussed why this is often not a good idea, and a method for avoiding the problem is described for the partial credit model while maintaining the integrity of the original response framework. This solution is …
The Analysis Of Partial Credit Scoring, Geoff Masters
The Analysis Of Partial Credit Scoring, Geoff Masters
Prof Geoff Masters AO
This article discusses a range of issues in the practical application of an item response theory (IRT) method for partial credit scoring. After a brief discussion of partial credit scoring as an alternative to right-wrong scoring in the measurment of educational achievement, an IRT model for partial credit analysis is developed and described. This model is presented as a straightforward and logical application of Rasch's dichotomous model to a sequence of ordered response alternatives. The distinctive nature of the item parameters in the model is described and these parameters are contrasted with two more familiar sets of parameters: Thurstone thresholds …
Educational Measurement: Prospects For Research And Innovation, Geoff Masters
Educational Measurement: Prospects For Research And Innovation, Geoff Masters
Prof Geoff Masters AO
No abstract provided.
Common Person Equating With The Rasch Model, Geoff Masters
Common Person Equating With The Rasch Model, Geoff Masters
Prof Geoff Masters AO
Two procedures, one based on item difficulties, the other based on person abilities, were used to equate 14 forms of a reading comprehension test using the Rasch model. These forms had no items in common. For practical purposes, the two procedures produced equivalent results. An advantage of common person equating for testing the unidimensionality assumption is pointed out, and the need for caution in interpreting tests of common item invariance is stressed.
The Essential Process In A Family Of Measurement Models, Geoff Masters, Benjamin Wright
The Essential Process In A Family Of Measurement Models, Geoff Masters, Benjamin Wright
Prof Geoff Masters AO
Five members of the Rasch family of latent trait models which have appeared more or less independently in the literature are brought together and identified as one model. In addition to sharing the distinguishing characteristic of the dichotomous Rasch model separable person and item parameters and hence sufficient statistics all five models share a common algebraic form and have as their basic element the fundamental process defined by Rasch's simple logistic expression. In these models, the sufficient statistics for person and item parameters are counts of events constructed to be indicative of the variable being measured, and the measures they …
Measuring Attitude To School With A Latent Trait Model, Geoff Masters, N Hyde
Measuring Attitude To School With A Latent Trait Model, Geoff Masters, N Hyde
Prof Geoff Masters AO
A latent trait model for rating scales is used to ana lyze responses to an attitude-to-school questionnaire as part of an evaluation of projects operating in 10 West ern Australian schools under the Australian Priority Schools Program. The invariance of item parameter estimates over the 10 schools is examined, and varia tions in item estimates from school to school are stud ied in the light of the different projects operating in these schools. Results show how the investigation of items that do not retain their difficulties from group to group can provide valuable insight into the ways in which calibration …
Dicot: Analysing Classroom Tests With The Rasch Model, Geoff Masters
Dicot: Analysing Classroom Tests With The Rasch Model, Geoff Masters
Prof Geoff Masters AO
A computer program (DICOT) for the Rasch analysis of classroom tests is described. Results are presented in a simple, self-explanatory form, and person ability and item difficulty estimates are expressed in a metric like the one with which teachers and parents are already familiar. Person and item fit statistics provide an opportunity to diagnose strengths and weaknesses of individual children and to identify items which are problematic.
Constructing An Item Bank Using Partial Credit Scoring, Geoff Masters
Constructing An Item Bank Using Partial Credit Scoring, Geoff Masters
Prof Geoff Masters AO
A method for banking test items scored in several ordered response categories is described. Each item is seen as an ordered sequence of steps, and test forms are equated using the estimated difficulties of the steps in their shared items. Procedures for analyzing the internal consistency of individual links and for analyzing the coherence of an entire linking structure are described. The methodology is used to link six forms of a mathematics problem solving test.