Open Access. Powered by Scholars. Published by Universities.®

Education Commons

Open Access. Powered by Scholars. Published by Universities.®

Articles 1 - 23 of 23

Full-Text Articles in Education

Designing Chemistry Practice Exams For Enhanced Benefits. An Instrument For Comparing Performance And Mental Effort Measures, Kristen L. Murphy, Thomas Holme Jan 2015

Designing Chemistry Practice Exams For Enhanced Benefits. An Instrument For Comparing Performance And Mental Effort Measures, Kristen L. Murphy, Thomas Holme

Chemistry Publications

The emergence of students using cell phones and particularly the photographic capabilities of smartphones to cheat on exams is noted. The specific threat this form of academic misconduct poses to the ACS Examinations Institute is addressed in particular.


Improving Instructional Design With Better Analysis Of Assessment Data, Kristen L. Murphy, Thomas Holme Jan 2014

Improving Instructional Design With Better Analysis Of Assessment Data, Kristen L. Murphy, Thomas Holme

Chemistry Publications

As more instructors articulate learning objectives for their students within one course, or academic staff collaborate to articulate learning outcomes for programs, a robust means to assess student performance within these becomes increasingly important. The Examinations Institute of the American Chemical Society (ACS), Division of Chemical Education, has recently published content maps that utilise a structure of subdiscipline-independent fundamental concepts narrowing down to content details that are specific to subdisciplines. This structure has then been utilised to align items and can be used to assess student performance throughout a program. Learning objectives that are designed for a course can then ...


Exploring The Apparent Motivational Impact Of Resurrection Points From Final Exam Performance, Jeffrey R. Raker, Thomas Holme Jan 2014

Exploring The Apparent Motivational Impact Of Resurrection Points From Final Exam Performance, Jeffrey R. Raker, Thomas Holme

Chemistry Publications

This chapter examines the motivational impact of resurrection points; a systematic method for encouraging students to earn back missed points on semester exams through performance on the final exam. This method for utilizing hour exams as a type of formative assessment was explored for three courses across three years. Four of the nine individual courses sampled offered resurrection points. A student’s grade trajectory based on semester exam performance was used to predict how well a student needed to perform on the final exam to receive a particular course grade. The need to over or under perform based on semester ...


Results Of A National Survey Of Biochemistry Instructors To Determine The Prevalence And Types Of Representations Used During Instruction And Assessment, Kimberly J. Linenberger, Thomas Holme Jan 2014

Results Of A National Survey Of Biochemistry Instructors To Determine The Prevalence And Types Of Representations Used During Instruction And Assessment, Kimberly J. Linenberger, Thomas Holme

Chemistry Publications

Chemists and chemistry educators have long sought meaningful ways to visualize fundamentally abstract components, such as atoms and molecules, of their trade. As technology has improved, computer-based visualization methods have infused both research and education in chemistry. Biochemistry, in particular, has become highly dependent on ways that large molecular systems can be represented, and how to best focus attention on the most critical aspects of the molecular system. To better understand the current state of educational efforts related to visual literacy, a needs assessment was developed and administered to a national sample of biochemistry instructors at four-year institutions (N = 536 ...


Identifying Differential Performance In General Chemistry: Differential Item Functioning Analysis Of Acs General Chemistry Trial Tests, Lisa Kendhammer, Thomas Holme, Kristen Murphy Jan 2013

Identifying Differential Performance In General Chemistry: Differential Item Functioning Analysis Of Acs General Chemistry Trial Tests, Lisa Kendhammer, Thomas Holme, Kristen Murphy

Chemistry Publications

The development of high-quality assessments can be an intensive process. At the American Chemical Society Examinations Institute (ACS-EI), this follows a general process of test design and content mapping, item construction, trial testing, item analysis, and final test setting. The item analysis portion of this procedure is an important step in using field-testing results to select the best items. This selection is based on validity analysis by experts, both field-test users and test writers, and by students in field testing. The traditional item analysis of the ACS-EI now includes a differential item functioning analysis with subgroups by gender when a ...


Validating Chemistry Faculty Members’ Self-Reported Familiarity With Assessment Terminology, Mary Emenike, Jeffrey R. Raker, Thomas Holme Jan 2013

Validating Chemistry Faculty Members’ Self-Reported Familiarity With Assessment Terminology, Mary Emenike, Jeffrey R. Raker, Thomas Holme

Chemistry Publications

With the increasing emphasis placed upon chemistry instructors and departments to assess and evaluate their courses and curricula, understanding the structure of chemistry faculty members’ knowledge and use of assessment terms and concepts can shed light on potential areas for targeted professional development. Survey research that might accomplish this objective often relies on self-reported responses from the target audience, and such information is sometimes difficult to assess in terms of validity. As an example of an internal mechanism to help establish validity, it is possible to include an “internal standard” item early in the survey. For the sake of understanding ...


Lessons Learned From Collaborations In Chemistry Assessment Across Universities: Challenges In Transfer And Scale, Pamela L. Paek, Thomas Holme Jan 2013

Lessons Learned From Collaborations In Chemistry Assessment Across Universities: Challenges In Transfer And Scale, Pamela L. Paek, Thomas Holme

Chemistry Publications

This chapter reviews a collaborative effort to cross-pollinate and share work around chemistry assessments across several universities. The goal was to find ways to synthesize separate projects and capitalize on applying developed instruments and assessments beyond a single university, and in new situations, to increase scale and check for generalizability. Discussion of the successes and challenges of scale and transfer of the collaboration is detailed in this chapter.


Using Structural Equation Modeling To Understand Chemistry Faculty Familiarity Of Assessment Terminology: Results From A National Survey, Jeffrey R. Raker, Mary Emenike, Thomas Holme Jan 2013

Using Structural Equation Modeling To Understand Chemistry Faculty Familiarity Of Assessment Terminology: Results From A National Survey, Jeffrey R. Raker, Mary Emenike, Thomas Holme

Chemistry Publications

Chemistry departments have felt pressure in recent years to produce quality data on student achievement of learning outcomes. External (e.g., accreditation agencies) and internal (e.g., academic deans) entities are demanding regular review of student achievement. It is thus necessary for the chemistry community to develop valid and reliable instruments to assess student learning. With chemistry faculty members’ integration into assessment practices, it is important that these faculty members have a sufficient understanding of the quality of and limitations to the interpretation of assessment data. As part of a larger national survey, 1505 chemistry faculty members from a diverse ...


A Trajectory Of Reform In General Chemistry For Engineering Students, Thomas Holme, Heather Caruthers Jan 2013

A Trajectory Of Reform In General Chemistry For Engineering Students, Thomas Holme, Heather Caruthers

Chemistry Publications

This chapter considers efforts to enhance learning within the general chemisty course taken by pre-engineering students. Because this course is inherently offered as a service course, often for students in a different college from the Chemistry Department at a university, there are both constraints and opportunities related to the manner in which reform can be enacted. Efforts spanning roughly 15 years are described and an emphasis on the nature of problem-solving within the course emerges as a common theme. The issue of student motivation is also considered with pre-engineering students serving as a prototype of a type of student who ...


Importance Of Considering Longitudinal Trajectories In Education Reform Efforts, Thomas Holme, Melanie M. Cooper, Pratibha Varma-Nelson Jan 2013

Importance Of Considering Longitudinal Trajectories In Education Reform Efforts, Thomas Holme, Melanie M. Cooper, Pratibha Varma-Nelson

Chemistry Publications

This chapter introduces the collection of articles in this book with an emphasis on why it is important to consider the way that educational research and reform efforts change over time. The importance of considering a longitudinal view of education reform is emphasized in two ways. First, the context of this work relative to current literature is considered. Second, the idea of a greater focus on the longer-term trajectories of reform efforts in considered in terms of suggestions for the future of chemistry education.


The Acs Exams Institute Undergraduate Chemistry Anchoring Concepts Content Map Ii: Organic Chemistry, Jeffrey R. Raker, Thomas Holme, Kristen L. Murphy Jan 2013

The Acs Exams Institute Undergraduate Chemistry Anchoring Concepts Content Map Ii: Organic Chemistry, Jeffrey R. Raker, Thomas Holme, Kristen L. Murphy

Chemistry Publications

As a way to assist chemistry departments with programmatic assessment of undergraduate chemistry curricula, the ACS Examinations Institute is devising a map of the content taught throughout the undergraduate curriculum. The structure of the map is hierarchal, with large grain size at the top and more content detail as one moves “down” the levels of the map, of which there are four levels total. This paper presents these four levels of the map with reference to second-year, organic chemistry.


Results From A National Needs Assessment Survey: A View Of Assessment Efforts Within Chemistry Departments, Mary Elizabeth Emenike, Jacob Schroeder, Kristen L. Murphy, Thomas Holme Jan 2013

Results From A National Needs Assessment Survey: A View Of Assessment Efforts Within Chemistry Departments, Mary Elizabeth Emenike, Jacob Schroeder, Kristen L. Murphy, Thomas Holme

Chemistry Publications

As is true for virtually all of higher education, chemistry departments are often required to provide evidence of student learning at both course and curricular levels through evaluation and assessment. The ACS Exams Institute conducted a needs assessment survey of 1500 chemistry faculty members from across the country to investigate motivation, role, instrument use, and challenges associated with assessment efforts. For the more than 70% of participants who reported departmental efforts related to assessment, these findings emerged: motivations were primarily external factors related to accreditation and certification, ACS Exams and in-house exams were the most common instruments used, and time ...


Polytomous Versus Dichotomous Scoring On Multiple-Choice Examinations: Development Of A Rubric For Rating Partial Credit, Megan L. Grunert, Jeffrey R. Raker, Kristen L. Murphy, Thomas Holme Jan 2013

Polytomous Versus Dichotomous Scoring On Multiple-Choice Examinations: Development Of A Rubric For Rating Partial Credit, Megan L. Grunert, Jeffrey R. Raker, Kristen L. Murphy, Thomas Holme

Chemistry Publications

The concept of assigning partial credit on multiple-choice test items is considered for items from ACS Exams. Because the items on these exams, particularly the quantitative items, use common student errors to define incorrect answers, it is possible to assign partial credits to some of these incorrect responses. To do so, however, it becomes vital that instructors reach general agreement as to the level of partial credit. Using workshops with instructors, ACS Exams has identified reasons why partial credit could be assigned in an exam set of 70 test items. With partial-credit assignments thus established, polytomous scoring is applied and ...


Adaptation Of An Instrument For Measuring The Cognitive Complexity Of Organic Chemistry Exam Items, Jeffrey R. Raker, Jaclyn M. Trate, Thomas Holme, Kristen Murphy Jan 2013

Adaptation Of An Instrument For Measuring The Cognitive Complexity Of Organic Chemistry Exam Items, Jeffrey R. Raker, Jaclyn M. Trate, Thomas Holme, Kristen Murphy

Chemistry Publications

Experts use their domain expertise and knowledge of examinees’ ability levels as they write test items. The expert test writer can then estimate the difficulty of the test items subjectively. However, an objective method for assigning difficulty to a test item would capture the cognitive demands imposed on the examinee as well as be assignable by any domain expert familiar with the examinee group. One such instrument for assigning objective complexity of general chemistry exam items has already been reported. A revised instrument for assigning objective complexity of organic chemistry exam items is presented including the reliability and validity studies ...


Investigating Factors That Influence Item Performance On Acs Exams, Jacob Schroeder, Kristen L. Murphy, Thomas Holme Jan 2012

Investigating Factors That Influence Item Performance On Acs Exams, Jacob Schroeder, Kristen L. Murphy, Thomas Holme

Chemistry Publications

General chemistry tests from the Examinations Institute of the Division of Chemical Education of the American Chemical Society have been analyzed to identify factors that may influence how individual test items perform. In this paper, issues of item order (position within a set of items that comprise a test) and answer order (position of correct answer relative to incorrect distractors) are discussed. Answer order is identified as potentially important, particularly for conceptually based items. When the correct answer appears earlier among the answer choices, there is some greater propensity for student performance to be better. Item-order effects are also possible ...


The Acs Exams Institute Undergraduate Chemistry Anchoring Concepts Content Map I: General Chemistry, Thomas Holme, Kristen L. Murphy Jan 2012

The Acs Exams Institute Undergraduate Chemistry Anchoring Concepts Content Map I: General Chemistry, Thomas Holme, Kristen L. Murphy

Chemistry Publications

To provide tools for programmatic assessment related to the use of ACS Exams in undergraduate chemistry courses, the ACS Exams Institute has built a content map that applies to the entire undergraduate curriculum. At the top two levels, the grain size of the content classification is large and spans the entire undergraduate curriculum. At the bottom two levels, the grain size of the content is more fine and tuned to specific course levels of the curriculum. This paper presents all four levels of the map as identified for first-year general chemistry.


Building The Acs Exams Anchoring Concept Content Map For Undergraduate Chemistry, Kristen L. Murphy, Thomas Holme, April Zenisky, Karen Knauss Jan 2012

Building The Acs Exams Anchoring Concept Content Map For Undergraduate Chemistry, Kristen L. Murphy, Thomas Holme, April Zenisky, Karen Knauss

Chemistry Publications

The ability to coherently assess content knowledge throughout an entire undergraduate career represents a significant advantage for programmatic assessment strategies. Chemistry, as a discipline, has an unusual tool in this regard because of the nationally standardized exams from the ACS Exams Institute. These exams are norm-referenced and allow chemistry departments to make comparisons between the performance of their own students relative to national samples; however, currently there appears to be no systematic means for noting students’ content knowledge growth over a four-year degree. The Exams Institute is undertaking the task of organizing content along an anchoring concept or “big ideas ...


A Valid And Reliable Instrument For Cognitive Complexity Rating Assignment Of Chemistry Exam Items, Karen Knaus, Kristen Murphy, Anja Blecking, Thomas Holme Jan 2011

A Valid And Reliable Instrument For Cognitive Complexity Rating Assignment Of Chemistry Exam Items, Karen Knaus, Kristen Murphy, Anja Blecking, Thomas Holme

Chemistry Publications

The design and use of a valid and reliable instrument for the assignment of cognitive complexity ratings to chemistry exam items is described in this paper. Use of such an instrument provides a simple method to quantify the cognitive demands of chemistry exam items. Instrument validity was established in two different ways: statistically significant correlations between expert-based cognitive complexity ratings and student performance (as measured through statistical difficulty of items), and statistically significant correlations between expert-based cognitive complexity ratings and student mental effort ratings. Key benefits associated with instrument use include an enhanced understanding of the cognitive complexity of chemistry ...


Assessing Conceptual Versus Algorithmic Knowledge: Are We Engendering New Myths In Chemical Education?, Thomas Holme, Kristen Murphy Jan 2011

Assessing Conceptual Versus Algorithmic Knowledge: Are We Engendering New Myths In Chemical Education?, Thomas Holme, Kristen Murphy

Chemistry Publications

Studies over the past two decades have emphasized a gap between relatively weak student performance on conceptual items versus traditional items. The ACS Examinations Institute has released a pair of exams for general chemistry in which items are intentionally paired with one conceptual and one traditional item. This paper describes data from statistical analysis of the item pairs, and notes that for these exams, this gap is not evident, as overall performance is better on conceptual items. Possible implications for teaching and for research in Chemistry Education are noted.


Assessing Conceptual And Algorithmic Knowledge In General Chemistry With Acs Exams, Thomas Holme, Kristen L. Murphy Jan 2011

Assessing Conceptual And Algorithmic Knowledge In General Chemistry With Acs Exams, Thomas Holme, Kristen L. Murphy

Chemistry Publications

In 2005, the ACS Examinations Institute released an exam for first-term general chemistry in which items are intentionally paired with one conceptual and one traditional item. A second-term, paired-questions exam was released in 2007. This paper presents an empirical study of student performances on these two exams based on national samples of students who took the exams as part of a general chemistry course sequence. Psychometric data for student performances are presented in terms of classical difficulty and discrimination indexes, as well as item characteristic curves, as are more commonly used in item response theory. Having these data provided for ...


Assessment Data And Decision Making In Teaching, Thomas Holme Jan 2011

Assessment Data And Decision Making In Teaching, Thomas Holme

Chemistry Publications

Finding avenues to collect and use data in our teaching parallel to the way we use data in research may be an important way to improve educational outcomes.


Book Review Of Quantum Chemistry, 2nd Edition, Thomas Holme Jan 2010

Book Review Of Quantum Chemistry, 2nd Edition, Thomas Holme

Chemistry Publications

This article presents a review of the second edition of Quantum Chemistry by D.A. McQuarrie.


The Impact Of Nanoscience Context On Multiple Choice Chemistry Items, Karen Knaus, Kristen Murphy, Thomas Holme Jan 2009

The Impact Of Nanoscience Context On Multiple Choice Chemistry Items, Karen Knaus, Kristen Murphy, Thomas Holme

Chemistry Publications

Modern chemistry topics are often introduced into classrooms long before they appear in standardized exams. This paper investigates the role of a nanoscience context in multiple choice items by using a comparative description of the cognitive load effects of such items on a practice exam that was given to students in 2nd-semester general chemistry and 1-semester pre-engineering general chemistry. It includes a classroom comparison study of performance and mental effort analyses in twelve chemistry content areas including a nanoscience and materials context category. In addition, cognitive load effects of paired items in four subcategories were evaluated. Results from ...