Open Access. Powered by Scholars. Published by Universities.®

Education Commons

Open Access. Powered by Scholars. Published by Universities.®

Articles 1 - 22 of 22

Full-Text Articles in Education

Updating The General Chemistry Anchoring Concepts Content Map, Thomas Holme, Cynthia Luxford, Kristen Murphy Jan 2015

Updating The General Chemistry Anchoring Concepts Content Map, Thomas Holme, Cynthia Luxford, Kristen Murphy

Chemistry Publications

The previously published Anchoring Concepts Content Map (ACCM) from the ACS Examinations Institute is updated. Through the development process of the ACCM for other subdisciplines, changes have been made at the top two levels, and these changes need to be reflected in the previously published maps. In addition, a large-scale project to align test items from ACS Exams over the past 20 years revealed specific omissions in the initially published General Chemistry ACCM.


Biochemistry Instructors’ Views Toward Developing And Assessing Visual Literacy In Their Courses, Kimberly J. Linenberger, Thomas Holme Jan 2015

Biochemistry Instructors’ Views Toward Developing And Assessing Visual Literacy In Their Courses, Kimberly J. Linenberger, Thomas Holme

Chemistry Publications

Biochemistry instructors are inundated with various representations from which to choose to depict biochemical phenomena. Because of the immense amount of visual know-how needed to be an expert biochemist in the 21st century, there have been calls for instructors to develop biochemistry students’ visual literacy. However, visual literacy has multiple aspects, and determining which area to develop can be quite daunting. Therefore, the goals of this study were to determine what visual literacy skills biochemistry instructors deem to be most important and how instructors develop and assess visual literacy skills in their biochemistry courses. In order to address these goals ...


Building A Database For The Historical Analysis Of The General Chemistry Curriculum Using Acs General Chemistry Exams As Artifacts, Cynthia J. Luxford, Kimberly J. Linenberger, Jeffrey R. Raker, John Y. Baluyut, Jessica J. Reed, Chamila De Silva, Thomas Holme Jan 2015

Building A Database For The Historical Analysis Of The General Chemistry Curriculum Using Acs General Chemistry Exams As Artifacts, Cynthia J. Luxford, Kimberly J. Linenberger, Jeffrey R. Raker, John Y. Baluyut, Jessica J. Reed, Chamila De Silva, Thomas Holme

Chemistry Publications

As a discipline, chemistry enjoys a unique position. While many academic areas prepared “cooperative examinations” in the 1930s, only chemistry maintained the activity within what has become the ACS Examinations Institute. As a result, the long-term existence of community-built, norm-referenced, standardized exams provides a historical artifact about the nature of content coverage in courses that stretches over decades. This work reports efforts to capture information and formulate it into a database about general chemistry content coverage over the past 20 years. Roughly 2000 items have been characterized in several ways, including (i) content, using an Anchoring Concepts Content Map; (ii ...


What Do Conceptual Holes In Assessment Say About The Topics We Teach In General Chemistry?, Cynthia J. Luxford, Thomas Holme Jan 2015

What Do Conceptual Holes In Assessment Say About The Topics We Teach In General Chemistry?, Cynthia J. Luxford, Thomas Holme

Chemistry Publications

Introductory chemistry has long been considered a service course by various departments that entrust chemistry departments with teaching their students. As a result, most introductory courses include a majority of students who are not chemistry majors, and many are health and science related majors who are required to take chemistry. To identify content areas that are either well covered or sparsely covered, approximately 2000 exam items from four types of General Chemistry ACS exams (Full Year General Chemistry, General Chemistry Conceptual, First Term General Chemistry, and Second Term General Chemistry) spanning two decades were aligned to the Anchoring Concepts Content ...


Designing Chemistry Practice Exams For Enhanced Benefits. An Instrument For Comparing Performance And Mental Effort Measures, Kristen L. Murphy, Thomas Holme Jan 2015

Designing Chemistry Practice Exams For Enhanced Benefits. An Instrument For Comparing Performance And Mental Effort Measures, Kristen L. Murphy, Thomas Holme

Chemistry Publications

The emergence of students using cell phones and particularly the photographic capabilities of smartphones to cheat on exams is noted. The specific threat this form of academic misconduct poses to the ACS Examinations Institute is addressed in particular.


Comparing Recent Organizing Templates For Test Content Between Acs Exams In General Chemistry And Ap Chemistry, Thomas Holme Jan 2014

Comparing Recent Organizing Templates For Test Content Between Acs Exams In General Chemistry And Ap Chemistry, Thomas Holme

Chemistry Publications

Two different versions of “big ideas” rooted content maps have recently been published for general chemistry. As embodied in the content outline from the College Board, one of these maps is designed to guide curriculum development and testing for advanced placement (AP) chemistry. The Anchoring Concepts Content Map for general chemistry from the ACS Exams Institute is a component of a larger content map for the four-year undergraduate curriculum. This article compares the structure and content in these two maps to provide perspective on the current nature of the general chemistry curriculum. This contribution is part of a special issue ...


Results Of A National Survey Of Biochemistry Instructors To Determine The Prevalence And Types Of Representations Used During Instruction And Assessment, Kimberly J. Linenberger, Thomas Holme Jan 2014

Results Of A National Survey Of Biochemistry Instructors To Determine The Prevalence And Types Of Representations Used During Instruction And Assessment, Kimberly J. Linenberger, Thomas Holme

Chemistry Publications

Chemists and chemistry educators have long sought meaningful ways to visualize fundamentally abstract components, such as atoms and molecules, of their trade. As technology has improved, computer-based visualization methods have infused both research and education in chemistry. Biochemistry, in particular, has become highly dependent on ways that large molecular systems can be represented, and how to best focus attention on the most critical aspects of the molecular system. To better understand the current state of educational efforts related to visual literacy, a needs assessment was developed and administered to a national sample of biochemistry instructors at four-year institutions (N = 536 ...


Investigating Faculty Familiarity With Assessment Terminology By Applying Cluster Analysis To Interpret Survey Data, Jeffrey R. Raker, Thomas Holme Jan 2014

Investigating Faculty Familiarity With Assessment Terminology By Applying Cluster Analysis To Interpret Survey Data, Jeffrey R. Raker, Thomas Holme

Chemistry Publications

A cluster analysis was conducted with a set of survey data on chemistry faculty familiarity with 13 assessment terms. Cluster groupings suggest a high, middle, and low overall familiarity with the terminology and an independent high and low familiarity with terms related to fundamental statistics. The six resultant clusters were found to be associated with key demographic variables such as institution type, chemistry subdiscipline, and years of teaching experience. Implications of this work include informing the creation of targeted professional development opportunities for faculty based on representative familiarity levels, leveraging both high and low familiarity with the 13 assessment terms.


Identifying Differential Performance In General Chemistry: Differential Item Functioning Analysis Of Acs General Chemistry Trial Tests, Lisa Kendhammer, Thomas Holme, Kristen Murphy Jan 2013

Identifying Differential Performance In General Chemistry: Differential Item Functioning Analysis Of Acs General Chemistry Trial Tests, Lisa Kendhammer, Thomas Holme, Kristen Murphy

Chemistry Publications

The development of high-quality assessments can be an intensive process. At the American Chemical Society Examinations Institute (ACS-EI), this follows a general process of test design and content mapping, item construction, trial testing, item analysis, and final test setting. The item analysis portion of this procedure is an important step in using field-testing results to select the best items. This selection is based on validity analysis by experts, both field-test users and test writers, and by students in field testing. The traditional item analysis of the ACS-EI now includes a differential item functioning analysis with subgroups by gender when a ...


Using Structural Equation Modeling To Understand Chemistry Faculty Familiarity Of Assessment Terminology: Results From A National Survey, Jeffrey R. Raker, Mary Emenike, Thomas Holme Jan 2013

Using Structural Equation Modeling To Understand Chemistry Faculty Familiarity Of Assessment Terminology: Results From A National Survey, Jeffrey R. Raker, Mary Emenike, Thomas Holme

Chemistry Publications

Chemistry departments have felt pressure in recent years to produce quality data on student achievement of learning outcomes. External (e.g., accreditation agencies) and internal (e.g., academic deans) entities are demanding regular review of student achievement. It is thus necessary for the chemistry community to develop valid and reliable instruments to assess student learning. With chemistry faculty members’ integration into assessment practices, it is important that these faculty members have a sufficient understanding of the quality of and limitations to the interpretation of assessment data. As part of a larger national survey, 1505 chemistry faculty members from a diverse ...


Validating Chemistry Faculty Members’ Self-Reported Familiarity With Assessment Terminology, Mary Emenike, Jeffrey R. Raker, Thomas Holme Jan 2013

Validating Chemistry Faculty Members’ Self-Reported Familiarity With Assessment Terminology, Mary Emenike, Jeffrey R. Raker, Thomas Holme

Chemistry Publications

With the increasing emphasis placed upon chemistry instructors and departments to assess and evaluate their courses and curricula, understanding the structure of chemistry faculty members’ knowledge and use of assessment terms and concepts can shed light on potential areas for targeted professional development. Survey research that might accomplish this objective often relies on self-reported responses from the target audience, and such information is sometimes difficult to assess in terms of validity. As an example of an internal mechanism to help establish validity, it is possible to include an “internal standard” item early in the survey. For the sake of understanding ...


The Acs Exams Institute Undergraduate Chemistry Anchoring Concepts Content Map Ii: Organic Chemistry, Jeffrey R. Raker, Thomas Holme, Kristen L. Murphy Jan 2013

The Acs Exams Institute Undergraduate Chemistry Anchoring Concepts Content Map Ii: Organic Chemistry, Jeffrey R. Raker, Thomas Holme, Kristen L. Murphy

Chemistry Publications

As a way to assist chemistry departments with programmatic assessment of undergraduate chemistry curricula, the ACS Examinations Institute is devising a map of the content taught throughout the undergraduate curriculum. The structure of the map is hierarchal, with large grain size at the top and more content detail as one moves “down” the levels of the map, of which there are four levels total. This paper presents these four levels of the map with reference to second-year, organic chemistry.


Polytomous Versus Dichotomous Scoring On Multiple-Choice Examinations: Development Of A Rubric For Rating Partial Credit, Megan L. Grunert, Jeffrey R. Raker, Kristen L. Murphy, Thomas Holme Jan 2013

Polytomous Versus Dichotomous Scoring On Multiple-Choice Examinations: Development Of A Rubric For Rating Partial Credit, Megan L. Grunert, Jeffrey R. Raker, Kristen L. Murphy, Thomas Holme

Chemistry Publications

The concept of assigning partial credit on multiple-choice test items is considered for items from ACS Exams. Because the items on these exams, particularly the quantitative items, use common student errors to define incorrect answers, it is possible to assign partial credits to some of these incorrect responses. To do so, however, it becomes vital that instructors reach general agreement as to the level of partial credit. Using workshops with instructors, ACS Exams has identified reasons why partial credit could be assigned in an exam set of 70 test items. With partial-credit assignments thus established, polytomous scoring is applied and ...


Adaptation Of An Instrument For Measuring The Cognitive Complexity Of Organic Chemistry Exam Items, Jeffrey R. Raker, Jaclyn M. Trate, Thomas Holme, Kristen Murphy Jan 2013

Adaptation Of An Instrument For Measuring The Cognitive Complexity Of Organic Chemistry Exam Items, Jeffrey R. Raker, Jaclyn M. Trate, Thomas Holme, Kristen Murphy

Chemistry Publications

Experts use their domain expertise and knowledge of examinees’ ability levels as they write test items. The expert test writer can then estimate the difficulty of the test items subjectively. However, an objective method for assigning difficulty to a test item would capture the cognitive demands imposed on the examinee as well as be assignable by any domain expert familiar with the examinee group. One such instrument for assigning objective complexity of general chemistry exam items has already been reported. A revised instrument for assigning objective complexity of organic chemistry exam items is presented including the reliability and validity studies ...


A Historical Analysis Of The Curriculum Of Organic Chemistry Using Acs Exams As Artifacts, Jeffrey R. Raker, Thomas Holme Jan 2013

A Historical Analysis Of The Curriculum Of Organic Chemistry Using Acs Exams As Artifacts, Jeffrey R. Raker, Thomas Holme

Chemistry Publications

Standardized examinations, such as those developed and disseminated by the ACS Examinations Institute, are artifacts of the teaching of a course and over time may provide a historical perspective on how curricula have changed and evolved. This study investigated changes in organic chemistry curricula across a 60-year period by evaluating 18 ACS Organic Chemistry Exams through the lenses of problem-type, visualization use, content covered, and percentile rankings. For all lenses, the early 1970s emerged as a focal point for change and stabilization of the organic chemistry curricula.


Results From A National Needs Assessment Survey: A View Of Assessment Efforts Within Chemistry Departments, Mary Elizabeth Emenike, Jacob Schroeder, Kristen L. Murphy, Thomas Holme Jan 2013

Results From A National Needs Assessment Survey: A View Of Assessment Efforts Within Chemistry Departments, Mary Elizabeth Emenike, Jacob Schroeder, Kristen L. Murphy, Thomas Holme

Chemistry Publications

As is true for virtually all of higher education, chemistry departments are often required to provide evidence of student learning at both course and curricular levels through evaluation and assessment. The ACS Exams Institute conducted a needs assessment survey of 1500 chemistry faculty members from across the country to investigate motivation, role, instrument use, and challenges associated with assessment efforts. For the more than 70% of participants who reported departmental efforts related to assessment, these findings emerged: motivations were primarily external factors related to accreditation and certification, ACS Exams and in-house exams were the most common instruments used, and time ...


Building The Acs Exams Anchoring Concept Content Map For Undergraduate Chemistry, Kristen L. Murphy, Thomas Holme, April Zenisky, Karen Knauss Jan 2012

Building The Acs Exams Anchoring Concept Content Map For Undergraduate Chemistry, Kristen L. Murphy, Thomas Holme, April Zenisky, Karen Knauss

Chemistry Publications

The ability to coherently assess content knowledge throughout an entire undergraduate career represents a significant advantage for programmatic assessment strategies. Chemistry, as a discipline, has an unusual tool in this regard because of the nationally standardized exams from the ACS Exams Institute. These exams are norm-referenced and allow chemistry departments to make comparisons between the performance of their own students relative to national samples; however, currently there appears to be no systematic means for noting students’ content knowledge growth over a four-year degree. The Exams Institute is undertaking the task of organizing content along an anchoring concept or “big ideas ...


Investigating Factors That Influence Item Performance On Acs Exams, Jacob Schroeder, Kristen L. Murphy, Thomas Holme Jan 2012

Investigating Factors That Influence Item Performance On Acs Exams, Jacob Schroeder, Kristen L. Murphy, Thomas Holme

Chemistry Publications

General chemistry tests from the Examinations Institute of the Division of Chemical Education of the American Chemical Society have been analyzed to identify factors that may influence how individual test items perform. In this paper, issues of item order (position within a set of items that comprise a test) and answer order (position of correct answer relative to incorrect distractors) are discussed. Answer order is identified as potentially important, particularly for conceptually based items. When the correct answer appears earlier among the answer choices, there is some greater propensity for student performance to be better. Item-order effects are also possible ...


The Acs Exams Institute Undergraduate Chemistry Anchoring Concepts Content Map I: General Chemistry, Thomas Holme, Kristen L. Murphy Jan 2012

The Acs Exams Institute Undergraduate Chemistry Anchoring Concepts Content Map I: General Chemistry, Thomas Holme, Kristen L. Murphy

Chemistry Publications

To provide tools for programmatic assessment related to the use of ACS Exams in undergraduate chemistry courses, the ACS Exams Institute has built a content map that applies to the entire undergraduate curriculum. At the top two levels, the grain size of the content classification is large and spans the entire undergraduate curriculum. At the bottom two levels, the grain size of the content is more fine and tuned to specific course levels of the curriculum. This paper presents all four levels of the map as identified for first-year general chemistry.


Assessment Data And Decision Making In Teaching, Thomas Holme Jan 2011

Assessment Data And Decision Making In Teaching, Thomas Holme

Chemistry Publications

Finding avenues to collect and use data in our teaching parallel to the way we use data in research may be an important way to improve educational outcomes.


A Valid And Reliable Instrument For Cognitive Complexity Rating Assignment Of Chemistry Exam Items, Karen Knaus, Kristen Murphy, Anja Blecking, Thomas Holme Jan 2011

A Valid And Reliable Instrument For Cognitive Complexity Rating Assignment Of Chemistry Exam Items, Karen Knaus, Kristen Murphy, Anja Blecking, Thomas Holme

Chemistry Publications

The design and use of a valid and reliable instrument for the assignment of cognitive complexity ratings to chemistry exam items is described in this paper. Use of such an instrument provides a simple method to quantify the cognitive demands of chemistry exam items. Instrument validity was established in two different ways: statistically significant correlations between expert-based cognitive complexity ratings and student performance (as measured through statistical difficulty of items), and statistically significant correlations between expert-based cognitive complexity ratings and student mental effort ratings. Key benefits associated with instrument use include an enhanced understanding of the cognitive complexity of chemistry ...


Assessing Conceptual And Algorithmic Knowledge In General Chemistry With Acs Exams, Thomas Holme, Kristen L. Murphy Jan 2011

Assessing Conceptual And Algorithmic Knowledge In General Chemistry With Acs Exams, Thomas Holme, Kristen L. Murphy

Chemistry Publications

In 2005, the ACS Examinations Institute released an exam for first-term general chemistry in which items are intentionally paired with one conceptual and one traditional item. A second-term, paired-questions exam was released in 2007. This paper presents an empirical study of student performances on these two exams based on national samples of students who took the exams as part of a general chemistry course sequence. Psychometric data for student performances are presented in terms of classical difficulty and discrimination indexes, as well as item characteristic curves, as are more commonly used in item response theory. Having these data provided for ...