Open Access. Powered by Scholars. Published by Universities.®

Education Commons

Open Access. Powered by Scholars. Published by Universities.®

Articles 1 - 11 of 11

Full-Text Articles in Education

Lessons Learned From Collaborations In Chemistry Assessment Across Universities: Challenges In Transfer And Scale, Pamela L. Paek, Thomas Holme Jan 2013

Lessons Learned From Collaborations In Chemistry Assessment Across Universities: Challenges In Transfer And Scale, Pamela L. Paek, Thomas Holme

Chemistry Publications

This chapter reviews a collaborative effort to cross-pollinate and share work around chemistry assessments across several universities. The goal was to find ways to synthesize separate projects and capitalize on applying developed instruments and assessments beyond a single university, and in new situations, to increase scale and check for generalizability. Discussion of the successes and challenges of scale and transfer of the collaboration is detailed in this chapter.


Importance Of Considering Longitudinal Trajectories In Education Reform Efforts, Thomas Holme, Melanie M. Cooper, Pratibha Varma-Nelson Jan 2013

Importance Of Considering Longitudinal Trajectories In Education Reform Efforts, Thomas Holme, Melanie M. Cooper, Pratibha Varma-Nelson

Chemistry Publications

This chapter introduces the collection of articles in this book with an emphasis on why it is important to consider the way that educational research and reform efforts change over time. The importance of considering a longitudinal view of education reform is emphasized in two ways. First, the context of this work relative to current literature is considered. Second, the idea of a greater focus on the longer-term trajectories of reform efforts in considered in terms of suggestions for the future of chemistry education.


Adaptation Of An Instrument For Measuring The Cognitive Complexity Of Organic Chemistry Exam Items, Jeffrey R. Raker, Jaclyn M. Trate, Thomas Holme, Kristen Murphy Jan 2013

Adaptation Of An Instrument For Measuring The Cognitive Complexity Of Organic Chemistry Exam Items, Jeffrey R. Raker, Jaclyn M. Trate, Thomas Holme, Kristen Murphy

Chemistry Publications

Experts use their domain expertise and knowledge of examinees’ ability levels as they write test items. The expert test writer can then estimate the difficulty of the test items subjectively. However, an objective method for assigning difficulty to a test item would capture the cognitive demands imposed on the examinee as well as be assignable by any domain expert familiar with the examinee group. One such instrument for assigning objective complexity of general chemistry exam items has already been reported. A revised instrument for assigning objective complexity of organic chemistry exam items is presented including the reliability and validity studies ...


Identifying Differential Performance In General Chemistry: Differential Item Functioning Analysis Of Acs General Chemistry Trial Tests, Lisa Kendhammer, Thomas Holme, Kristen Murphy Jan 2013

Identifying Differential Performance In General Chemistry: Differential Item Functioning Analysis Of Acs General Chemistry Trial Tests, Lisa Kendhammer, Thomas Holme, Kristen Murphy

Chemistry Publications

The development of high-quality assessments can be an intensive process. At the American Chemical Society Examinations Institute (ACS-EI), this follows a general process of test design and content mapping, item construction, trial testing, item analysis, and final test setting. The item analysis portion of this procedure is an important step in using field-testing results to select the best items. This selection is based on validity analysis by experts, both field-test users and test writers, and by students in field testing. The traditional item analysis of the ACS-EI now includes a differential item functioning analysis with subgroups by gender when a ...


Using Structural Equation Modeling To Understand Chemistry Faculty Familiarity Of Assessment Terminology: Results From A National Survey, Jeffrey R. Raker, Mary Emenike, Thomas Holme Jan 2013

Using Structural Equation Modeling To Understand Chemistry Faculty Familiarity Of Assessment Terminology: Results From A National Survey, Jeffrey R. Raker, Mary Emenike, Thomas Holme

Chemistry Publications

Chemistry departments have felt pressure in recent years to produce quality data on student achievement of learning outcomes. External (e.g., accreditation agencies) and internal (e.g., academic deans) entities are demanding regular review of student achievement. It is thus necessary for the chemistry community to develop valid and reliable instruments to assess student learning. With chemistry faculty members’ integration into assessment practices, it is important that these faculty members have a sufficient understanding of the quality of and limitations to the interpretation of assessment data. As part of a larger national survey, 1505 chemistry faculty members from a diverse ...


Validating Chemistry Faculty Members’ Self-Reported Familiarity With Assessment Terminology, Mary Emenike, Jeffrey R. Raker, Thomas Holme Jan 2013

Validating Chemistry Faculty Members’ Self-Reported Familiarity With Assessment Terminology, Mary Emenike, Jeffrey R. Raker, Thomas Holme

Chemistry Publications

With the increasing emphasis placed upon chemistry instructors and departments to assess and evaluate their courses and curricula, understanding the structure of chemistry faculty members’ knowledge and use of assessment terms and concepts can shed light on potential areas for targeted professional development. Survey research that might accomplish this objective often relies on self-reported responses from the target audience, and such information is sometimes difficult to assess in terms of validity. As an example of an internal mechanism to help establish validity, it is possible to include an “internal standard” item early in the survey. For the sake of understanding ...


A Trajectory Of Reform In General Chemistry For Engineering Students, Thomas Holme, Heather Caruthers Jan 2013

A Trajectory Of Reform In General Chemistry For Engineering Students, Thomas Holme, Heather Caruthers

Chemistry Publications

This chapter considers efforts to enhance learning within the general chemisty course taken by pre-engineering students. Because this course is inherently offered as a service course, often for students in a different college from the Chemistry Department at a university, there are both constraints and opportunities related to the manner in which reform can be enacted. Efforts spanning roughly 15 years are described and an emphasis on the nature of problem-solving within the course emerges as a common theme. The issue of student motivation is also considered with pre-engineering students serving as a prototype of a type of student who ...


The Acs Exams Institute Undergraduate Chemistry Anchoring Concepts Content Map Ii: Organic Chemistry, Jeffrey R. Raker, Thomas Holme, Kristen L. Murphy Jan 2013

The Acs Exams Institute Undergraduate Chemistry Anchoring Concepts Content Map Ii: Organic Chemistry, Jeffrey R. Raker, Thomas Holme, Kristen L. Murphy

Chemistry Publications

As a way to assist chemistry departments with programmatic assessment of undergraduate chemistry curricula, the ACS Examinations Institute is devising a map of the content taught throughout the undergraduate curriculum. The structure of the map is hierarchal, with large grain size at the top and more content detail as one moves “down” the levels of the map, of which there are four levels total. This paper presents these four levels of the map with reference to second-year, organic chemistry.


Polytomous Versus Dichotomous Scoring On Multiple-Choice Examinations: Development Of A Rubric For Rating Partial Credit, Megan L. Grunert, Jeffrey R. Raker, Kristen L. Murphy, Thomas Holme Jan 2013

Polytomous Versus Dichotomous Scoring On Multiple-Choice Examinations: Development Of A Rubric For Rating Partial Credit, Megan L. Grunert, Jeffrey R. Raker, Kristen L. Murphy, Thomas Holme

Chemistry Publications

The concept of assigning partial credit on multiple-choice test items is considered for items from ACS Exams. Because the items on these exams, particularly the quantitative items, use common student errors to define incorrect answers, it is possible to assign partial credits to some of these incorrect responses. To do so, however, it becomes vital that instructors reach general agreement as to the level of partial credit. Using workshops with instructors, ACS Exams has identified reasons why partial credit could be assigned in an exam set of 70 test items. With partial-credit assignments thus established, polytomous scoring is applied and ...


A Historical Analysis Of The Curriculum Of Organic Chemistry Using Acs Exams As Artifacts, Jeffrey R. Raker, Thomas Holme Jan 2013

A Historical Analysis Of The Curriculum Of Organic Chemistry Using Acs Exams As Artifacts, Jeffrey R. Raker, Thomas Holme

Chemistry Publications

Standardized examinations, such as those developed and disseminated by the ACS Examinations Institute, are artifacts of the teaching of a course and over time may provide a historical perspective on how curricula have changed and evolved. This study investigated changes in organic chemistry curricula across a 60-year period by evaluating 18 ACS Organic Chemistry Exams through the lenses of problem-type, visualization use, content covered, and percentile rankings. For all lenses, the early 1970s emerged as a focal point for change and stabilization of the organic chemistry curricula.


Results From A National Needs Assessment Survey: A View Of Assessment Efforts Within Chemistry Departments, Mary Elizabeth Emenike, Jacob Schroeder, Kristen L. Murphy, Thomas Holme Jan 2013

Results From A National Needs Assessment Survey: A View Of Assessment Efforts Within Chemistry Departments, Mary Elizabeth Emenike, Jacob Schroeder, Kristen L. Murphy, Thomas Holme

Chemistry Publications

As is true for virtually all of higher education, chemistry departments are often required to provide evidence of student learning at both course and curricular levels through evaluation and assessment. The ACS Exams Institute conducted a needs assessment survey of 1500 chemistry faculty members from across the country to investigate motivation, role, instrument use, and challenges associated with assessment efforts. For the more than 70% of participants who reported departmental efforts related to assessment, these findings emerged: motivations were primarily external factors related to accreditation and certification, ACS Exams and in-house exams were the most common instruments used, and time ...