Educational Assessment, Evaluation, and Research Commons^™

Proceedings Of The Cuny Games Conference 9.0, Robert O. Duncan, Grace L. Axler-Diperte, Joseph Bisz, Christina Boyle, Devorah Kletenik, Carolyn Stallard

Publications and Research

The CUNY Games Network is an organization dedicated to encouraging research, scholarship and teaching in the developing field of games-based learning. We connect educators from every campus and discipline at CUNY and beyond who are interested in digital and non-digital games, simulations, and other forms of interactive teaching and inquiry-based learning.

Transfer Information Online: Websites And Articulation Agreements At The City University Of New York, A. W. Logue, Chet Jordan, Matthew Townsell, Nicol Bellettiere, Rhina Torres

Publications and Research

Objective: Transfer students face a range of potential challenges as they prepare to move from one college to another. Websites are critical resources for transfer students because they can be easily accessible sources of information concerning credit transfer, advising, articulation agreements, and additional transfer administrative policies and procedures. Detailed assessments of colleges’ website transfer information can provide information useful to higher education institutions regarding how transfer information should and should not be communicated online. Methods: The present study examined transfer information on the websites of 19 colleges of The City University of New York (CUNY). This examination assessed the presence …

Challenges For Successful Transfer From Community To Bachelor’S Colleges: Views Of Staff And Faculty With Transfer Responsibilities, A. W. Logue, Kerstin Gentsch, Yoshiko Oka, David Wutchiett, Stephanie Abbeyquaye

Publications and Research

Staff and faculty have influential roles in the success of students transferring from associate’s- to bachelor’s-degree programs (vertical transfer students). Our survey compared the reported views on transfer of 607 staff and faculty with transfer responsibilities in associate’s or bachelor’s programs at 19 City University of New York colleges. The findings included: (1) Staff reported feeling more confident in their responsibilities than did faculty. (2) Participants working with associate’s-degree students were more likely to report their colleges had sufficient transfer-service resources. (3) Associate’s-degree faculty were the least likely participant group to respond that the biggest barrier to the transfer process …

Possible Causes Of Leaks In The Transfer Pipeline: Student Views At The 19 Colleges Of The City University Of New York, A. W. Logue, Yoshiko Oka, David Wutchiett, Kerstin Gentsch, Stephanie Abbeyquaye

Publications and Research

Only 11% of community college (associate’s-degree) students transfer vertically and obtain a bachelor’s degree within six years, despite over 80% originally intending to do so. These leaks in the transfer pipeline disproportionately affect students from underrepresented groups, who are more likely to attend community colleges and to leak out of the pipeline. To obtain insights about how to decrease these leaks, a survey was distributed to all City University of New York undergraduates; 31,511 responded. The survey concerned students’ life and academic circumstances, as well as their information about and views on transfer. Analyses particularly compared responses of never-transferred associate’s …

A Party Platter Of Peer-Reviewed Oer Assignments, Elizabeth Jardine, Ece Aykol, Justin Rogers-Cooper

Publications and Research

Many colleges and universities have adopted core competencies during the last two decades. Their adoption reflects larger national trends in outcomes assessment, teaching and learning, and regional accreditation. Faculty at LaGuardia Community College sought model assignments with reflections on their design that speak to the college’s core competencies— integrative learning, inquiry and problem-solving, and global learning—and the communication abilities that students use to express them—written, oral, and digital communication abilities. In response, the college’s assessment leaders looked to the assignment library created by the National Institute for Learning Outcomes Assessment (NILOA) for inspiration. They developed the Learning Matters Assignment Library, …

A Psychologist’S Perspective For Coordinating Interdisciplinary Courses, Amanda L. Almond

Publications and Research

Developing a student evaluation for interdisciplinary teaching revealed a clearer goal for interdisciplinary course assessment. This chapter summarizes how interdisciplinary course assessment is a cooperative and reflexive process. Using professional judgment and a working group of peers, interdisciplinary courses maintain their integrity through regular reviews. A reflection on experiences with team-teaching, guest lecturing, and learning-communities is also included. Best practices for interdisciplinary course maintenance and concepts of validity are applied to the debate between evaluation and assessment methods. By fostering transparency, accountability, and peer-led critiques, interdisciplinary learning objectives within courses are sustained each semester. Recognizing concerns regarding evaluation, flexible approaches …

The Impact Of Undergraduate Research And Student Characteristics On Student Success Metrics At An Urban, Minority Serving, Commuter, Public Institution, Sheila I. Baron, Pamela Brown, Tammie Cumming, Michelle Mengeling

Publications and Research

Challenges to establishing and maintaining undergraduate research programs include how to demonstrate impact as evidence for future funding, establish eligibility criteria when resources are limited, and assess new components. To address these challenges, undergraduate researcher GPA, credit accumulation and time to graduation were evaluated longitudinally, at an urban, public, minority and Hispanic serving, commuter college. Students who participated in undergraduate research and matched peers were also compared. Evaluation revealed that all groups benefited from participation in undergraduate research, whether they had full- or part-time status, were STEM or non-STEM majors, or participated in single or multiple semesters of research. Addition …

Open To What? A Critical Evaluation Of Oer Efficacy Studies, Ian Mcdermott

Publications and Research

This selective literature review evaluates open educational resources (OER) efficacy studies through the lens of critical pedagogy. OER have radical potential as transformative tools for critical pedagogy or they can serve as a cost-free version of the status quo, inclined toward propagating austerity. This review analyzes studies published since 2008 with regard to cost, access, pedagogy, commercialization, and labor. These criteria are used to make explicit subjects indirectly addressed, if not ignored completely, in the existing literature. Typically, ample attention is paid to a study’s design and methodology but the underlying institutional infrastructure and decision-making process is unexamined. What emerges …

Academic Assessment: Best Practices For Successful Outcomes With Accreditation Evaluation Teams, Tammie Cumming, M. David Miller

Publications and Research

This chapter describes the role of faculty in conducting quality assessment and includes suggestions for preparing for successful meetings with reviewers on accreditation teams.

Accelerating Systemic Change Network (Ascn): Demonstrating Impact, David Bressoud, Pamela Brown, Archie Holmes, Kate White

Publications and Research

The goals of this presentation for participants was:

•Learn about the Accelerating Systemic Change Network (ASCN), why it is important, what it can do for you, and how you can get involved. •Learn about work done by ASCN on demonstrating impact and how measurement and communication can be used to promote change. •Learn about areas where educational research is needed and you can contribute.

Who Can Excel In Mathematics?, Marla A. Sole

Publications and Research

Students may erroneously believe that mathematics ability is largely innate and fixed (Kimball and Smith 2013). The evidence seems to support these mistaken beliefs. Women and minorities score lower on the mathematics section of the SAT (College Board 2017). When assessing students’ mathematics college readiness, a larger percentage of women, African Americans, and Hispanics fell below the benchmark (ACT 2017); and, in general, in science, engineering, math, and technology (STEM) fields, the gender and racial gaps in interest, performance, and degrees awarded have not closed (Neuhauser 2015). Persistent performance gaps can serve to reinforce students’ mistaken belief that women and …

Place-Based Learning Across The Disciplines: A Living Laboratory Approach To Pedagogy, Karen Goodlad, Anne E. Leonard

Publications and Research

Faculty participants in a fellowship designed to engage students at an urban commuter college of technology in their general education curriculum evaluated and redesigned their courses to include place-based learning (PBL) using the Living Laboratory model of pedagogy. Focused on faculty perception of the relationship between PBL and its influence on general education, the study illustrates how faculty from across disciplines apply PBL techniques to revitalize general education learning outcomes. Findings include the influence of the fellowship on the design of PBL activities and perceived levels of student engagement, especially when compared to more traditional classroom instruction.

Future Goals And Actions Of Faculty Development, Catherine Haras, Margery Ginsberg, Eva Fernández, Emily Daniell Magruder

Publications and Research

No abstract provided.

Institutional Investment In Teaching Excellence, Steven C. Taylor, Catherine Haras, Emily Daniell Magruder, Eva Fernández, Margery Ginsberg, Jake Glover

Publications and Research

No abstract provided.

Incorporation And Evaluation Of Authentic Research Experiences Into The Curriculum Through Development Of A Theory Of Action, Pamela Brown, Tammie Cumming, Joan D. Pasley

Publications and Research

A theory of action outlining undergraduate research program inputs and desired outcomes was developed and used to guide implementation of Course-Based Undergraduate Research Experience (CURE) sections and to create assessment tools to measure attainment of program goals in both apprentice-model undergraduate research and CURE. Student survey results for these two research programs were compared and suggest that many aspects of the academic goals such as designing an experiment, using equipment, collecting and analyzing data, and collaborating with others were achieved in both groups. Regarding the relationship with mentors, both groups reported receiving academic advisement in course selection and career options. …

Introduction: History And Conceptual Basis Of Assessment In Higher Education, Peter Ewell, Tammie Cumming

Publications and Research

Assessment and accountability are now inescapable features of the landscape of higher education, and ensuring that these assessments are psychometrically sound has become a high priority for accrediting agencies and therefore also for higher education institutions. Bringing together the higher education assessment literature with the psychometric literature, this book focuses on how to practice sound assessment.

This volume provides comprehensive and detailed descriptions of tools for and approaches to assessing student learning outcomes in higher education. The book is guided by the core purpose of assessment, which is to enable faculty, administrators, and student affairs professionals with the information they …

What Impacts Do Oer Have On Students? Students Share Their Experiences With A Health Psychology Oer At New York City College Of Technology, Cailean Cooney

Publications and Research

This article reports findings from a study conducted with students in three sections of a Health Psychology course that replaced a traditional textbook with open educational resources (OER) as the primary course material. The purpose of the study was to learn how OER impacted students. Data were collected in Fall 2015 with students from New York City College of Technology (City Tech), of the City University of New York (CUNY), a comprehensive college located in Brooklyn. Students were assigned the OER by their course instructor, who developed it as part of a library funded OER pilot initiative. Two research instruments …

Incorporating Undergraduate Research Experiences In An Engineering Technology Curriculum, Benito Mendoza, Pamela Brown

Publications and Research

Undergraduate research is a high-impact practice leading to student success, engagement, interest in higher education, and skills development. There are two well-known models for incorporating research experiences in a program: Undergraduate Research Experiences (UREs) and Course-based Undergraduate Research Experiences (CUREs). UREs represent the apprentice model. They feature individual students in faculty research laboratories and provide the opportunity for one-on-one mentoring. On the other hand, CUREs are embedded into the curriculum and are open to most students. CUREs put high demands on one or a few mentors to guide many students. UREs and CUREs vary in selectivity, duration, setting, mentoring, and …

Should The New England Education Research Organization Start A Journal In The Age Of Audit Culture? Reflections On Academic Publishing, Metrics, And The New Academy, Edward Lehner, Kate Finley

Publications and Research

A large regional educational research association can straightforwardly establish a scholarly journal associated with its annual meeting. However, this work underscores the complicated scholarly ecosystem that an association enters when publishing a journal. The social sciences’ scholarly literature exists in a related series of networks that could be described as a type of “audit culture.” Within audit culture, two major academic publishers, Elsevier and Thomson Reuters, have established competing, yet strikingly collinear, journal metrics systems: Scopus and Web of Science, respectively. These and other bibliometrics systems are used to assess, order, and rank the supposed value of a researcher’s work. …

Should Students Assessed As Needing Remedial Mathematics Take College-Level Quantitative Courses Instead? A Randomized Controlled Trial, Alexandra W. Logue, Mari Watanabe-Rose, Daniel Douglas

Publications and Research

This data set is for Should Students Assessed as Needing Remedial Mathematics Take College-Level Quantitative Courses Instead? A Randomized Controlled Trial (Logue, Watanabe-Rose, & Douglas, 2016).

Hostos Online Learning Assessment: A Survey Of Student Perceptions, Kate Wolfe, Sarah Hoiland, Kate Lyons, Carlos Guevara, Kristopher B. Burrell, Jacki Disanto, Sandy Figueroa, Aaron Davis, Iber Poma, Wilfredo Rodriguez, Linda L. Ridley

Publications and Research

The Office of Education Technology (EdTech) at Hostos Community College and faculty members from various departments created the Hostos Online Learning Assessment (HOLA) Task Force to design a survey for gathering and assessing data about students’ perceptions of their online learning experiences. The task force wanted to utilize the survey results to identify strengths and weaknesses in online instruction and student preparedness for the online learning environment. Student perceptions of online learning are integral to building upon current best practices and also gauging the preparedness of the students for the online learning environment, particularly in an urban, Hispanic-serving community college. …

Classification Accuracy Of Mixed Format Tests: A Bi-Factor Item Response Theory Approach, Wei Wang, Fritz Drasgow, Liwen Liu

Publications and Research

Mixed format tests (e.g., a test consisting of multiple-choice [MC] items and constructed response [CR] items) have become increasingly popular. However, the latent structure of item pools consisting of the two formats is still equivocal. Moreover, the implications of this latent structure are unclear: For example, do constructed response items tap reasoning skills that cannot be assessed with multiple choice items? This study explored the dimensionality of mixed format tests by applying bi-factor models to 10 tests of various subjects from the College Board’s Advanced Placement (AP) Program and compared the accuracy of scores based on the bi-factor analysis with …

“I Am More Productive In The Library Because It’S Quiet”: Commuter Students In The College Library, Mariana Regalado, Maura A. Smale

Publications and Research

This article discusses commuter students’ experiences with the academic library, drawn from a qualitative study at the City University of New York. Undergraduates at six community and baccalaureate colleges were interviewed to explore how they fit schoolwork into their days, and the challenges and opportunities they encountered. Students identified physical and environmental features that informed their ability to successfully engage in academic work in the library. They valued the library as a distraction-free place for academic work, in contrast to the constraints they experienced in other places—including in their homes and on the commute.

Benefits Of Hybrid Classes In Community Colleges, Joel Barker

Publications and Research

This article discusses hybrid courses and their impact on educational facilities, their students, and instructors. Instructors now have over ten years of data related to hybrid courses and by trial and error have devised different strategies to plan and execute lesson plans via partly online forums. Programs are in place that gives students the opportunity to excel; these types of courses promote a unique balance of guidance by the instructor and acceptance of responsibility by the students. Students have responded in a positive manner in pursuing these types of courses.

The Create Strategy For Intensive Analysis Of Primary Literature Can Be Used Effectively By Newly Trained Faculty To Produce Multiple Gains In Diverse Students, Leslie M. Stevens, Sally G. Hoskins

Publications and Research

The CREATE (Consider Read, Elucidate the hypotheses, Analyze and interpret the data, and Think of the next Experiment) strategy aims to demystify scientific research and scientists while building critical thinking, reading/analytical skills, and improved science attitudes through intensive analysis of primary literature. CREATE was developed and piloted at the City College of New York (CCNY), a 4-yr, minority-serving institution, with both upper-level biology majors and first-year students interested in science, technology, engineering, and mathematics. To test the extent to which CREATE strategies are broadly applicable to students at private, public, research-intensive, and/or primarily undergraduate colleges/universities, we trained a cohort of …

An Overview Of Undergraduate Research In The Cuny Community College System, Avrom J. Caplan, Effie S. Maclachlan Phd

Publications and Research

CUNY community colleges occupy a unique niche because they are part of a larger geographically focused university system in which all faculty members are governed by a single set of standards for professional development. Research is clearly a part of the wider institutional culture, and dedicated faculty members who obtained support from state and federal funding agencies have conducted successful student-research programs. Close partnerships between community colleges and their four-year counterparts can contribute to positive student outcomes and to the subsequent transfer of students. The main roadblock to broadening participation is the small number of students who can be supported …

Create Cornerstone: Introduction To Scientific Thinking, A New Course For Stem-Interested Freshmen, Demystifies Scientific Thinking Through Analysis Of Scientific Literature, Alan J. Gottesman, Sally G. Hoskins

Publications and Research

The Consider, Read, Elucidate hypotheses, Analyze and interpret data, Think of the next Experiment (CREATE) strategy for teaching and learning uses intensive analysis of primary literature to improve students’ critical-thinking and content integration abilities, as well as their self-rated science attitudes, understanding, and confidence. CREATE also supports maturation of undergraduates’ epistemological beliefs about science. This approach, originally tested with upper-level students, has been adapted in Introduction to Scientific Thinking, a new course for freshmen. Results fromthis course’s initial semesters indicate that freshmen in a one-semester introductory course that uses a narrowly focused set of readings to promote development of analytical …

The C.R.E.A.T.E. Approach To Primary Literature Shifts Undergraduates’ Self-Assessed Ability To Read And Analyze Journal Articles, Attitudes About Science, And Epistemological Beliefs, Sally G. Hoskins, David Lopatto, Leslie M. Stevens

Publications and Research

The C.R.E.A.T.E. (Consider,Read, Elucidate hypotheses, Analyze and interpret data, Think of the next Experiment)method uses intensive analysis of primary literature in the undergraduate classroom to demystify and humanize science. We have reported previously that the method improves students’ critical thinking and content integration abilities, while at the same time enhancing their self-reported understanding of “who does science, and why.” We report here the results of an assessment that addressed C.R.E.A.T.E. students’ attitudes about the nature of science, beliefs about learning, and confidence in their ability to read, analyze, and explain research articles. Using a Likert-style survey administered pre- and postcourse, …

The Future Of Natural Selection Knowledge Measurement: A Reply To Anderson Et Al. (2010), Ross Nehm, Irvin Sam Schonfeld

Publications and Research

The development of rich, reliable, and robust measures of the composition, structure, and stability of student thinking about core scientific ideas (such as natural selection) remains a complex challenge facing science educators. In their recent article (Nehm & Schonfeld 2008), the authors explored the strengths, weaknesses, and insights provided by a detailed exploration of three commonly used measures of student thinking about natural selection in a large sample of underrepresented minority students. One of their core findings was that all of the tools they studied--including the CINS--have strengths and weaknesses that must be carefully taken into consideration by those …

Measuring Knowledge Of Natural Selection: A Comparison Of The C.I.N.S., An Open-Response Instrument, And An Oral Interview, Ross Nehm, Irvin Sam Schonfeld

Publications and Research

Growing recognition of the central importance of fostering an in-depth understanding of natural selection has, surprisingly, failed to stimulate work on the development and rigorous evaluation of instruments that measure knowledge of it. We used three different methodological tools, the Conceptual Inventory of Natural Selection (CINS), a modified version of Bishop and Anderson's (Bishop and Anderson [1990] Journal of Research in Science Teaching 27: 415-427) open-response test that we call the Open Response Instrument (ORI), and an oral interview derived from both instruments, to measure biology majors' understanding of and alternative conceptions about natural selection. We explored how these instruments …