Educational Assessment, Evaluation, and Research Commons^™

The Evaluation Of Village Building Program/Thematic Real Work Lecture (Kknt) Mbkm's Implementation At Uin Sunan Kalijaga, Muyassaroh Muyassaroh, Sudiyatno Sudiyatno, Nur Hidayanto Pancoro Setyo Putro, Rahma Yunita, Syafiqurrahman Syafiqurrahman

Jurnal Penelitian dan Evaluasi Pendidikan

This study aims to determine the standards of MBKM Thematic KKN, how the implementation of this program at UIN Sunan Kalijaga, as well as to determine the gap between the existing standards and the form of implementation. This evaluation research uses the descrepancy evaluation model with a descriptive qualitative approach. Data collection in this study used interview techniques, documentation, and was equipped with an open questionnaire. The data sources in this study at the interview stage were 7 resource persons 1 Head of PPM UIN Sunan Kalijaga, 2 Field Supervisor (DPL), 2 heads of Study Programs and 2 students, as …

Dominant Factors That Determine College Students Completing Studies In Mathematics Education Study Programs, Sintha Sih Dewanti, Aji Joko Budi Pramono

REID (Research and Evaluation in Education)

Failure to graduate on time is a problem for both the university and the students themselves. Beside the competencies possessed by students, many other factors affect the completion of student studies. This study aims to reduce the variables that really affect the completion of student studies. The approach of this research was survey research on students of the Mathematics Education Study Program at least in semester 7 (currently taking a final project course). There are 17 factors that determine the completion of student studies, namely achievement motivation, discipline, interest, intelligence, study habits, health, part-time work activities, organizational activities, curriculum, mentoring …

Validity, Reliability, And Item Characteristics Of Cell Material Science Literacy Assessment Instruments, Asrizal Wahdan Wilsa, Ani Rusilowati, Endang Susilaningsih, Jaja Jaja, Veni Nurpadillah

Jurnal Penelitian dan Evaluasi Pendidikan

This study aims to examine the validity, reliability, and item characteristics of scientific literacy assessment instruments on cell material. The scientific literacy ability category includes identifying valid scientific arguments, evaluating the validity of a source, evaluating the use and misuse of scientific information, understanding research design elements and how they impact scientific findings, and reading and interpreting graphical representations of data. The research subjects were students of the 5th semester elementary school teacher education study program. The validity test included content validity using Aiken V validity index and construct validity using Exploratory Factor Analysis (EFA). Reliability test is carried out …

Misconception Tendency Of Differentiated Instruction In Publicly Shared Teaching And Learning Videos On Youtube: A Mixed-Methods Exploration Study, Guruh Sukarno Putra

Jurnal Penelitian dan Evaluasi Pendidikan

Differentiated instruction is a pedagogical approach in which teachers fulfill students' needs based on their interests and learning profiles. This paradigm is new to the Indonesian educational sector as the government of Indonesia introduced the Merdeka curriculum in 2022, which offers a new style of teaching and learning in Indonesia through differentiation in the classroom. As an unfamiliar approach, the recent practice of differentiated instruction may lead to a misconception, particularly in its implementation. This paper focuses on witnessing teachers' misconceptions regarding differentiated instruction in three major aspects: teacher-student relations, teacher-learning design, and teacher-learning goals, using a mixed method with …

Measuring The Perceptions Of Prospective Teachers: Planning Technology-Based Learning During The Covid-19 Pandemic, Memet Sudaryanto, M. Zainudin

Jurnal Penelitian dan Evaluasi Pendidikan

Online learning planning presents unique challenges that aspiring teachers must prepare for. This study aims to identify potential teachers' perspectives on designing technology-based instruction. Competency dimensions were created based on the findings of focused group discussions (FGD) with teachers and lecturers, which resulted in aspects of social networking, managing digital identity, filtering, creating content, and reusing. Following the FGD, indicators of nine dimensions"”pre-learning, opening planning, material mastery, learning models, learning resources, student involvement, assessment of processes and results, interaction mastery, and closing learning"”were created. Samples were taken through simple random sampling of 250 prospective teachers in education study programs at …

Analysis Of Critical Thinking Skills, Cognitive Learning Outcomes, And Student Activities In Learning The Human Excretory System Using An Interactive Flipbook, Novita Hawsari Aswanti, Wiwi Isnaeni

REID (Research and Evaluation in Education)

This study aims to: (1) analyze the achievement of critical thinking skills, (2) analyze the achievement of cognitive learning outcomes, (3) determine the learning activities carried out, and (4) determine student responses to the use of interactive flipbook media. This study is Quasi Experiment research with a posttest-only control design employing a purposive sampling technique. The data in this study are (1) data on the achievement of critical thinking ability indicators, (2) data on achievement of cognitive levels (C4, C5, and C6), (3) data on student learning activities carried out, and (4) data on student responses to use of interactive …

Revealing The Characteristics Of Indonesian Language Test Used In The National-Standardized School Examinations, Marwah Ulwatunnisa, Heri Retnawati, Muhardis Muhardis, Eri Yusron

REID (Research and Evaluation in Education)

This study aimed to determine the characteristics of Indonesian language test used in the national standardized school examinations (Ujian Sekolah Berstandar Nasional, USBN). We used the response data of 218 students from a public senior high school in the Special Region of Yogyakarta, Indonesia, to the test consisting of two packages, A and B, in the 2018/2019 academic year to investigate the characteristics of the test and its items. Quantitative analysis using classical test theory (CTT) and one-parameter logistic item response theory (1-PL IRT) model was conducted to investigate the characteristics of the test and its items based …

Pre-Service Teachers' Agentive Projections Toward Innovation In Online English Language Teaching (Elt) Classes, Santi Farmasari, Lalu Ali Wardana, Baharuddin Baharuddin, Amrullah Amrullah, Mh Isnaeni, Husnul Lail

REID (Research and Evaluation in Education)

This study examines the pre-service teachers' agentive projections toward innovation in online English Language Teaching (ELT) classes. Employing teacher agency theory, this instrumental case study views projections as agentive when they are informed by the students' ecological aspects (past and present), oriented to solve potential learning problems and improve learning outcomes. The study involved 84 pre-service teachers who were voluntarily asked to fill in a questionnaire, submit a lesson plan, and be interviewed. nVivo Pro was used to organize themes. The study indicates that the pre-service teachers, M=3.81, SD=.590, perceive that innovation in online ELT classes is closely related to …

The Implementation Of Geography Learning With Spatial Representation Using The Discrepancy Evaluation Model, Muhammad Nursa'ban, Mukminan Mukminan

REID (Research and Evaluation in Education)

This study aims to evaluate the learning process of geography with spatial representation in high school using the Discrepancy Evaluation Model (DEM) from Provus. Evaluation goes through five stages: design, installation, process, and product, as well as implementation comparison. Respondents were five teachers and 208 students from SMA 1 Srandakan, MAN 2 Yogyakarta, and SMA 1 Pundong in Yogyakarta. The evaluation was carried out in February-July 2021. The focus of the evaluation includes (1) learning design, (2) learning tools, (3) learning implementation, (4) learning outcomes, geographic critical thinking, and (5) implementing learning comparisons. Data were collected through document studies, observations, …

Gaining A Deeper Understanding Of The Meaning Of The Carelessness Parameter In The 4pl Irt Model And Strategies For Estimating It, Timbul Pardede, Agus Santoso, Diki Diki, Heri Retnawati, Ibnu Rafi, Ezi Apino, Munaya Nikma Rosyada

REID (Research and Evaluation in Education)

Three popular models are used to describe the characteristics of the test items and estimate the ability of examinees under the dichotomous IRT model, namely the one-, two-, and three-parameter logistic models. The three-item parameters are discriminating power, difficulty, and pseudo-guessing. In the development of the dichotomous IRT model, carelessness or upper asymptote parameter was proposed, which forms a four-parameter logistic (4PL) model to accommodate a condition where a high-ability examinee gives an incorrect response to a test item when he/she should be able to respond to the test item correctly. However, the carelessness parameter and the 4PL model have …

Smartphone Application For Assessing Teacher Performance, Hesty Marlianawati, Suranto Suranto, Badrun Kartowagiran

REID (Research and Evaluation in Education)

This work seeks to design a smartphone application for assessing teacher performance utilizing a 4D development model modified with the Mardapi instrument development model of (1) define (defining); (2) design (deciding instrument requirements, determi­ning application media specifications, creating flowcharts, and creating storyboards); (3) develop (assemble instruments, code, execute tests, evaluate test results, and interpret measurement findings); and (4) disseminate (spread). Proof of the application's viability was established through product testing with 26 principals acting as application users and evaluations by material and media specialists. The content validity of the acquired results was evaluated using the Aiken method, and the construct …

Evaluation Of Music Learning During The Covid-19 Pandemic In Senior High Schools, Putra Andino Nugrahhu

REID (Research and Evaluation in Education)

The purpose of this study is to see the extent to which the implementation of teaching and learning activities for Cultural Arts subjects, especially in the field of Music Arts at high schools in Palangka Raya, was carried out during the COVID-19 pandemic. This research is evaluation research with a descriptive qualitative approach that uses a discre­pancy evaluation model that measures the gaps in the learning process. Data was collec­ted through observation, interview, and documentation techniques and validated using triangulation and source triangulation. The results of the study found that in the evalua­tion of input, music art learning consisting of …

Development And Validation Of A Self-Assessment-Based Instrument To Measure Elementary School Students' Attitudes In Online Learning, Ari Setiawan, Wiputra Cendana, Mark Ayres, Azim Abdurakhmanovich Yuldashev, Sri Panca Setyawati

REID (Research and Evaluation in Education)

Online learning during the COVID-19 pandemic makes it difficult for teachers to assess student learning attitudes. Limited availability of instruments to measure attitudes of students when they are engaged in online learning leads to difficulty of teachers to conduct appropriate assessments on that measure. The current study, therefore, mainly was intended to produce a self-assessment-based instrument that is feasible to use to measure students' attitudes in online learning. In order to produce such instrument, we used developmental research method by following steps in the instrument design that is proposed by McCoach, Gable, and Madura. Furthermore, in order to provide feasibility …

Determinant Factors Affecting The Research Performance Of Lecturers Receiving External Funds, Haryanto Haryanto, Nur Kholis, Samsul Hadi, Ezi Apino, Muh. Asriadi Am, Cerry Kartika Trizkyana

REID (Research and Evaluation in Education)

Quality research can positively impact the development of science and technology and improve the reputation of universities. This study investigates the determinant factors that affect the research performance of lecturers who obtain external funding. The sample was 41 lecturers who obtained external research funds from various institutions (e.g., Directorate of Research, Technology and Community Service, Ministry of Education, Culture, Research and Technology of the Republic of Indonesia; National Research and Innovation Agency of the Republic of Indonesia; and similar institutions) for the last three years (2021 - 2023). Cluster analysis using secondary data (Science and Technology Index or SINTA data) …

Comparison Of Item Characteristic Analysis Models Of Reading Literacy Test With Polytomous Item Response Theory, Firyal Nabila Harsana, Heri Retnawati, Septinda Rima Dewanti, Rogers Andrew Lumenyela, Maulana Fatahillah Adzima, Atut Reni Septiana

REID (Research and Evaluation in Education)

This study aims to compare the analysis model of the characteristics of reading literacy items with the polytomous item response theory, which uses the Graded Response Model (GRM), Partial Credit Model (PCM), Generalized Partial Credit Model (GPCM), and Nominal Reasons Model (NRM). This research is quantitative research in nature, and secondary datawere used from about 1000 test takers’ responses to reading literacy items in the 2018 reading literacy study analyzed with the R program. This model comparison was carried out so that the analysis results obtained were more accurate in representing the level of reading literacy skills in Indonesia. The …

Measuring Students' Generic Skills Through National Assessment, Toha Makhshun, Bassam Abul A'La, Kusaeri Kusaeri

Jurnal Penelitian dan Evaluasi Pendidikan

This study aims to measure the generic skills (GS) of Madrasah Aliyah (MA) students in Indonesia. Respondents who participated in this study were madrasah students scattered in East Java and Central Java as representatives of MAs in Indonesia. The method used in this research is a survey distributed via Google Form. Three dimensions are mentioned in the research instrument for measuring GS: religious moderation, critical and creative thinking skills, and interpersonal skills. Using SPPS version 16 software, a descriptive analysis was employed as the data analysis technique. The results of this study showed that the GS of students with religious …

Instrument And Evaluation Of Students' Involvement In Learning Mathematics In 21-Century Skills At Senior High Schools Of Riau Province, Dedek Andrian, Leo Adhar Effendi

Jurnal Penelitian dan Evaluasi Pendidikan

Student engagement is the most crucial factor in learning activities because it is the main factor in improving student learning outcomes in class. This study aims to develop instruments and evaluate student involvement in learning mathematics to achieve maximum learning outcomes at school. The population in this study was all high school students in Riau Province. The sample is 150 high school students selected by multiple-stage random sampling based on high, medium, and low student achievement levels. Data analysis in this study was carried out four times: (1) data analysis based on the results of expert assessment of the instrument …

A Meta-Analysis Of The Relationship Between Self-Assessment And Mathematics Learning Achievement, Fransidha Sidhara Hadi, Janu Arlinwibowo, Gupita Nadindra Fatima

Jurnal Penelitian dan Evaluasi Pendidikan

This study uses a meta-analysis model to describe the relationship between self-assessment and achievement in learning mathematics. This meta-analysis covers articles published from 2011 to 2022 and is restricted to studies published in English. Indexed articles by Google Scholar were chosen. The data must meet these criteria: quantitative research, containing correlational research on the relationship between self-assessment and mathematics learning achievement (including correlation values and sample size). Through the data screening process with predetermined criteria, 12 research results were obtained, containing 43 studies to be analyzed. This meta-analysis uses a random effect model due to the heterogeneous data distribution. A …

Developing And Implementing An Instrument For Assessing Critical Thinking And Visual Representations In Learning Physics Materials Of Optical Instruments, Dewi Nurulhasni, Eka Ayu Nurbaiti, Rosa Safirotun Nabila, Jumadi Jumadi

Jurnal Penelitian dan Evaluasi Pendidikan

This study aims to develop an assessment instrument to measure students' critical thinking and visual representation abilities in optical instrument material. This research and development (R&D) employed the ADDIE (Analysis, Design, Development, Implementation, and Evaluation) instructional model. This research develops a test instrument that has more specific stages so the ADDIE development stage is integrated with the development stage of the test instrument developed by Mardapi. The subjects in this study are grade XI students of State Madrasah Aliyah (MAN) 4 Bantul. This study produced a test instrument for measuring students' critical thinking and visual representation abilities, with high validity,and …

Item Quality Analysis Using The Rasch Model To Measure Critical Thinking Ability In The Material Of The Human Digestive System Of Biology Subject In High School, Wanda Agus Prasetya, Anggi Tias Pratama

Jurnal Penelitian dan Evaluasi Pendidikan

This study aims to determine the quality of the Biology instrument items on the digestive system material with the Rasch model for analyzing critical thinking skills. The research employed a quantitative descriptive method involving 63 students of senior high schools in Yogyakarta. The data were collected using a critical thinking skills description test and processed using the Rasch model with the Winstep 5.0.3.4 program. The study shows that the overall validity is acceptable. The item validity did not require improvement in items 14, 3, 8, 13, 12, 1, 4, 10, 6, 2, 7, 11, 15, and 9, and required improvement …

A Study Of The Readiness Of Post-Pandemic Computer-Based Four-Tier Diagnostic Test (Cbftdt): A Review Of Economic Level, School Grades, And Device Accessibility, Edi Istiyono, Wipsar Sunu Brams Dwandaru, Made Rai Shanti Ayub, Duden Saepuzaman, Rizki Zakwandi, Anisyah Rachman, Eri Yusron, Purwoko Haryadi Santoso

Jurnal Penelitian dan Evaluasi Pendidikan

Student readiness for using computer-based assessment was affected by economic level, school grade, and device accessibility. This study analyzed student readiness using a computer-based assessment, the four-tier diagnostic test (CBFTDT). The data in this study were students' responses to a questionnaire consisting of 16 statements about three aspects: mental access, skill access, and usage. The questionnaire consists of 16 statements with three aspects: items, media, and effectivity. The questionnaire was proved valid by experts and its estimated reliability score was 0.99. This study proved that the Student is still not ready to use the computer-based assessment, with an average readiness …

An Analysis Of Interaction Patterns In Students' Research Essay Guidance To Build Six Main Parameters Of The Indonesian National Qualification Framework: Covid-19 Pandemic Period, Undang Rosidin, M. Setyarini, Widyastuti Widyastuti

Jurnal Penelitian dan Evaluasi Pendidikan

This study aims to reveal the interaction patterns and inhibiting factors for the final project guidance in the Mathematics and Natural Sciences Education Department, Universitas Lampung, to develop the six main parameters of the Indonesian National Qualifications Framework (INQF). The study employed a mixed method involving 67 students who composed the final project for the 2021/2022 academic year selected from 314 final project students using the cluster random sampling technique. The data in this study were primary and secondary data obtained by questionnaires, interviews, and documents, analyzed qualitatively, quantitatively, and descriptively. The closed-response questionnaire was used regarding the interaction pattern, …

Comparing Item-Total Correlation And Item-Theta Correlation In Test Item Selection: A Simulation And Empirical Study, Sukaesi Marianti, Ana Rufaida, Nur Hasanah, Sofia Nuryanti

Jurnal Penelitian dan Evaluasi Pendidikan

One of the important processes in the evaluation of the psychometric properties of a test is item selection. The item selection process usually uses a very popular technique called item-total correlation. This study attempts to describe the item-total correlation technique and explore it using a similar technique called item-theta correlation. Both techniques are applied using simulation studies by creating several conditions related to test length and sample size. After the simulation study, the next step is the study using empirical data as an illustration of the results of the simulation study. The results of this study show that there are …

The Online Evaluation Of The Teacher Certification Program Using The Cipp Model, Intan Kemala Sari, Lili Kasmini, Rosdiana Rosdiana, Mayor M. H. Manurung

Jurnal Penelitian dan Evaluasi Pendidikan

The Teacher Certification Program (PPG) is a government program to support competency improvement and standardization of teachers in Indonesia. Program organizers are LPTKs that have met the requirements for further preparation and fulfillment of human resources, infrastructure, and supporting policies. Evaluation of the PPG Program is essential, especially for LPTKs still new to implementation. Evaluation of the PPG Program is carried out using the CIPP Model, an evaluation model in terms of Context, Input, Process, and Product. The evaluation was carried out using a qualitative descriptive approach using observation, documentation, interviews, and survey instruments. The PPG program on the subject …

Assessment Of Geography Learning Process Standards In The Indonesian Curriculum: Instrument Development And Quality, Tuti Amalia, Muhammad Nursa'ban

Jurnal Penelitian dan Evaluasi Pendidikan

The need for mapping the achievement of geography learning process standards in the Indonesian curriculum can provide recommendations in accordance with the conditions in the field. This study aims to determine and describe the quality of geography learning process standard instruments in the Indonesian curriculum, namely: 1) content validity 2) construct validity 3) reliability. This research is instrument development research. The development stage used is the development stage of the instrument test by Heri Retnawati. The process of preparing the instrument, namely: determining the purpose of instrument preparation, searching for relevant theories, compiling indicators of instrument items, compiling instrument items, …

Development Of A Scientific Attitude Measurement Instrument For Dharmacariya Department Students At Stabn Using The Fishbein And Ajzen Model, Rahmat Syah, Daddy Darmawan, Syaiful Syamsuddin

Jurnal Penelitian dan Evaluasi Pendidikan

This study applies the Fishbein and Ajzen model to develop a Scientific Attitude Measurement Instrument for Students, focusing on attitudes, beliefs, and intentions towards scientific behaviors. The quantitative research employs an instrument development method, initially constructing dimensions reflecting various scientific attitudes such as attitudes towards the scientific method, collaboration, openness to knowledge, criticality, respect, honesty, independence, and interest in data analysis. The instrument measures students' scientific attitudes across three main dimensions: attitude, subjective norms, and behavioral control, aligning with the Fishbein and Ajzen model. It covers key aspects like attitudes towards the scientific method, emphasizing rigorous approaches like Popper's Falsification …

Boundaries Of Empirical Approaches In Educational Research, Christopher Olusola Omoregie

Journal of Research Initiatives

This paper critically reviews the research done in education faculties in Nigerian universities. This research, though categorized in postgraduate schools or colleges as mainly in the liberal arts/humanities and the social sciences, depends on the theories and methodologies from other disciplines. The arts and social sciences are disciplines where undergraduates in education take courses in teaching to earn bachelor’s degrees, the postgraduate level offers varied opportunities for educational research to maximize the uniqueness of mixed method research for education.

The Impact Of Teacher Preparedness And Professional Development On Fourth-Grade Students' Science Achievement, Craig L. Mayo, Faye Bradley

Journal of Research Initiatives

Science scores among US fourth-grade students have declined compared to their international counterparts in recent years. Recent results show that teachers are the most impactful influence on student success and accountability. Teacher preparedness and professional development are two key areas that serve as indicators of providing relevant and essential information for students' success. A correlational quantitative study was conducted to assess the relationship between teacher preparedness and professional development on fourth-grade students’ science achievement. The TIMSS 2019 data were secured from the Boston College, TIMSS, and PIRLS International websites. The data was evaluated using the SPSS 27 Hierarchical Linear Regression. …

Reducing Food Scarcity: The Benefits Of Urban Farming, S.A. Claudell, Emilio Mejia

Journal of Nonprofit Innovation

Urban farming can enhance the lives of communities and help reduce food scarcity. This paper presents a conceptual prototype of an efficient urban farming community that can be scaled for a single apartment building or an entire community across all global geoeconomics regions, including densely populated cities and rural, developing towns and communities. When deployed in coordination with smart crop choices, local farm support, and efficient transportation then the result isn’t just sustainability, but also increasing fresh produce accessibility, optimizing nutritional value, eliminating the use of ‘forever chemicals’, reducing transportation costs, and fostering global environmental benefits.

Imagine Doris, who is …

Health Profession Education Students’ Satisfaction Toward Online Courses During The Covid-19 Pandemic, Diantha Soemantri, Zhafirah Salsabila, Vernonia Yorasaki

Makara Journal of Health Research

Background: Given the extensive use of e-learning, especially during the COVID-19 pandemic, a validated instrument is deemed important for continuous evaluation. This study aimed to measure health profession education students’ satisfaction toward the e-learning process within all health sciences cluster (HSC) courses, following further validation of the satisfaction scale for the e-learning process.

Methods: A cross-sectional study using an exploratory factor analysis (EFA) was conducted to validate the 29-item Indonesian version of the satisfaction scale for the e-learning process. That the validated tool was used to measure 2,471 students’ satisfaction toward the e-learning process.

Results: The EFA …