Educational Assessment, Evaluation, and Research Commons^™

Australian University Outcomes: A National Study Comparing Ibdp And Non-Ibdp Students, Kylie Hillman, Daniel Edwards, Leyna Clarke

Higher education research

This study compares the university outcomes of students who completed the International Baccalaureate (IB) Diploma Programme (DP) in Australia compared to non-DP students nationally. The findings are based on population-level data for all students who applied to and entered universities in Australia between 2013 and 2018. To address differences in DP and non-DP student characteristics, the researchers further examined outcomes by groups within the populations that are more alike, for example, by gender and socio-economic status. Across all cohorts, DP students gain offers of university admission, continue to their second year of study, and complete university for all check points …

The Australian Council For Educational Research (Acer) Response To The Australian Universities Accord Panel Interim Report, Australian Council For Educational Research (Acer)

Higher education research

The Australian Council for Educational Research (ACER) recognises the importance of universities, and the higher education sector more broadly, to the prosperity and success of Australia. ACER welcomes the intent and focus of the Universities Accord in creating a system that enables all Australians to consider the option of university education and be supported in embracing this opportunity. In our submission to the Accord Panel, we would like to provide reflection and insight into the following key areas:

• Recognising the higher education ‘life cycle’ begins well before university enrolment.
• Continuing to build the emphasis on equality of opportunity and equity-focussed …

Rethinking Assessment In Response To Generative Artificial Intelligence, Jacob Pearce, Neville Chiavaroli

Higher education research

The use of decision-making support tools during assessments, such as electronic differential diagnosis in examinations, is just the tip of the iceberg when it comes to how technology is currently changing assessment practice. We have reached a transformative stage in the development of artificial intelligence (AI). We can no longer rely on non-invigilated assessments and submitted ‘artefacts’ to demonstrate student learning and competence. This is bringing many long-term demands on educators, course coordinators and curriculum designers, forcing us to rethink assessment approaches. Going forward, we see an important distinction between ‘assisted’ assessments and ‘unassisted’ assessments. With the recent increase and …

Australian General Practice Training Program: National Report On The 2022 National Registrar Survey, Rebecca Taylor, Leyna Clarke, Ali Radloff

Higher education research

The Australian General Practice Training National Registrar Survey (AGPT NRS) is an annual, national survey of GP registrars currently training in the AGPT program. It collects information via an online questionnaire about registrar satisfaction, experience and future career plans. It also collects information about registrars’ demographics and training contexts and other aspects of their training experience. This survey is part of the Department of Health and Aged Care’s (the Department) monitoring and quality improvement activities. From June 27 until August 12, 2022, the Australian Council for Educational Research (ACER) administered the AGPT NRS to registrars enrolled in active training on …

Australian General Practice Training Program: National Report On The 2021 National Registrar Survey, Rebecca Taylor, Leyna Clarke, Ali Radloff

Higher education research

The Australian General Practice Training National Registrar Survey (AGPT NRS) is an annual, national survey of GP registrars currently training in the AGPT program. It collects information via an online questionnaire about registrar satisfaction, experience and future career plans. It also collects information about registrars’ demographics and training contexts and other aspects of their training experience. This survey is part of the Department of Health’s (the Department) monitoring and quality improvement activities. The information collected in the AGPT NRS can be used to assure the quality of training provision in the program, enables continuous improvement and allows responses to be …

Australian General Practice Training Program: National Report On The 2020 National Registrar Survey, Rebecca Taylor, Leyna Clarke, Ali Radloff

Higher education research

The Australian General Practice Training National Registrar Survey (AGPT NRS) is an annual, national survey of GP registrars currently training in the AGPT program. It collects information via an online questionnaire about registrar satisfaction, experience and future career plans. It also collects information about registrars’ demographics and training contexts and other aspects of their training experience. This survey is part of the Department of Health’s (the Department) monitoring and quality improvement activities. From October 12 to December 9, 2020, the Australian Council for Educational Research (ACER) administered the AGPT NRS to registrars enrolled in active training on the AGPT program …

Systematic Review Of The Literature On Professional Education Accreditation, Neville Chiavaroli, Kate Reid, Pru Mitchell, Jenny Trevitt

Higher education research

This report is critical and systematic review of the literature on professional education accreditation models, associated methods, and the effectiveness and impact of each model. The review was conducted to inform a review and update of the Royal College of Vetinerary Surgeons (RCVS) standards of accreditation for veterinary degrees. The purpose of the study was to rapidly appraise the evidence for the effectiveness, impact, and feasibility of different accreditation approaches, in order to inform best practices for the accreditation of professional education programs. The study focused on accreditation for programs which produce practice-ready graduates, including for veterinary programs. The authors …

International Graduate Outcomes Survey 2018 Final Report, Darren Matthews, Ali Radloff, Jo Doyle, Leyna Clarke

Higher education research

The 2018 International Graduates Outcomes Survey (IGOS) provides an insight into international graduates from Australian universities post-study employment, work readiness, and reflections on course experiences, study motivation and overall satisfaction. Over 10,000 international students who graduated from an Australian university in the last decade (international graduates) were asked a range of questions about their current employment status and their opinions on their Australian qualification. In brief the results indicate that: 90 per cent of graduates who are available for employment are currently employed - incorporating 93 per cent who returned home and 85 percent who were still in Australia; 77 …

Australian Higher Education Equity Ranking Project: Final Report, Tim Pitman, Paul Koshy, Daniel Edwards, Liang-Cheng Zhang, Julie Mcmillan

Higher education research

This report details the findings of a feasibility study for the Department of Education and Training (DET) into the development of a higher education student equity ranking index. The purpose of study was to determine whether it was possible to measure higher education equity performance at the institutional level and convey each institution’s relative performance through an ‘equity rank’. The ranking was to be based on institutional performance in regard to equity-group students, including students from low socio-economic backgrounds; students from regional/remote areas of Australia; Indigenous students; students with disability; and students from non-English speaking backgrounds.

Australian General Practice Training Program National Report On The 2018 National Registrar Survey, Rebecca Taylor, Leyna Clarke, Daniel Edwards

Higher education research

The Australian General Practice Training National Registrar Survey (AGPT NRS) is an annual, national survey of GP registrars currently training in the AGPT program. It collects information via an online questionnaire about registrar satisfaction, experience and future career plans. It also collects information about registrars’ demographics and training contexts and other aspects of their training experience. This survey was previously known as the AGPT Registrar Satisfaction Survey (AGPT RSS). From July 31 to September 7 2018, the Australian Council for Educational Research (ACER) administered the AGPT NRS to registrars enrolled in active training on the AGPT program across 11 training …

The Japan University Experience Survey (Jues), Darren Matthews, Leyna Clarke

Higher education research

An overview of the Japan University Experience Survey (JUES) including its background and aims, and the ACER - Kawaijuku Group partnership. The presentation discusses the development, delivery, reporting and results of the survey, and considers Japanese and Australian UES/SES comparisons and the potential for benchmarking.

Assessment Of University Support Services For Regional And Remote Students On Transition To University: Final Report, Darren Matthews, Gina Milgate, Leyna Clarke

Higher education research

This report details the development and piloting of an Assessment of university support services for regional and remote students on transition to university. The project forms part of the Australian Government’s response to the Independent Review into Regional, Rural and Remote Education led by Emeritus Professor John Halsey. In its recommendations, the Review highlighted how critical it is to “Support RRR [regional, rural and remote] students to make successful transitions from school to university, training, employment and combinations of them”. In relation to this recommendation, this project is designed to support the Australian Government’s response to the Halsey Review by …

Australian General Practice Training Program National Report On The 2017 National Registrar Survey, Rebecca Taylor, Ali Radloff, Daniel Edwards, Leyna Clarke

Higher education research

The Australian General Practice Training National Registrar Survey (AGPT NRS) is an annual, national survey of GP registrars currently training in the AGPT program that collects information about registrar satisfaction, experience and future career plans. This survey is part of the Department of Health’s (the Department) monitoring and quality improvement activities and the information can be used to assure the quality of training provision, enable continuous improvement and allow results to be benchmarked nationally. Previously known as the AGPT Registrar Satisfaction Survey (AGPT RSS), the AGPT NRS also collects information on registrars’ demographics and training contexts and other aspects of …

Final Report: Development Of An Online Engagement Scale, Darren Matthews, Ling Tan, Daniel Edwards

Higher education research

The Commonwealth Department of Education and Training commissioned the Australian Council for Educational Research (ACER) to review the Learner Engagement Scale (LES), a key measure of student experience. It is administered as part of the Student Experience Survey, an annual survey of a sample of the 400 000 first and final year undergraduates studying at Australian Higher Education institutions – 80 per cent on-campus and around 8 per cent off-campus. Over the years, LES results have consistently shown substantially lower engagement levels in off-campus students – far in excess of the variation between teaching modes that might be expected – …

Assessment And Feedback In Higher Education: A Review Of Literature For The Higher Education Academy, Brad Jackel, Jacob Pearce, Ali Radloff, Daniel Edwards

Higher education research

This literature review has been undertaken for the Higher Education Academy (HEA). It explores recent scholarly contributions in the area of assessment and feedback in higher education (HE). The report is structured as follows. The introduction provides some background and a context for the approach taken to developing this review and the ‘lens’ through which the review was undertaken. The review then comprises three broad sections: ‘The fundamentals’ – a section that explores the significant principles underpinning best practice in assessment. ‘Practice and innovations’ – an exploration of recent literature that demonstrates the kinds of assessment practice currently being undertaken …

Investigating The Relationship Between Equity And Graduate Outcomes In Australia, Sarah Richardson, Dawn Bennett, Lynne Roberts

Higher education research

Australian higher education equity policy focusses mostly on access and participation with the implicit assumption that disadvantage will be ameliorated through educational achievement. Less is known as to whether patterns of disadvantage continue post-completion. In a context in which graduate employability is becoming an important yardstick against which to measure institutional effectiveness, this question is of fundamental importance to higher education equity practitioners and policymakers. This study employed Commonwealth graduate outcome data to investigate relationships between disadvantage and graduate outcomes in Australia, with disadvantage defined as a graduate belonging to one or more of the following groups – low SES, …

Higher Education Standards Panel Consultation : Transparency Of Higher Education Admissions Processes: Response From The Australian Council For Educational Research, Daniel Edwards, Brad Jackel, Australian Council For Educational Research (Acer)

Higher education research

This document has been prepared by the Australian Council for Educational Research (ACER) in response to the Higher Education Standards Panel’s Consultation on the Transparency of Higher Education Admissions Processes. In developing the document, ACER has drawn on previous experience working in admissions research and admissions testing, and on discussions over time with universities in Australia as a result of this work relating to the issues of relevance to this consultation. This document explores four key areas identified by ACER as critical to improving the admissions processes of higher education providers in Australia. In developing this document, we have approached …

Mapping Researcher Mobility: Measuring Research Collaboration Among Apec Economies, Ali Radloff

Higher education research

Researcher mobility is an important form of cross-border education (CBE). It has the potential to generate significant benefits for economies as expert scholars and scientists come together to solve some of the most pressing challenges in the contemporary world. Among members of the Asia-Pacific Economic Community (APEC) researcher mobility can strengthen ties between economies and enable the minimisation of barriers to economic growth and sustainability. There are not currently any comparable or rigorous data available on researcher mobility among APEC economies. Proxy measures are needed to gain a sense of the extent to which researchers in APEC economies are collaborating …

Developing A Global Health Assessment Collaboration: Ancillary Report, Daniel Edwards, Jacob Pearce, David Wilkinson

Higher education research

This document reports on a project designed to develop an assessment collaboration between medical schools in both Australia and the United Kingdom. The project was funded by the Office for Learning and Teaching (OLT), utilising surplus funding from a broader assessment collaboration project – the Australian Medical Assessment Collaboration (OLT ID12-2482). The Global Health Assessment Collaboration (GHAC) involved five universities in Australia and the United Kingdom (UK). It developed an assessment framework and item specifications, undertook assessment item drafting workshops, built in a process of review and resulted in the development of a focused suite of assessment items. This report …

National Research On The Postgraduate Student Experience : Case Presentation: Volume 1: First Year Postgraduate Student Experience, Gary Hamlin, Shelley Kinash, Linda Crane, Amy Bannatyne, Madelaine-Marie Judd, Ashley Stark, Bill Eckersley, Helen Partridge, Ken Udas

Higher education research

This is volume one of a set of case studies that explore the postgraduate student experience. The theme of this case study is first year postgraduate student experience and is based on experiences derived from student engagement breakfasts, interviews, and focus groups with 366 people across the stakeholder groups of postgraduate students, educators, and university executives from 26 institutions. The case studies constitute part of the output from the project, Engaging postgraduate students and supporting higher education to enhance the 21st century student experience.

National Research On The Postgraduate Student Experience : Case Presentation : Volume 2 : Postgraduate Student Diversity, Bill Eckersley, Linda Crane, Shelley Kinash, Amy Bannatyne, Gary Hamlin, Helen Partridge, Sarah Richardson, Harry Rolf, Ken Udas

Higher education research

This is volume two of a set of three case studies that explore the postgraduate student experience. The theme of this case study is postgraduate student diversity and is based on experiences derived from student engagement breakfasts, interviews, and focus groups with 366 people across the stakeholder groups of postgraduate students, educators, and university executives from 26 institutions. The case studies constitute part of the output from the project, Engaging postgraduate students and supporting higher education to enhance the 21st century student experience.

Researcher Mobility Workshop Report: Researcher Mobility Among Apec Economies, Sarah Richardson, Julie Mcmillan, Ren Yi

Higher education research

There is growing acknowledgement around the world that contemporary research is increasingly – and needs to be – international. In a globalised world common problems can only be solved through the sharing of expertise and pooling of resources. At the same time, research developments are critical in stimulating economic growth in knowledge economies. Ensuring that researchers can access best practice through international research collaboration is essential in facilitating innovation. Policies and structures can go a long way to supporting increased international research collaboration. Online communication technologies have become a vital part of researcher mobility, enabling researchers to unite in conducting …

Agpt Registrar Satisfaction Survey November 2015, Rebecca Taylor, Ali Radloff, Jennifer Hong, Daniel Edwards

Higher education research

The Australian General Practice Training (AGPT) Registrar Satisfaction Survey (RSS) is used for ensuring continuous improvement in the training of doctors in the AGPT program. It was developed to gauge the level of registrar satisfaction with the quality of their training, with training providers, and with career progression.

Completing University In A Growing Sector: Is Equity An Issue?, Daniel Edwards, Julie Mcmillan

Higher education research

Access to university has always been an issue for students from disadvantaged backgrounds. In the recent context of an expanding higher education system in Australia, some accessibility issues have been alleviated. This context offers an opportunity to explore the pathways of disadvantaged students through university. In this expanded system, will disadvantaged students be more or less likely to complete university? Will demographic or enrolment characteristics influence the likelihood of these students to complete? This report details the findings from a research project funded by the National Centre for Student Equity in Higher Education (NCSEHE) that explores new data tracking student …

Work Integrated Learning In Stem In Australian Universities: Final Report: Submitted To The Office Of The Chief Scientist, Daniel Edwards, Kate Perkins, Jacob Pearce, Jennifer Hong

Higher education research

The Australian Council for Educational Research (ACER) undertook this study for the Office of the Chief Scientist (OCS). It explores the practice and application of Work Integrated Learning (WIL) in STEM, with a particular focus on natural and physical sciences, information technology, and agriculture departments in Australian universities. The project involved a detailed ‘stocktake’ of WIL in practice in these disciplines, with collection of information by interview, survey instruments, consultation with stakeholders and literature reviews. Every university in Australia was visited as part of this project, with interviews and consultation sessions gathering insight from more than 120 academics and support …

Enhancing Cross-Border Higher Education Institution Mobility In The Apec Region, Sarah Richardson

Higher education research

This report presents findings of a study of the policy context around higher education institution (HEI) mobility in eight APEC economies – China, Indonesia, Malaysia, Mexico, Peru, the Philippines, Singapore and Viet Nam. The report provides insights into different policy contexts gathered from desktop research and from interviews with 68 participants. These all had direct involvement in HEI mobility in the eight economies in the study, including through senior positions in ministries of education, in bodies aligned to ministries, in accreditation and quality assurance agencies, and in foreign and local HEIs. The report identifies regulations, practices, policies, legislative contexts and …

Implementing Common Assessment: Lessons And Models From Amac Developed By The Australian Medical Assessment Collaboration, Daniel Edwards

Higher education research

The aim of this document is to provide insight into the implementation of common assessments in higher education in order to assist in future work on conducting these kinds of projects. The discussion here draws heavily on the AMAC experience, attempting to broaden the learning from this project for use in future collaborations. The focus of this project has been on medical education, and as such, much of the detail is related to this field. However, it is hoped that the general ideas discussed here can be seen as informative for other fields and disciplines in higher education and at …

Australian Medical Assessment Collaboration: From Proof Of Concept To Proof Of Sustainability: Final Report 2014, Daniel Edwards, David Wilkinson

Higher education research

This is the final report for AMAC-2, entitled Australian Medical Assessment Collaboration: from proof of concept to proof of sustainability (OLT project ID12-2482). This project advanced previous work funded by the ALTC and was undertaken from early 2013 to mid 2014. AMAC-2 took the proof of concept achieved through the initial AMAC project with the aim of building an ongoing, sustainable and successful collaboration between medical schools in Australia and New Zealand.

Determining The Quality Of Assessment Items In Collaborations: Aspects To Discuss To Reach Agreement Developed By The Australian Medical Assessment Collaboration, Lambert Schuwirth, Jacob Pearce

Higher education research

The Australian Medical Assessment Collaboration (AMAC) project, funded by the Office of Learning and Teaching, seeks to provide an infrastructure and a road map to support collaboration between Australian medical schools in matters of assessment. This may not seem very new perhaps, because there are already several collaborations taking place in Australia, and, typically, they relate to joint item banks, (such as the IDEAL consortium), or joint test administration, (such as the International Foundation of Medicine tests). The AMAC project seeks to build on these existing collaborations in two ways: first, by tying these initiatives together and thus bundling the …

Promoting Regional Education Services Integration: Apec University Associations Cross-Border Education Cooperation Workshop: Discussion Paper, Sarah Richardson, Ali Radloff

Higher education research

This discussion paper was prepared to inform the APEC workshop on Promoting Regional Education Services Integration: APEC University Associations Cross-Border Education Cooperation held in Kuala Lumpur from 20–22 May 2014. The workshop brings together university associations to support, pursue and work towards the achievement of the priorities outlined by APEC Economic Leaders. This includes the recognition of best practice in cross-border education (CBE), the identification of existing barriers to CBE, and an examination of ways to progress the priority areas identified. The paper highlights the key considerations for APEC economies in strengthening collaboration around CBE in the university sector and …