Fri, Nov 22, 2024
[Archive]
Volume 17, Issue 55 (2024)
JMED 2024, 17(55): 120-128 |
Back to browse issues page
Ethics code: IR.IUMS.REC 1395.30150
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Ravaghi H, Janani L, Mohamadi R, Mazaherinezhad A, Rafiei N. Educational ranking model of faculties in Iranian universities of medical sciences. JMED 2024; 17 (55) :120-128
URL: http://edujournal.zums.ac.ir/article-1-1635-en.html
URL: http://edujournal.zums.ac.ir/article-1-1635-en.html
Associate Professor, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran.
Keywords: educational ranking, university, educational services, graduate education, postgraduate education, educational field
Full-Text [PDF 558 kb]
(79 Downloads)
| Abstract (HTML) (184 Views)
Full-Text: (24 Views)
Abstract
Background & Objective: It is crucial to have a transparent and comprehensive tool in place to ensure accurate information on the educational performance of faculties. With this in mind, the aim of this study was to develop a model for ranking the educational performance of faculties in Iranian medical universities.
Material & Methods: This study involved the development of an educational ranking tool for the faculties, which was carried out in 4 stages: compilation of indicators, conducting a focus group discussion, utilizing the Delphi technique, and finally consulting with an expert panel. In the given process, the tool's primary indicators were gathered based on various factors, such as the "educational ranking of medical sciences universities," the "ranking of educational services of universities," and the “postgraduate education performance evaluation system in medical sciences universities." These indicators were designed over a course of 17 sessions. After compiling the primary indicators, the indicators were thoroughly reviewed in four sessions in the presence of educational experts. Subsequently, the Delphi technique was employed, and eight expert panel meetings were held to finalize the indicators and areas, the weight of each indicator and domain, data collection methods, and the method of ranking the faculties. Finally, a scoring guide was prepared for all criteria to ensure a fair and objective evaluation process.
Results: The educational ranking model of faculties was designed to evaluate three educational fields - "educational services", "postgraduate education", and "the educational field". The educational field was assessed based on 16 indicators in five areas, including education development, governance, education management, quality development of education, and moving in line with the comprehensive scientific map of the country. These indicators were carefully chosen to ensure a comprehensive evaluation of the faculties' educational excellence. The educational services sector has developed 11 checklists across five key areas: objective factors, reliability, accountability and responsibility, assurance, and empathy. Additionally, the evaluation of postgraduate education performance includes 36 indicators across seven areas: educational program, student evaluation, students, faculty members, educational resources, program evaluation, and senior and executive management.
Conclusion: An educational ranking tool has been developed for faculties in medical science universities. This tool evaluates all educational fields of faculties, including educational services, postgraduate education, and the educational field. It also offers the possibility of comparing faculties within a university. The use of this ranking model can lead to improvements in both quantitative and qualitative educational indicators and ultimately improve the educational rank of the university among other medical science universities.
A comprehensive ranking system that examines all aspects of a university's abilities can provide valuable insights into its strengths, weaknesses, and future development path (3). The ranking results can impact the organizational mission, strategies, employees, and public relations (4) and significantly influence the decision-making of postgraduate students (5, 6). A principled evaluation that strengthens strengths and corrects weaknesses can serve as a basis for educational decisions and plans, leading to the improvement of the academic level of the university (7). In order to improve the quality of educational services, ranking models should consider a comprehensive set of criteria and indicators (8). Hence, producing a reliable and comprehensive tool for evaluating the performance of universities and colleges is crucial (9).
In Iran, the first ranking of the country's medical schools was conducted by the Vice-Chancellor of Education and Student Affairs of the Ministry of Health in 2000. The ranking was based on three areas: education, research, and facilities and equipment. In 2002, a review of the status of providing educational services in universities of medical sciences led to the division of the Department of Medical Sciences' disciplines into ten main groups and the ranking of all faculties providing these disciplines at the national level (10). In 2005, the educational ranking of the country's universities of medical sciences was conducted by aggregating the results of the group-disciplines ranking (11). The Ministry of Health Education Vice-Chancellor implemented the educational ranking of medical sciences universities in 2010, 2014, and 2019, but one of the limitations of this ranking was the exclusion of all aspects of education in universities based on designed criteria (10, 12, 13).
Based on the literature review, no tool has been designed in Iran to rank the individual faculties of a university in terms of education. Existing tools either rank universities or similar colleges across the country, such as medical schools, but these rankings are not solely focused on education. Thus, this study aimed to develop an educational ranking model for faculties in Iranian Universities of Medical Sciences in order to prepare them for national rankings and to facilitate comparisons between faculties within a university.
Material & Methods
Design and setting(s)
The design of the educational ranking tool for faculties in medical sciences universities in Iran was conducted in 2017 as part of this study.
Participants and sampling
The study selected 43 educational experts as participants for the Focus Group Discussion (FGD) sessions and Delphi technique. These experts were chosen based on the purpose of the study, including vice chancellors and education personnel of the university, vice chancellors, education officers, and education personnel of the faculties who had at least 5 years of experience in the education department of the Iran University of Medical Sciences. The expert panel meetings consisted of six educational experts, including the Vice Chancellor of the University, the Director of Postgraduate Studies, the Director and two members of the academic staff of EDC, and the Deputy Director of Postgraduate Studies.
Tools/Instruments
The research team designed and compiled the initial educational ranking tool for faculties as a questionnaire with 63 closed questions, based on the educational ranking models of universities. Each question was accompanied by two separate sections titled "importance" and "feasibility" in the form of a 9-mode Likert scale. The experts were required to assign a score between 1 and 9 for each question related to these categories, with 1 representing the least importance and 9 representing the greatest importance and implementation capability. The content validity of the tool was confirmed by the research team.
Data collection methods
The data collection process for compiling the tool was conducted in four stages: compilation of indicators, FGD, Delphi technique, and expert panel. The stages are as follows:
Compilation of indicators
During this stage, the research team prepared the initial draft of the educational ranking tool for faculties in Iranian medical sciences universities. The team designed educational indicators that could be evaluated at the faculty level over the course of 17 sessions, based on tools such as the "educational ranking of medical sciences universities" (12), "ranking of educational services of universities" (14), and "postgraduate education performance evaluation system in medical sciences universities" (15).
Focus group discussion
Four 70-minute FGD sessions were held to gather new ideas from educational experts. The time and place of the meetings were determined by agreement among all the participants and held at the vice chancellor's office of the university. The first and second sessions were attended by 15 educational experts, including university vice chancellors, faculty vice chancellors, the director of educational affairs and postgraduate studies, the director of the Medical Education Development Center (EDC), the director of faculty affairs, the director of the Talent Office, and the director of continuing education. The meeting was chaired by the secretary, and all participants had at least 5 years of experience in the educational field of the university. The objectives of the educational ranking of the faculties were presented, and the extracted indicators were introduced. Attendees were asked to provide their suggested indicators and opinions for the next meeting. In the second meeting, held 14 days later, the secretary collected the suggested opinions and reviewed the opinions of the attendees about the existing indicators and proposed indicators. Two additional FGD sessions were held, with 28 education personnel and postgraduate education experts from all faculties. Also participating were education personnel, the office manager of postgraduate education, the office manager of the specialized and sub-specialist education department, the office manager of the admission and registration unit, and the office manager of the postgraduate department affiliated with the university's vice chancellor of education. The meetings were conducted by four members of the scientific EDC board, and all participants had at least 5 years of experience in university education. The meetings were conducted smoothly and efficiently.
Delphi technique
After collecting suggested comments in the FGD sessions, the indicators were designed in the form of a questionnaire. Each indicator was evaluated and scored in terms of importance and feasibility using a 9-point Likert scale. The developed questionnaire was sent via e-mail to 20 educational experts, who were given two weeks to respond with their answers.
Panel of experts
The expert panel, consisting of six educational experts, held eight meetings at the Office of the Vice Chancellor of University Education. The panel decided on the indicators and domains, the weight of each indicator and domain, data collection methods, and the method of ranking the faculties. Each meeting lasted approximately two hours. In the first two meetings, the proposed indicators and opinions collected from the FGD sessions and Delphi technique were reviewed. After selecting the desired indicators, four meetings were held to weigh each of the indicators and domains of educational fields, educational services, and postgraduate education. The weights of each index and domain were calculated by a biostatistician and reviewed in the meetings. The weighting of the indicators was done based on the main tools' weighting while considering the faculties' conditions and expert opinions. The 7th and 8th meetings focused on compiling the evaluation guide for faculties. The panel reviewed and formulated the method of evaluating and ranking faculties, the selection of evaluators from each faculty, and the way of scoring each faculty based on indicators. A scoring guide was prepared for all the criteria, which included all the scoring and weighting points, enabling each faculty to independently calculate its score.
Data analysis
During the Delphi phase, participants were asked to provide feedback on each indicator in the designed model based on the 9-point Likert scale for the two criteria of "importance" and "feasibility." The median index was used for data analysis because it is not affected by outlier data. After obtaining the median of the questions through SPSS.18 software, if the median score of each question was between 1 and 3, the desired index was removed from the model. If the median score was between 4 and 6, the desired question was included in the next stage. If the median score was 7 or higher, the question was accepted in the first round and included in the final model (16, 17).
Results
One of the main results of this study was the development of an educational ranking model for the faculties of medical sciences universities in three educational fields: educational services, postgraduate education, and the educational field. The ranking model evaluates 16 indicators in the field of college education across five areas: education development, governance, education management, quality development of education, and alignment with the comprehensive scientific map of the country. The field of faculty educational services includes 11 checklists across five areas: objective factors, reliability, accountability and responsibility, assurance, and empathy. The field of postgraduate education performance evaluation measures 36 indicators in the field of postgraduate education. Moreover, it was noted that indicators related to the educational field and applicable to all faculties should be compiled (Figure 1).
Figure 1. Educational ranking model of faculties in Iranian universities of medical sciences
1- Indicators of the educational field
The 16 forms compiled for the educational ranking model of faculties in medical sciences universities included: newly established programs/degrees as well as existing programs, programs/degrees considered invalid, annual performance report according to the long-term plan, faculty educational council, faculty postgraduate education council, faculty recruitment program, managerial stability, transparency and availability of information on the faculty's website, activities of the Education Development Office, opinion of the University Vice-Chancellor's directors regarding the faculty's performance, Shahid Motahari Festival, special activities of the faculty, professor evaluation project by students, planning and implementation aligned with the goals of the comprehensive scientific map of the country, the faculty's response to society's needs, and professional ethics (Table 1).
3- Indicators of postgraduate education
The field of postgraduate education performance evaluation includes 36 indicators across seven areas: educational program, student evaluation, students, faculty members, educational resources, program evaluation, and senior and executive management (Table 3).
4- Scoring and weighting of model indicators
Considering the existence of three separate educational fields: educational services and postgraduate education, each with different scoring systems in the university version, a three-ranking system was used for the faculty education section. Eleven separate checklists of 100 points were considered for the faculty educational services section, with equal weighting. The postgraduate education tool had 36 indicators across seven fields, with 100 points allocated for total fields. To summarize the final points of each faculty across all three domains—educational fields, educational services, and postgraduate education—the fields were weighted at 45%, 30%, and 25%, respectively.
The ranking of each faculty is obtained by combining the points of all three domains: educational fields, educational services, and postgraduate education. This is suggested to measure the current status of each faculty across all indicators in the first year of implementing the ranking and provide the necessary training to related experts. In the second year, the improvement of each index should be measured compared to the current status of the faculty in the first year, with the faculty that shows the most improvement receiving the top rank. From the second year onward, the degree of improvement of each index compared to the previous year of the same faculty, and the degree of improvement or maintaining the indicators at the desired level, should be ranked in comparison with other faculties.
It is also possible to introduce the best faculty in any educational field, educational services, or postgraduate studies.
Discussion
The primary objective of this study was to create an educational ranking model for the medical science faculties within Iranian universities. The most significant aspect of this study is its ability to determine the grade and rank of all faculties based on completely objective criteria. The overall rank, as well as the rank in each of the "educational fields," "educational services," and “postgraduate education," can be extracted for each faculty.
To adequately measure the educational processes of colleges, the researchers aimed to develop suitable indicators. The current educational ranking systems, educational services, and postgraduate education were primarily designed for the educational ranking of universities, and many of the indicators could not be used at the faculty level. Therefore, the research team held several meetings to extract indicators that can be calculated in the faculties and also sought the opinions of experts.
According to a study conducted by Safari Farfar (18), the ranking of educational groups in Iran's state universities was based on a set of criteria in various fields. The input field criteria included faculty members, admitted students, structural spaces, facilities and equipment, and management and leadership, which were evaluated using 24 indicators. The process field criteria were evaluated based on teaching and learning, utilization of information and communication technology, and academic progress using 16 indicators. Meanwhile, the output field criteria included the publication of articles and books, the implementation of research projects and patents, winning awards, membership in scientific societies, the duration of study, and the GPA of postgraduates, evaluated using 21 indicators. Finally, the consequences field criteria evaluated the continuation of education for postgraduates, employment and entrepreneurship for postgraduates, and the scientific and professional impact, using 8 indicators in total.
At the international level, the Academic Ranking of World Universities (ARWU), also known as the Shanghai Ranking, evaluates universities based on several criteria. These include the number of articles published in Nature and Science magazines, the number of Nobel Prize winners, and the Fields Prize in the field of mathematics. Meanwhile, the Times Higher Education (THE) ranking system has been in place since 2004 and evaluates universities based on 13 indicators. These include teaching (an indicator of the learning environment), citations (an indicator of the effectiveness of scientific research), research (volume, income, and credit), industrial income and innovation, and international factors such as faculty members and foreign students. The Quacquarelli Symonds (QS) ranking system evaluates universities based on the citation rate of each faculty member to Thomson Scientific and Scopus databases, the ratio of the number of students to professors, the number of foreign students enrolled, and the number of researchers and foreign professors hired. This ranking system was the first international ranking system approved by the Academic Ranking and Excellence (IREG Observatory) in 2013 and has become one of the most reliable evaluation systems. However, it should be noted that the focus of international ranking systems is primarily on research, whereas the present study focuses on the educational aspects of colleges (19–21).
In a previous study, the ranking indicators of medical science groups in 2005 were compiled into 10 groups, including medicine, dentistry, pharmacy, nursing, nutritional sciences, management and information, paramedicine, rehabilitation, and health. The study evaluated faculties based on three areas: the input criteria, including the average score balance of students, the absolute and per capita number of faculty members, and facilities and equipment such as library resources, physical facilities of the faculty, computers, and teaching facilities of hospitals. The process criteria included student management, faculty member management, system management, a supervisor, and an introduction meeting upon arrival. Meanwhile, the output criteria included graduation rate, continuing education, and the publication of books and articles. Although the criteria in this study are much wider, some aspects, such as compliance with educational rules and the percentage of students who graduated on time, are similar to the previous study (11). In 2010 and 2014, the Ministry of Health's Education Vice-Chancellor implemented the educational ranking of medical sciences universities throughout Iran. However, one limitation of this ranking is that it does not include all aspects of education in universities based on designed criteria (12). In the present study, the indicators designed in the educational field were derived from university educational ranking indicators, but they were adjusted to evaluate faculty-level performance. Additionally, in 2019, the Ministry of Health's Education Vice-Chancellor introduced the educational ranking of universities with a new approach, which ranked medical sciences universities based on three areas: effective outputs, functional areas, and the implementation of education transformation and innovation packages (13). The new approach aimed to provide a more comprehensive assessment of medical science universities.
Conclusion
For the first time, a comprehensive educational ranking tool was compiled for the faculties of medical science universities. This tool evaluates all fields of education, including the field of education with 16 indicators, the field of educational services with 11 checklists, and the field of postgraduate education with 36 indicators. For a university to grow and develop while also meeting the expectations of students, faculty members, and employees, it must have a written program for monitoring and evaluating its educational system. The existence of this ranking model will provide an incentive for faculties to engage in positive competition in line with standards, ultimately leading to the improvement of quantitative and qualitative indicators of education and the improvement of the educational rank of the university among other medical sciences universities.
Ethical considerations
All of the questionnaires were anonymous, and all participants in the study were assured that they had the right to withdraw from the study at any time. This article is the result of a research project entitled "Development of the Educational Ranking System of the Faculties of Iran University of Medical Sciences," which was approved by Iran University of Medical Sciences and assigned the ethics code number IR.IUMS.REC1395.30150 by the National Committee of Ethics in Biomedical Research.
Artificial intelligence utilization for article writing
No.
Acknowledgments
We would like to express our gratitude to all of the reviewers, respected colleagues of the Vice-Chancellor of Education, as well as the faculties of Iran University of Medical Sciences who cooperated in the implementation of this study.
Conflict of interest statement
The authors declare that they have no conflict of interest.
Author contributions
H.R. conceived and designed the study, compiled the indicators, and critically reviewed the article before submission. L.J. compiled the indicators, scored the indicators, and critically reviewed the article before submission. R.M. compiled the indicators, and critically reviewed the article before submission. A.M. compiled the indicators, and critically reviewed the article before submission. N.R. designed the study, compiled the indicators, wrote the article and provided final approval before submission. All authors have approved the final version of the article.
Supporting resources
No financial support was received for this research.
Data availability statement
Data and materials could be made available upon reasonable request.
Background & Objective: It is crucial to have a transparent and comprehensive tool in place to ensure accurate information on the educational performance of faculties. With this in mind, the aim of this study was to develop a model for ranking the educational performance of faculties in Iranian medical universities.
Material & Methods: This study involved the development of an educational ranking tool for the faculties, which was carried out in 4 stages: compilation of indicators, conducting a focus group discussion, utilizing the Delphi technique, and finally consulting with an expert panel. In the given process, the tool's primary indicators were gathered based on various factors, such as the "educational ranking of medical sciences universities," the "ranking of educational services of universities," and the “postgraduate education performance evaluation system in medical sciences universities." These indicators were designed over a course of 17 sessions. After compiling the primary indicators, the indicators were thoroughly reviewed in four sessions in the presence of educational experts. Subsequently, the Delphi technique was employed, and eight expert panel meetings were held to finalize the indicators and areas, the weight of each indicator and domain, data collection methods, and the method of ranking the faculties. Finally, a scoring guide was prepared for all criteria to ensure a fair and objective evaluation process.
Results: The educational ranking model of faculties was designed to evaluate three educational fields - "educational services", "postgraduate education", and "the educational field". The educational field was assessed based on 16 indicators in five areas, including education development, governance, education management, quality development of education, and moving in line with the comprehensive scientific map of the country. These indicators were carefully chosen to ensure a comprehensive evaluation of the faculties' educational excellence. The educational services sector has developed 11 checklists across five key areas: objective factors, reliability, accountability and responsibility, assurance, and empathy. Additionally, the evaluation of postgraduate education performance includes 36 indicators across seven areas: educational program, student evaluation, students, faculty members, educational resources, program evaluation, and senior and executive management.
Conclusion: An educational ranking tool has been developed for faculties in medical science universities. This tool evaluates all educational fields of faculties, including educational services, postgraduate education, and the educational field. It also offers the possibility of comparing faculties within a university. The use of this ranking model can lead to improvements in both quantitative and qualitative educational indicators and ultimately improve the educational rank of the university among other medical science universities.
Introduction
Evaluating and determining the performance of universities has always been a topic of interest among researchers. Ranking is one of the methods used to evaluate the status of educational centers (1). Currently, there is a growing emphasis on evaluating the scientific quality of universities. University ranking systems provide valuable information for students, universities, and policymakers in the education field, highlighting the strengths and weaknesses of universities in the global education market (2).A comprehensive ranking system that examines all aspects of a university's abilities can provide valuable insights into its strengths, weaknesses, and future development path (3). The ranking results can impact the organizational mission, strategies, employees, and public relations (4) and significantly influence the decision-making of postgraduate students (5, 6). A principled evaluation that strengthens strengths and corrects weaknesses can serve as a basis for educational decisions and plans, leading to the improvement of the academic level of the university (7). In order to improve the quality of educational services, ranking models should consider a comprehensive set of criteria and indicators (8). Hence, producing a reliable and comprehensive tool for evaluating the performance of universities and colleges is crucial (9).
In Iran, the first ranking of the country's medical schools was conducted by the Vice-Chancellor of Education and Student Affairs of the Ministry of Health in 2000. The ranking was based on three areas: education, research, and facilities and equipment. In 2002, a review of the status of providing educational services in universities of medical sciences led to the division of the Department of Medical Sciences' disciplines into ten main groups and the ranking of all faculties providing these disciplines at the national level (10). In 2005, the educational ranking of the country's universities of medical sciences was conducted by aggregating the results of the group-disciplines ranking (11). The Ministry of Health Education Vice-Chancellor implemented the educational ranking of medical sciences universities in 2010, 2014, and 2019, but one of the limitations of this ranking was the exclusion of all aspects of education in universities based on designed criteria (10, 12, 13).
Based on the literature review, no tool has been designed in Iran to rank the individual faculties of a university in terms of education. Existing tools either rank universities or similar colleges across the country, such as medical schools, but these rankings are not solely focused on education. Thus, this study aimed to develop an educational ranking model for faculties in Iranian Universities of Medical Sciences in order to prepare them for national rankings and to facilitate comparisons between faculties within a university.
Material & Methods
Design and setting(s)
The design of the educational ranking tool for faculties in medical sciences universities in Iran was conducted in 2017 as part of this study.
Participants and sampling
The study selected 43 educational experts as participants for the Focus Group Discussion (FGD) sessions and Delphi technique. These experts were chosen based on the purpose of the study, including vice chancellors and education personnel of the university, vice chancellors, education officers, and education personnel of the faculties who had at least 5 years of experience in the education department of the Iran University of Medical Sciences. The expert panel meetings consisted of six educational experts, including the Vice Chancellor of the University, the Director of Postgraduate Studies, the Director and two members of the academic staff of EDC, and the Deputy Director of Postgraduate Studies.
Tools/Instruments
The research team designed and compiled the initial educational ranking tool for faculties as a questionnaire with 63 closed questions, based on the educational ranking models of universities. Each question was accompanied by two separate sections titled "importance" and "feasibility" in the form of a 9-mode Likert scale. The experts were required to assign a score between 1 and 9 for each question related to these categories, with 1 representing the least importance and 9 representing the greatest importance and implementation capability. The content validity of the tool was confirmed by the research team.
Data collection methods
The data collection process for compiling the tool was conducted in four stages: compilation of indicators, FGD, Delphi technique, and expert panel. The stages are as follows:
Compilation of indicators
During this stage, the research team prepared the initial draft of the educational ranking tool for faculties in Iranian medical sciences universities. The team designed educational indicators that could be evaluated at the faculty level over the course of 17 sessions, based on tools such as the "educational ranking of medical sciences universities" (12), "ranking of educational services of universities" (14), and "postgraduate education performance evaluation system in medical sciences universities" (15).
Focus group discussion
Four 70-minute FGD sessions were held to gather new ideas from educational experts. The time and place of the meetings were determined by agreement among all the participants and held at the vice chancellor's office of the university. The first and second sessions were attended by 15 educational experts, including university vice chancellors, faculty vice chancellors, the director of educational affairs and postgraduate studies, the director of the Medical Education Development Center (EDC), the director of faculty affairs, the director of the Talent Office, and the director of continuing education. The meeting was chaired by the secretary, and all participants had at least 5 years of experience in the educational field of the university. The objectives of the educational ranking of the faculties were presented, and the extracted indicators were introduced. Attendees were asked to provide their suggested indicators and opinions for the next meeting. In the second meeting, held 14 days later, the secretary collected the suggested opinions and reviewed the opinions of the attendees about the existing indicators and proposed indicators. Two additional FGD sessions were held, with 28 education personnel and postgraduate education experts from all faculties. Also participating were education personnel, the office manager of postgraduate education, the office manager of the specialized and sub-specialist education department, the office manager of the admission and registration unit, and the office manager of the postgraduate department affiliated with the university's vice chancellor of education. The meetings were conducted by four members of the scientific EDC board, and all participants had at least 5 years of experience in university education. The meetings were conducted smoothly and efficiently.
Delphi technique
After collecting suggested comments in the FGD sessions, the indicators were designed in the form of a questionnaire. Each indicator was evaluated and scored in terms of importance and feasibility using a 9-point Likert scale. The developed questionnaire was sent via e-mail to 20 educational experts, who were given two weeks to respond with their answers.
Panel of experts
The expert panel, consisting of six educational experts, held eight meetings at the Office of the Vice Chancellor of University Education. The panel decided on the indicators and domains, the weight of each indicator and domain, data collection methods, and the method of ranking the faculties. Each meeting lasted approximately two hours. In the first two meetings, the proposed indicators and opinions collected from the FGD sessions and Delphi technique were reviewed. After selecting the desired indicators, four meetings were held to weigh each of the indicators and domains of educational fields, educational services, and postgraduate education. The weights of each index and domain were calculated by a biostatistician and reviewed in the meetings. The weighting of the indicators was done based on the main tools' weighting while considering the faculties' conditions and expert opinions. The 7th and 8th meetings focused on compiling the evaluation guide for faculties. The panel reviewed and formulated the method of evaluating and ranking faculties, the selection of evaluators from each faculty, and the way of scoring each faculty based on indicators. A scoring guide was prepared for all the criteria, which included all the scoring and weighting points, enabling each faculty to independently calculate its score.
Data analysis
During the Delphi phase, participants were asked to provide feedback on each indicator in the designed model based on the 9-point Likert scale for the two criteria of "importance" and "feasibility." The median index was used for data analysis because it is not affected by outlier data. After obtaining the median of the questions through SPSS.18 software, if the median score of each question was between 1 and 3, the desired index was removed from the model. If the median score was between 4 and 6, the desired question was included in the next stage. If the median score was 7 or higher, the question was accepted in the first round and included in the final model (16, 17).
Results
One of the main results of this study was the development of an educational ranking model for the faculties of medical sciences universities in three educational fields: educational services, postgraduate education, and the educational field. The ranking model evaluates 16 indicators in the field of college education across five areas: education development, governance, education management, quality development of education, and alignment with the comprehensive scientific map of the country. The field of faculty educational services includes 11 checklists across five areas: objective factors, reliability, accountability and responsibility, assurance, and empathy. The field of postgraduate education performance evaluation measures 36 indicators in the field of postgraduate education. Moreover, it was noted that indicators related to the educational field and applicable to all faculties should be compiled (Figure 1).
The 16 forms compiled for the educational ranking model of faculties in medical sciences universities included: newly established programs/degrees as well as existing programs, programs/degrees considered invalid, annual performance report according to the long-term plan, faculty educational council, faculty postgraduate education council, faculty recruitment program, managerial stability, transparency and availability of information on the faculty's website, activities of the Education Development Office, opinion of the University Vice-Chancellor's directors regarding the faculty's performance, Shahid Motahari Festival, special activities of the faculty, professor evaluation project by students, planning and implementation aligned with the goals of the comprehensive scientific map of the country, the faculty's response to society's needs, and professional ethics (Table 1).
Table 1. Domain and weight of indicators in the educational domain of the faculty
| Main domain | Indicators | ||
| Title | Weight | Title | Weight |
| Education development | 15 | Newly established programs/degrees and existing programs | 50 |
| Programs/degrees of unrecognized validity | 50 | ||
| Governance | 20 | Annual performance report in line with long-term plans | 15 |
| Faculty education council | 25 | ||
| Faculty postgraduate studies council | 25 | ||
| Faculty recruitment program | 15 | ||
| Management stability | 20 | ||
| Educational management | 15 | Faculty websites | 50 |
| Quality education development | 20 | Office of Educational Development activities | 35 |
| Comments from university educational deputy managers | 20 | ||
| Shahid Motahhari festival | 15 | ||
| Faculty special activities | 15 | ||
| MAD* project | 15 | ||
| National comprehensive scientific map | Planning and implementation of the major objectives of the country's comprehensive scientific map. 10 | Cooperative agreement | 20 |
| Joint fields of study | 20 | ||
| International students | 20 | ||
| Revenue from education | 20 | ||
| Authoring reference books | 20 | ||
| Responsiveness of the faculty to the needs of society. 10 | Needs-based planning | 25 | |
| Implementation of appropriate programs | 50 | ||
| Direct education and effective activities for community health improvement | 25 | ||
| Professional ethics. 10 |
Faculty empowerment | 50 | |
| Evaluation of professional ethics performance of faculty | 50 | ||
Note: MAD stands for Professor Evaluation Project by Students in Farsi language.
2- Indicators of educational services
This area includes 11 checklists for evaluating faculty educational services, which are: status of human resources in the field of educational services; model of providing educational services; implementation and evaluation of the educational calendar; faculty's educational decision-making councils; physical facilities and support; software and hardware facilities; compliance with educational rules and guidelines; satisfaction and responsiveness; professional competence; credibility and reliability; and interaction with internal and external stakeholders (Table 2).
This area includes 11 checklists for evaluating faculty educational services, which are: status of human resources in the field of educational services; model of providing educational services; implementation and evaluation of the educational calendar; faculty's educational decision-making councils; physical facilities and support; software and hardware facilities; compliance with educational rules and guidelines; satisfaction and responsiveness; professional competence; credibility and reliability; and interaction with internal and external stakeholders (Table 2).
Table 2. Ranking checklists for educational services of medical sciences university colleges
| Items No | Items |
| 1 | Human resources organizational status for educational services |
| 2 | Educational services delivery model |
| 3 | Implementation and evaluation of educational calendar |
| 4 | Educational decision-making councils and committees |
| 5 | Physical facilities and support |
| 6 | Software and hardware facilities |
| 7 | Compliance with educational laws and guidelines |
| 8 | Satisfaction and responsiveness |
| 9 | Professional competence |
| 10 | Credibility and confidence |
| 11 | Interaction with internal and external stakeholders |
3- Indicators of postgraduate education
The field of postgraduate education performance evaluation includes 36 indicators across seven areas: educational program, student evaluation, students, faculty members, educational resources, program evaluation, and senior and executive management (Table 3).
Table 3. Indicators and scores of postgraduate education domains
| Domain | Score per domain | Sub-Domain | Number of indicators | Sub-domain score |
| Curriculum | 5 | Educational structure, content, and duration of curriculum program | 2 | 5 |
| Evaluation of the students | 23 | Evaluation methods | 10 | 23 |
| Students | 17 | Student admission and recruitment policies | 3 | 9 |
| Support and counseling services for students | 2 | 8 | ||
| Faculty members/staff | 3 | Faculty member policy and development | 1 | 3 |
| Educational resources | 22 | Information technology | 2 | 10 |
| Research and technology | 3 | 12 | ||
| Program evaluation | 21.5 | Evaluation mechanisms and program monitoring | 6 | 11.5 |
| Use of student performance | 1 | 2 | ||
| Monitoring educational processes | 3 | 8 | ||
| High-level management & execution | 8.5 | High-level management | 1 | 4 |
| Executive affairs | 2 | 4.5 | ||
| - | 36 | 100 | ||
4- Scoring and weighting of model indicators
Considering the existence of three separate educational fields: educational services and postgraduate education, each with different scoring systems in the university version, a three-ranking system was used for the faculty education section. Eleven separate checklists of 100 points were considered for the faculty educational services section, with equal weighting. The postgraduate education tool had 36 indicators across seven fields, with 100 points allocated for total fields. To summarize the final points of each faculty across all three domains—educational fields, educational services, and postgraduate education—the fields were weighted at 45%, 30%, and 25%, respectively.
The ranking of each faculty is obtained by combining the points of all three domains: educational fields, educational services, and postgraduate education. This is suggested to measure the current status of each faculty across all indicators in the first year of implementing the ranking and provide the necessary training to related experts. In the second year, the improvement of each index should be measured compared to the current status of the faculty in the first year, with the faculty that shows the most improvement receiving the top rank. From the second year onward, the degree of improvement of each index compared to the previous year of the same faculty, and the degree of improvement or maintaining the indicators at the desired level, should be ranked in comparison with other faculties.
It is also possible to introduce the best faculty in any educational field, educational services, or postgraduate studies.
Discussion
The primary objective of this study was to create an educational ranking model for the medical science faculties within Iranian universities. The most significant aspect of this study is its ability to determine the grade and rank of all faculties based on completely objective criteria. The overall rank, as well as the rank in each of the "educational fields," "educational services," and “postgraduate education," can be extracted for each faculty.
To adequately measure the educational processes of colleges, the researchers aimed to develop suitable indicators. The current educational ranking systems, educational services, and postgraduate education were primarily designed for the educational ranking of universities, and many of the indicators could not be used at the faculty level. Therefore, the research team held several meetings to extract indicators that can be calculated in the faculties and also sought the opinions of experts.
According to a study conducted by Safari Farfar (18), the ranking of educational groups in Iran's state universities was based on a set of criteria in various fields. The input field criteria included faculty members, admitted students, structural spaces, facilities and equipment, and management and leadership, which were evaluated using 24 indicators. The process field criteria were evaluated based on teaching and learning, utilization of information and communication technology, and academic progress using 16 indicators. Meanwhile, the output field criteria included the publication of articles and books, the implementation of research projects and patents, winning awards, membership in scientific societies, the duration of study, and the GPA of postgraduates, evaluated using 21 indicators. Finally, the consequences field criteria evaluated the continuation of education for postgraduates, employment and entrepreneurship for postgraduates, and the scientific and professional impact, using 8 indicators in total.
At the international level, the Academic Ranking of World Universities (ARWU), also known as the Shanghai Ranking, evaluates universities based on several criteria. These include the number of articles published in Nature and Science magazines, the number of Nobel Prize winners, and the Fields Prize in the field of mathematics. Meanwhile, the Times Higher Education (THE) ranking system has been in place since 2004 and evaluates universities based on 13 indicators. These include teaching (an indicator of the learning environment), citations (an indicator of the effectiveness of scientific research), research (volume, income, and credit), industrial income and innovation, and international factors such as faculty members and foreign students. The Quacquarelli Symonds (QS) ranking system evaluates universities based on the citation rate of each faculty member to Thomson Scientific and Scopus databases, the ratio of the number of students to professors, the number of foreign students enrolled, and the number of researchers and foreign professors hired. This ranking system was the first international ranking system approved by the Academic Ranking and Excellence (IREG Observatory) in 2013 and has become one of the most reliable evaluation systems. However, it should be noted that the focus of international ranking systems is primarily on research, whereas the present study focuses on the educational aspects of colleges (19–21).
In a previous study, the ranking indicators of medical science groups in 2005 were compiled into 10 groups, including medicine, dentistry, pharmacy, nursing, nutritional sciences, management and information, paramedicine, rehabilitation, and health. The study evaluated faculties based on three areas: the input criteria, including the average score balance of students, the absolute and per capita number of faculty members, and facilities and equipment such as library resources, physical facilities of the faculty, computers, and teaching facilities of hospitals. The process criteria included student management, faculty member management, system management, a supervisor, and an introduction meeting upon arrival. Meanwhile, the output criteria included graduation rate, continuing education, and the publication of books and articles. Although the criteria in this study are much wider, some aspects, such as compliance with educational rules and the percentage of students who graduated on time, are similar to the previous study (11). In 2010 and 2014, the Ministry of Health's Education Vice-Chancellor implemented the educational ranking of medical sciences universities throughout Iran. However, one limitation of this ranking is that it does not include all aspects of education in universities based on designed criteria (12). In the present study, the indicators designed in the educational field were derived from university educational ranking indicators, but they were adjusted to evaluate faculty-level performance. Additionally, in 2019, the Ministry of Health's Education Vice-Chancellor introduced the educational ranking of universities with a new approach, which ranked medical sciences universities based on three areas: effective outputs, functional areas, and the implementation of education transformation and innovation packages (13). The new approach aimed to provide a more comprehensive assessment of medical science universities.
Conclusion
For the first time, a comprehensive educational ranking tool was compiled for the faculties of medical science universities. This tool evaluates all fields of education, including the field of education with 16 indicators, the field of educational services with 11 checklists, and the field of postgraduate education with 36 indicators. For a university to grow and develop while also meeting the expectations of students, faculty members, and employees, it must have a written program for monitoring and evaluating its educational system. The existence of this ranking model will provide an incentive for faculties to engage in positive competition in line with standards, ultimately leading to the improvement of quantitative and qualitative indicators of education and the improvement of the educational rank of the university among other medical sciences universities.
Ethical considerations
All of the questionnaires were anonymous, and all participants in the study were assured that they had the right to withdraw from the study at any time. This article is the result of a research project entitled "Development of the Educational Ranking System of the Faculties of Iran University of Medical Sciences," which was approved by Iran University of Medical Sciences and assigned the ethics code number IR.IUMS.REC1395.30150 by the National Committee of Ethics in Biomedical Research.
Artificial intelligence utilization for article writing
No.
Acknowledgments
We would like to express our gratitude to all of the reviewers, respected colleagues of the Vice-Chancellor of Education, as well as the faculties of Iran University of Medical Sciences who cooperated in the implementation of this study.
Conflict of interest statement
The authors declare that they have no conflict of interest.
Author contributions
H.R. conceived and designed the study, compiled the indicators, and critically reviewed the article before submission. L.J. compiled the indicators, scored the indicators, and critically reviewed the article before submission. R.M. compiled the indicators, and critically reviewed the article before submission. A.M. compiled the indicators, and critically reviewed the article before submission. N.R. designed the study, compiled the indicators, wrote the article and provided final approval before submission. All authors have approved the final version of the article.
Supporting resources
No financial support was received for this research.
Data availability statement
Data and materials could be made available upon reasonable request.
Article Type : Brief Report |
Subject:
Medical Education
Received: 2022/06/20 | Accepted: 2024/03/14 | Published: 2024/09/10
Received: 2022/06/20 | Accepted: 2024/03/14 | Published: 2024/09/10
References
1. Dugerdil A, Sponagel L, Babington-Ashaye A, Flahault A. Rethinking international university ranking systems in the context of academic public health. International Journal of Public Health. 2022;67:1605252. [DOI]
2. See KF, Ng YC, Yu MM. An alternative assessment approach to national higher education system evaluation. Evaluation and Program Planning. 2022;94:102124. [DOI]
3. Kaidesoja T. A theoretical framework for explaining the paradox of university rankings. Social Science Information. 2022;61(1):128-53. [DOI]
4. Osareh F, Afifian F, Nourmohammadi HA. Designing a conceptual and operational model for scientific evaluation and ranking of educational departments of humanities in universities and institutes of higher education in Iran. Scientometrics Research Journal. 2019; 5(1):23-46. [DOI]
5. Dugerdil A, Sponagel L, Babington-Ashaye A, Flahault A. International university ranking systems and their relevance for the medical and health sciences-a scoping review. International Journal of Higher Education. 2022;11(5):102-33. [DOI]
6. Dowsett L. Global university rankings and strategic planning: a case study of Australian institutional performance. Journal of Higher Education Policy & Management. 2020;42(4):478-94. [DOI]
7. Uslu B. A path for ranking success: what does the expanded indicator-set of international university rankings suggest? Higher Education. 2020; 80(5):949-72. [DOI]
8. Vernon MM, Balas EA, Momani S. Are university rankings useful to improve research? A systematic review. PLoS One. 2018;7:13(3):e0193762. [DOI]
9. Bautista-Puig N, Orduña-Malea E, Perez-Esparrells C. Enhancing sustainable development goals or promoting universities? An analysis of the times higher education impact rankings. International Journal of Sustainability in Higher Education. 2022;23(8):211-31. [DOI]
10. Hagh doost A, Ranjbar H, Shoughi Shafagh Aria F, Momtaz manesh N, et al. Ranking of universities of medical sciences based on their educational activities: setting indicators. Teb Tazkiyeh. 2010; 19(1): 65-72. [DOI]
11. Mohammadi A, Mojtahedzadeh R, Karimi A. Ranking of Sciences and Universities of Medical Sciences: Introducing the best patterns in the country,2005. Teb Tazkiyeh. 2006;15(3,4):65-73. [DOI]
12. Hagh doost A, Ranjbar H, Shoughi Shafagh Aria F, Larijani B, Yazdani S. Ranking of educational system in universities of medical sciences & health services in Iran, 2015. Tehran: Educational Deputy of Ministry of Health and Medical Education, 2015. [Article]
13. Ershad SR, Bahaaldini K, Jahani Y, et al. Educational ranking of the Iranian Universities of Medical Sciences 2019-2020:(Rad 3).
14. Ahmadi S. Ranking of educational services. Tehran: Educational Service Center, Educational Deputy of Ministry of Health and Medical Education, 2014.
15. Tabatabaei Z, Darabi Mahbob M. Performance evaluation system of postgraduate level in universities of medical sciences & health services in Iran. Mashhad: Mashhad University of Medical Sciences; 2015.
16. Gharibi F, Tabrizi JS. Development of an accreditation model for health education and promotion programs in the Iranian primary healthcare system: a Delphi study. Health Promotion Perspectives. 2018;8(2):155. [DOI]
17. Gavarskhar F, Matlabi H, Gharibi F. An Iranian model for elderly care in residential long-term care institutions: A Delphi-based study. Middle East Journal of Rehabilitation and Health Studies. 2021;8(3):e112944. [DOI]
18. Safari Farfar R, Abolghasemi M, Ferasatkhah M, Nave Ebrahim A. Designing a conceptual framework for ranking the educational groups in Iranian public universities. Educational Measurement and Evaluation Studies. 2018;8(21):191-225. [DOI]
19. Galleli B, Teles NE, Santos JA, Freitas-Martins MS, Hourneaux Junior F. Sustainability university rankings: a comparative analysis of UI green metric and the times higher education world university rankings. International Journal of Sustainability in Higher Education. 2022;23(2):404-25. [DOI]
20. Veidemane A. Education for sustainable development in higher education rankings: Challenges and opportunities for developing internationally comparable indicators. Sustainability. 2022;14(9):5102. [DOI]
21. Fernandes JO, Singh B. Accreditation and ranking of higher education institutions (HEIs): review, observations and recommendations for the Indian higher education system. The TQM Journal. 2022;34(5):1013-38. [DOI]
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
