Volume 17, Issue 54 (2024)                   JMED 2024, 17(54): 44-50 | Back to browse issues page

Ethics code: T/IM-NF/Guwahati/21-22/02


XML Print


Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:

Krishnan S, Sundareswaran L, Sinha A, Naveen P, Mahanta A, Bhattacharjee M. Effectiveness of a blended viva format in cardiovascular physiology for first-year MBBS students: An initial report. JMED 2024; 17 (54) :44-50
URL: http://edujournal.zums.ac.ir/article-1-1944-en.html
1- Department of Physiology, AIIMS Guwahati, India , drkrish10@gmail.com
2- Department of Physiology, AIIMS Guwahati, India
Full-Text [PDF 301 kb]   (277 Downloads)     |   Abstract (HTML)  (1546 Views)
Full-Text:   (454 Views)
Abstract
Background & Objective: Oral examinations or viva is a common mode of assessment for medical students. It is not possible to cover the entire syllabus and students often feel that the “luck factor” plays a major role in deciding their performance. Efforts have been made to address these shortcomings associated with traditional viva exams, mainly by attempting to formulate a format which is both objective as well as structured. With this background, aim of the present study was to create an objective structured viva questions in cardiovascular physiology as a prototype of viva voce in physiology.
Materials & Methods: The study included creation of an Objective Structured Viva Examination (OSVE) question bank in cardiovascular physiology. The first year medical (MBBS) students were randomized by use of random number table and attended all three formats of theory viva (OSVE, traditional & blended) on designated dates and time slots. Feedback was obtained from the participants using standard format and the feedback from faculty involved in the study was collected in a qualitative format at the end of the assessment.
Results: There was significant difference in the mean value of marks obtained by the students in the OSVE format when compared to traditional and blended format (p < 0.05). No correlation was observed between marks scored between the three formats of viva. Student’s feedback revealed that OSVE format was well structured and easy to score marks when compared to other formats. Faculty feedback revealed that OSVE format was less time consuming as compared to the other formats.
Conclusion: Since there was no correlation between performance of students in the three formats of viva, each type of viva format can be used as a separate assessment tool for the students.  Further studies of this kind can help to reorganize the viva assessment and also can give a platform for innovation of newer assessment method in medical education which can be extrapolated to other streams.


Introduction
Assessment is an important component of any teaching learning activity, and most learning is assessment driven (1). Level of cognitive learning in undergraduate medical students can be enhanced by a commensurate change in teaching, learning and assessment strategies (2). Hence, it is desirable to improve assessment tools to maximize learning. Oral examinations or viva voce is a common mode of assessment in medical schools (3). In addition to subject matter per-se, viva voce also helps in assessing communication skills, creative thinking and correlative skills among students (4). Experienced examiners are also able to encourage anxious students to answer questions. However, this traditional tool of assessment, is rarely utilized to its full potential and fails to achieve the desired objective of assessing an integration of knowledge and other soft skills during examinations. However, traditional viva voce examinations are very subjective. Students tend to be apprehensive owing to the uncertainty and usually intimidating atmosphere associated with these examinations (5). Moreover, oral examination has the chance of biases such as the "dove/hawk" effect, characterizing some examiners as more lenient or tough than others, the "halo effect", scoring an overall high or low mark based on carryover from a score in one section of the exam (6). Often, topics are left out, depending on personal preference and experience of examiners. Also, it is not possible to cover the entire syllabus and students often feel that the “luck factor” plays a major role in deciding their performance. Efforts have been made to address these demerits associated with traditional viva exams, primarily by attempting to formulate an objectively structured format (4). However, educationists believe that too much structuring leads to loss of the very essence of the traditional viva voce wherein a student has freedom of expression, and the creative thinking skills, presence of mind and communication skills of students can be assessed to a considerable extent (7, 8). Moreover, formation of a topic wise objectively structured viva is a very cumbersome task. The questions need to be framed keeping in mind various domains of learning. Validation of the questions by subject experts is also required (9). Hence, structuring the entire syllabus in any particular discipline is a daunting task. Forming structured viva questions on subtopics may be a pragmatic solution to address this issue. In view of the pros and cons of either traditional or objectively structure viva examinations, it is imperative to strike a balance, and a blend of traditional viva and objectively structured viva may be a rational option. This could then serve as a guide for creation of objectively structured viva examination (OSVE) in the other sub topics in Physiology. With this background, aim of the present study was to create an objective structured viva questions in cardiovascular physiology as a prototype of viva voce in physiology.

Materials & Methods
Design and Setting(s)
It was a developmental study conducted among 1st year MBBS students in an upcoming institute of national importance in Assam.
Participants and Sampling
The sample size was calculated using G-Power computer software. A total sample size of 44 was calculated to detect a medium effect (d = 0.50). The power of the test was fixed at 90% and the alpha at 0.05 (10). The study included the creation of an OSVE question bank in cardiovascular physiology. The questions were validated by three external experts who had experience in the same field for more than fifteen years.
Development of OSVE
A. Creation of an Objective Structured Via (OSVE) Question Bank: The objectively structured viva questions were made in the following sequence:
1. Identification of subtopics in cardiovascular physiology.
2. Segregation of each topic into must-know and desirable-to-know areas after a brainstorming session amongst faculty members of the department.
3. Creation of questions in each topic with levels according to Bloom’s taxonomy
4. Validation of the questions created in step 3 by senior faculty members of the department (internal validation).
5. Validation of questions created in step 3 by external subject experts (external validation).
6. Creation of OSVE cards (with marks allotted to each cognitive level or domain) for examination.
B. Obtaining feedback using a questionnaire from students and faculty
A validated questionnaire, which includes various aspects of feedback about the assessment, was administered to the students and faculty in the form of a Google Form to obtain feedback immediately upon completion of the study. Ethical clearance was obtained from the IEC of the mentor institute.
After obtaining informed consent, all 50 first-year MBBS students were randomized by using a random number table, and they attended all three viva formats (OSVE, traditional, and blended) on designated dates and time slots. An orientation session was arranged for all the participants, explaining the pattern of the study and assessment.
Data collection methods
Station A: OSVE method of assessment. The students chose a card and answered the questions therein. He or she was marked as applicable. A second chance was provided for the students in case they were unable to attempt the question on the first card.
Station B: Traditional method of assessment. The examiner asked the question of his or her choice on the topic concerned and awarded marks as he or she deemed fit.
Station C: Blended method (70% OSVE and 30% traditional). 70% of the viva examination was conducted via objectively structured questions, and 30% of the questions were subjective in nature and examiner-dependent. The students chose a card and answered the questions therein. He or she was marked as applicable. A second chance was provided for the students in case they were unable to attempt the question on the first card. The examiner asked questions of his or her choice to allow freedom of expression and assess the correlating capabilities and creative thinking of the student.
The participants went to all three stations (A, B, and C) on designated dates and time slots. Feedback was obtained from the participants regarding the evaluation of OSVE attributes, the evaluation of the quality of OSVE performance, the evaluation of validity and reliability, and also from the faculty at the end of the assessment. The flow chart below shows the sequence of study methodologies. The brief methodology is depicted in the consort flow diagram (Figure 1) for reference.
Figure 1: Consort flow diagram


Data Analysis
After completion of the study, the data was entered into MS Excel and analyzed using SPSS 26.0 software. The data was tested for normal distribution. The differences between the three groups were tested by a one-way ANOVA. The correlation between the three groups was tested by the Kruskal-Walli’s test.

Results
The total number of subjects recruited for the study was 50. The mean age group of the study population was 18.85 ± 1.12 years. There was a significant difference in the mean scores of marks obtained by the students in the OSVE format when compared to the traditional and blended formats using a one-way ANOVA test (p < 0.05) (Table 1).  
Table 1: Comparison of marks obtained by students between the OSVE, traditional and blended groups
OSVE
(Mean ± SD)
Traditional
(Mean ± SD)
Blended
(Mean ± SD)
16.8±2.4 13.5±1.7$* 14.4±2.6#*
 
Note: Data expressed in Mean ± SD, $ OSVE compared with traditional, # OSVE compared with blended, * p < 0.05 is statistically significant.
Abbreviations:  OSVE, objective structured viva examination; SD, standard deviation

No correlation was observed between the performance (marks scored) in the three formats of viva by the students (Figure 2).
Figure 2: Correlation between marks of OSVE, traditional, and blended groups

The students' feedback revealed that the OSVE format was fair, well-structured, and easy to score. It was reported that 34 (69%) of the students felt that the exam was conducted in a fair manner, and 31 (62%) of the students) felt that a wide knowledge area was covered. It was reported that 8 (16%) of the participants felt that exams were stressful. It was reported that 33 (66%) of the participants felt that the exam process was well structured and sequenced. It was reported that 26 (52%) felt that this type of exam minimized the chance of failing. It was reported that 34 (68%) of the students felt that this type of examination helped them identify their areas of weakness.
With regard to the quality of the exam, it was reported that 39 (77%) of the students felt that the questions reflected topics that were taught before, 42 (84%) felt the contents were authentic, 39 (77%) felt that the scores were standardized, and 34 (68%) reported that the exam provided them the opportunity to learn. It was reported that 39 (77%) felt that OSVE was practical and useful. It was reported that 36 (73%) felt that personality and social issues would not affect the OSVE scores. (Table 2). Faculty feedback revealed that the OSVE format was less time-consuming and, due to its structured nature, may help students build key concepts while at the same time scoring more marks.
Table 2: Feedback from students regarding the OSVE method of assessment
Evaluation of OSVE attributes
Statements Strongly Disagree n (%) Disagree
n (%)
Neutral
n (%)
Agree
n (%)
Strongly Agree
n (%)
  1. Exams were fair
0(0) 1(1) 4(9) 34(69) 10(21)
  1. Wide knowledge area covered
0(0) 1(3) 6(11) 31(62) 12(24)
  1. Exams were very stressful
1(2) 23(46) 13(27) 9(16) 5(9)
  1. Exams were well structured & sequenced
0(0) 1(2) 1(2) 33(66) 15(30)
  1. Exams minimized chance of failing
0(0) 3(5) 13(27) 26(52) 8(16)
  1. Highlighted areas of weaknesses
0(0) 2(5) 9(18) 34(68) 5(9)
  1. Student aware of level of information needed
0(0) 5(9) 5(9) 34(68) 6(14)
Evaluation of the quality of OSVE performance
  1. Fully aware of the nature of exam
1(2) 3(7) 1(2) 36(71) 9(18)
  1. Tasks reflected those taught
0(0) 0(0) 5(11) 39(77) 6(12)
  1. Time at each station was adequate
0(0) 5(9) 7(13) 33(66) 5(11)
  1. Setting and context at each station felt authentic
0(0) 0(0) 1(1) 42(84) 8(15)
  1. Instructions were clear and unambiguous
0(0) 0(0) 0(0) 49(99) 1(1)
  1. Tasks asked to perform were fair
0(0) 0(0) 0(0) 46(92) 4(8)
  1. Sequence of stations logical and appropriate
0(0) 2(4) 10(20) 31(63) 7(13)
Evaluation of OSVE content
  1. OSVE exam scores provide true measure of essential subject knowledge
0(0) 0(0) 1(2) 48(96) 1(2)
  1. OSVE scores are standardized
0(0) 2(5) 1(2) 39(77) 8(16)
  1. OSVE is practical and useful experience
0(0) 0(0) 4(9) 39(77) 7(14)
  1. Personality and social relations will not affect OSVE scores
0(0) 0(0) 13(27) 30(59) 7(14)
 
Note: All data is expressed as a percentage.
Abbreviation: OSVE, objective structured viva examination; n, number of participants

 Discussion
Assessment in medical education is usually in the form of examinations held at regular intervals to test students' level of knowledge, and the students are advanced to the next level based on their performance. There have been numerous attempts to transform the evaluation pattern from subjective to one that is dependable and objective. The oral examination is a crucial technique for evaluation since it enables the examiner to evaluate the student in practically all cognitive domains 4). Educators have attempted to introduce novel assessment techniques to get beyond the drawbacks of the traditional viva examination (11). In this study, the statistical analysis of the students' performance on both forms of viva-voce exams revealed a substantial improvement in the outcome. The mean score of OSVE method was higher than the blended and traditional group (p < 0.05), indicating that OSVE method can be considered as more standard format when compared to traditional viva. Students’ perception revealed that OSVE method was easy, fair and less stressful when compared with the traditional format and also helped students to understand core concepts in a better way.  Faculty feedback revealed that OSVE format was less time consuming as compared to the other formats and that this method will help students to score more marks by understanding the core concepts of physiology (Must know areas).
Our study is the first of its kind wherein a prototype structured viva format was developed in a particular system in physiology. In our study, an attempt was made to find the usefulness and benefits for students and the results were found to be significant. Some studies have been conducted in structured format in practical examination and the results of those studies are in favor of a structured viva format (12-14). A study by Verma et al showed a statistically significant difference in students marks, indicating that structured viva is more reliable & uniform method for assessment of students (15). A study done by Khilnani et al (16) has suggested that for implementing structured oral examination, orientation and training of examiners in assessment strategies is necessary. Compared to a standard oral test, a structured oral examination is more pleasant, thorough in covering the syllabus, and explores the subject more methodically. With this kind of oral examination, students experience less anxiety or panic. As all students have equal chance of being subjected to a particular OSVE card, OSVE is a very fair assessment method (17). A good test should have validity, reliability, objectivity, practicality, relevance and promotion of learning, the ability to differentiate between students, a relaxed environment and positive reactions from students as observed by Paul et al. (18). OSVE seems to fulfill many of these criteria. OSVE allows us to directly observe the student, ask similar questions to all students, control small details to standardize and focus assessment, and be impartial in grading as elucidated by Baig et al (19). In OSVE, the use of a lottery system, questions of the same level of difficulty, standardized grade and equal time for each student help to minimize the demerits of traditional viva (20). Therefore, it is worthwhile to try OSVE in other medical departments and in several medical disciplines.
Only cardiovascular system was included, and the number of participants were less. This will be addressed in subsequent study in future.

Conclusion
OSVE can facilitate the students’ performance in examinations. Since there was no correlation between the performance of students in the three formats of viva, each type of viva format can be used as an assessment method for the students.  However, owing to the optimistic feedback from students and faculty regarding the merits of OSVE, it is worthwhile to encourage OSVE in some topics, if not the entire syllabus. Further studies of this kind can help streamline the viva assessment and also provide a platform for the innovation of newer assessment methods like OSVE in medical education, which can be extrapolated to other streams. Therefore, it is suggested for future studies to administer the cardiovascular physiology OSVE questions in other institutes and obtain feedback regarding the same from faculty members. It is also suggested to create OSVE questions in other sub-topics of physiology.

 
 Ethical considerations
This study was approved by institutional ethics committee of the institute with the approval no T/IM-NF/Guwahati/21-22/02.
Artificial intelligence utilization for article writing
No.
Acknowledgment
Authors would like to acknowledge the students for their active participation. The authors would like to acknowledge the external experts Dr Renu Bhatia, AIIMS Delhi, Dr Renuka Sharma, VMMC, Delhi, Dr Manisha Kar, AIIMS Bhubaneshwar for their inputs during the external validation process.
Conflict of interest statement
Author does not have any financial disclosure or conflict of interest.
Author contributions
All authors contributed to the concept, development of tools and involved in data collection and manuscript writing.
Supporting resources
No grants, equipment were received for this manuscript/ study.
Data availability statement
Data Sheets are available with the authors.

 
Article Type : Orginal Research | Subject: Medical Education
Received: 2023/05/16 | Accepted: 2023/11/28 | Published: 2024/05/10

References
1. Gargi Dangre-Mudey, Sachin Damke, Neelima Tankhiwale, Abhay Mudey. Assessment of perception for objectively structured viva voce amongst undergraduate medical students and teaching faculties in a medical college of central India. International Journal of Research in Medical Sciences. International Journal of Research in Medical Sciences. 2016;l(7):2951-2954. [DOI]
2. Shenwai MR, Patil KB. Introduction of structured oral examination as a novel assessment tool to first year medical students in physiology. Journal of Clinical and Diagnostic Research. 2013;7(11);2544-7. [DOI]
3. Tabish SA. Assessment methods in medical education. International Journal of Health Sciences. 2008;2(2):3–7. [DOI]
4. Bloom BS, Engelhart MD, Furst EJ, Hill WH, Krathwohl DR. Taxonomy of educational objectives, handbook I: The Cognitive Domain. New York: David McKay Co Inc. 1956;1:29-34.
5. Shaikh S. Objective structured viva examination versus traditional viva examination in evaluation of medical students. Indian Journal of Clinical Anatomy and Physiology. 2015;5: 175-179. [DOI]
6. Muzzin LJ, Hart L. Oral examinations. In: Neufeld, Victor R, Norman GR (eds), Assessing Clinical Competence. Springer, New York. 71-93. 1985
7. Gor SK, Budh D, Athanikar BM. Comparison of conventional viva examination with objective structured viva in second year pathology students. International Journal of Medical Sciences and Public Health. 2014; 3(5): 537-9. [DOI]
8. Kaur N., Manzoor S., Kaur S., Goyal A. Comparative evaluation of structured viva-voce examination and conventional viva-voce examination in II MBBS students. International Journal of Medical Sciences Education. 2019;6(4):1-5.
9. Shah HK, Vaz FS, Motghare DD. Structured oral examination: from subjectivity to objectivity - an experience in community medicine. Journal of Educational Research & Medical Teaching. 2013;1(1):25-7.
10. Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods. 2007;39:175-91. [DOI]
11. Mckimm J. Current trends in undergraduate medical education: teaching learning and assessment. Samoa Medical Journal 2010;2:38-44. [DOI]
12. Puppalwar PV, Rawekar A, Chalak A, Dhok A, Khapre M Introduction of objectively structured viva-voce in formative assessment of medical and dental undergraduates in biochemistry. Journal of Research in Medical Education & Ethics. 2014: 4, (3) 321-5. [DOI]
13. Rahman H, Rahman MA. A Review on structured oral examination (SOE). Dinajpur Medical College Journal. 2016; 9(1): 113-7.
14. Vankudre AJ, Almale BD, Patil MS, Patil AM. Structured oral examination as an
15. assessment tool for third year indian MBBS undergraduates in community medicine. MVP Journal of Medical Sciences. 2016; 3(1), 33-6. [DOI]
16. Verma A, Mahajan N, Jasani K, Patel J. Evaluation & comparison of result: conventional viva Vs. structured viva. Global Research Analysis. 2013: (5); 188-9.
17. Khilnani AK, Charan J, Thaddanee R, Pathak, RR, Makwana S, Khilnani G. Structured oral examination in pharmacology for undergraduate medical students: Factors influencing its implementation. Indian Journal of Pharmacology. 2015; 47(5): 546–50. [DOI]
18. Singel T, Shah C, Dixit D. Small group structured oral examination: An innovation in oral examination. National Journal of Integrated Research in Medicine. 2014;5(1);141-6.
19. Paul V. Assessment of clinical competence of undergraduate medical students. Indian Journal of Pediatrics.1994;61:145-151. [DOI]
20. Baig LA, Violato C, Crutcher RA. Assessing clinical communication skills in physicians: Are the skills context specific or generalizable? BMC Medical Education. 2009;9:22-25. [DOI]
21. Hashim R, Ayyub A, Fatima-tuz-Zuhra, Hameed S, Ali S. Structured viva as an assessment tool: perceptions of undergraduate medical students. Pakistan Armed Forces Medical Journal. 2015;65(1);141-4.

Add your comments about this article : Your username or Email:
CAPTCHA

Send email to the article author


Rights and permissions
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.