Multidisciplinary exams in medical studies : Interest of docimologic analysis

##plugins.themes.academic_pro.article.sidebar##

Published Apr 12, 2019
Download
pdf
Vol. 97 No. 1 (2019): Issue Jan 2019

##plugins.themes.academic_pro.article.main##

Naouel Ben Salah
Issam Salouage
Zied Zein El Abidine Hatira
Rim Goucha
Ahmed Meherzi
Kalthoum Kallel

Abstract

Docimology has allowed the development of evaluative processes assuring valid, reliable and objective assessments. It was adopted within the faculty of Medicine of Tunis since  2007.
The aim of this study was to analyze the docimological survey results of hematology-oncology exams, to evaluate the interest of this analysis in the elaboration of exams and the construction of an item bank, and propose some corrections  in order to improve assessment.    
Methods :We have analyzed the hematology-oncology exams of SCMS1 (Second cycle of Medical Studies 1) from educational year 2008-2009 to 2013-2014. The data input was already done with Excel. The test includes 4 disciplines (Hematology, Oncology, Genetics and the Anatomic Pathology). We have calculated docimological parameters allowing global analysis, by discipline and by item.
Results : A total of 3281 papers and 1004 questions were analyzed. The average success rate per year was 91,54% ± 7,12. The highest average success rate was found in hematology (80,51% ± 10,18). The lowest rate was found in the anatomic pathology (51,61% ± 23,76). The average rate of students succeeding the test without having average note in hematology was 5,36%. It was 42,29% in the anatomic pathology. Average difficulty index was 0,57 ± 0,05. Items analysis showed that 38,04% were easy and 19,02% were difficult. Average discrimination index was 0,25 ± 0,02. Discrimination was very good in 20,51% of items and good in 17,13%. Useless and bad discrimination items were about 40,53%. The average of Cronbach Alpha coefficient was 0,84 ± 0,03, showing a good internal-consistency.
Conclusion :This study allowed an objective evaluation of “contributive disciplines“ in multidisciplinary evaluation and showed the interest of integrating questions. Question analysis with teachers would be important to reevaluate and improve these items.  
 

Keywords:

Docimology, Assessment, pedagogy, Medical Sciences

##plugins.themes.academic_pro.article.details##

References

  1. Martin J. Aux origines de la « science des examens » (1920-1940). Histoire de l'éducation. 2002;94:177-99.
  2. Valin E. La valeur des examens : Etude docimologique réalisée au Liban. Paris: UNESCO; 1961.
  3. Leclercq D, Nicaise J, Demeuse M. Docimologie critique: Des difficultés de noter des copies et d'attribuer des notes aux élèves. In : Demeuse M, dir. Introduction aux théories et aux méthodes de la mesure en sciences psychologiques et en sciences de l'éducation. Liège: Les Editions de l'Université de Liège; 2004. p. 273-92.
  4. Gonzalez-Cabezas C, Anderson OS, Wright MC, Fontana M. Association Between Dental Student-Developed Exam Questions and Learning at Higher Cognitive Levels. J Dent Educ. 2015;79(11):1295‑304.
  5. Jozefowicz RF, Koeppen BM, Case S, Galbraith R, Swanson D, Glew RH. The quality of in-house medical school examinations. Acad Med. 2002;77(2):156-61.
  6. Steinert Y, Nasmith L, McLeod PJ, Conochie L. A teaching scholars program to develop leaders in medical education. Acad Med. 2003;78(2):142-9.
  7. Armstrong EG, Doyle J, Bennett NL. Transformative professional development of physicians as educators: assessment of a model. Acad Med. 2003;78(7):702-8.
  8. Laveault D, Grégoire J. Introduction aux théories des tests. 3ème édition.Bruxelles: De Boeck Supérieur; 2014.
  9. George D, Mallery P. (2003). SPSS for Windows step by step: A simple guide and reference11.0 update. 4ème édition. Boston: Allyn & Bacon; 2003.
  10. George S, Haque MS, OyebodeF.Standard setting: comparison of two methods.BMC Med Educ. 2006;6:46.
  11. Caulfield M, Redden Geoffrey, Sondheimer H. Graduation Rates andAttrition Factors for U.S. Medical School Students. Anal Brief AAMC. 2014;14(5):1-2.
  12. Lefebvre O. Note d'information : Réussite et échec en premier cycle [En ligne]. Ministère de l'éducation nationale, de l'enseignement supérieur et de la recherche, 21/11/2013 [Mise à jour le 10/04/2015 ; cité le 20/12/2015] ; [environ 7 écrans]. Disponible à l'URL :http://cache.media.enseignementsup-recherche.gouv.fr/file/2013/44/7/NI_MESR_13_10_283447.pdf
  13. Arlet JB, Ranque B, Seidowski A, Passeron A, Dupeux S, Foïs E, et al. Les étudiants en médecine savent-ils répondre à des questions faciles aux examens ?.Rev Med Interne. 2014;35(2):74‑5.
  14. Chatelain D, Charfi S, Cordonnier C, Leclercq F, Sevestre H. Étudiants en médecine et enseignement de l'anatomie pathologique : résultats d'une enquête amiénoise.ANN PATHOL. 2009;29(3):173‑9.
  15. Compérat EM. Mémoire Enseignement de l'Anatomie et Cytologie Pathologique en Autriche et en France. Service d'Anatomie et Cytologie Pathologique Hôpital La Pitié Salpêtrière Université Pierre et Marie Curie Paris VI [Cité le 15/12/2015] ; [environ 18 écrans]. Disponible à l'URL : http://www.edu.upmc.fr/medecine/pedagogie/memoire/memoire%20Dr%20Compera.pdf
  16. HATIVA N. Teaching for Effective Learning in Higher Education. Dordrecht: Kluwer Academic; 2000.
  17. Albanese M, Case SM. Progress testing: critical analysis and suggested practices. Adv Health SciEduc Theory Pract. 2016;21(1):221-34.
  18. Demeuse M, Henry G. L'analyse classique d'items. In : Demeuse M, dir. Introduction aux théories et aux méthodes de la mesure en sciences psychologiques et en sciences de l'éducation. Liège: Les Editions de l'Université de Liège; 2004. P.173-86.
  19. Peterson RA. Une méta-analyse du coefficient alpha de Cronbach. Recherche et Applications en Marketing. 1995;10(2):75-88.
  20. Rodriguez MC, Maeda Y. Meta-analysis of coefficient alpha. Psychol Methods. 2006;11(3):306-22.
  21. Laveault D. Soixante ans de bons et mauvais usages du alpha de Cronbach.MEE. 2012;35(2):1-7.