After a brief report by Dr. Romrell about the newest developments in the creation of computer-based examinations by the NBME (The NBME is looking for test sites and Dr. Watson has offered the UFCOM for such a site. However, whatever the nature of the examination, USMLE can and should not take the place of authentic evaluation of students idiosyncratic for the institution), the group turned to the issue of our own evaluation system. Much of the meeting was devoted to hearing the students' and recent graduates' views of the impact of the evaluation system on student behavior. There is an excessive emphasis on gain of factual knowledge and on inadequate focus on professional development. This reinforces the competitive instincts that many of our students bring with them from pre med. Students' idealism is challenged by a conflict between studying "to become good physicians" and studying "to get good grades." For example, students perceive that memorizing large volumes of details will get them good grades on MCQ exams, but will not help them become good doctors. Our conclusion is that the evaluation system must have a higher percentage of authentic exams, i.e. exams wherein the subject evaluated has a clear relationship to subsequent professional performance.
The question then is can we design an evaluation system to both measure progress toward acquiring (formative) and ultimately having acquired (summative) these competencies? It seems that the group agreed that we should look at sequential, progressive evaluations that assess the students' progress and final accomplishments. However, in regard to a summative exam, the students dislike the idea of a single two-day event (examination) determining the fate of their professional lives.
In order to design an evaluation system we need to define the end product and intermediary steps of competency development in correspondence with the educational program design. The Curriculum Committee decided about a year ago to base its future program design on a list of competencies to be reached by the students prior to graduation, with the scope of the cognitive, psychomotor and attitudinal domains being defined by 120 most common clinical (patient) presentations.
The discussion also turned to the issue as to what extent one can break down competency into small, clearly identified components and what the role of global assessment should be. Again, the view prevailed that we need both, domain-based and global assessments. The potential value of peer evaluations that are by necessity more global was mentioned. Such a global assessment would have to include aspects of personality, such as, character, relationship to others, helpful behavior.
Those present agreed that at every step the students' development should be comprehensive in the sense that instruction and assessment should address all domains of competency (i.e. cognitive, psychomotor, attitudinal, etc.) of the span of the physician's activities--obviously commensurate with the stage of the students' development. The examination system should reinforce such a strategy.