The meeting was devoted to a discussion of the reports by the Task Forces on Student Evaluation (Margaret Duerson, chair) and Course and Clerkship Evaluation (Parker Small and Lynn Romrell, co-chairs).
Margaret distributed an addendum/amendment to her Task Force's report (a copy is enclosed for those who were not able to attend the meeting) which was discussed. The document addresses the evaluation of three competency domains: clinical skills, professional behavior, and knowledge.
During the Basic Clinical Skills courses I-IV (year's 1 and 2) and the clerkships (year 3) the students are expected to acquire clinical skills and expand them in depth and breadth over the three year period. Successful development of their skills is assessed by performance-based examinations originally called OSCE I and II given after the first and second years, respectively, and clinical skills examination given upon conclusion of all third year clerkships. A separate clinical skills examination is administered at the end of the interdisciplinary clerkship.
The discussion addressed the following: (1) assessments using standardized patients or written scenarios should be called performance-based examinations (PBEs) I and II for the first and second year; III for PBE at completion of individual clerkships, and PBE IV at the beginning of the fourth year. (2) Since the basic clinical skills course IV (clinical diagnosis) has a large lecture component, it was suggested that the PBE II be expanded to test the students' ability to apply this content to their patient assessment procedures; (3) to achieve internal consistency of teaching and evaluating clinical skills, course and clerkship objectives should be reviewed and, if necessary, revised; (4) a special effort is necessary to involve basic science faculty in the design of interstation examination to enhance assessment of the students' integration of basic science knowledge in the clinical reasoning process; (5) PBE I-IV should be of progressive complexity, reflecting the expected advancement of the students' clinical skills.
Further important issues discussed were: (1) relationship of evaluation to grading--a pass/fail policy was briefly discussed for the first two years; (2) a renewed effort must be made to enhance the observation of student performance by faculty and housestaff. Probably a new, universal rating form should be developed; (3) the feedback loop must be closed. In concert with faculty, this should be the role of the Associate Dean for Student Affairs and possibly the Academic Status Committee.
An interesting discussion ensued about the difference between the novice's and clinical expert's mode of clinical thinking. At what stage of development does the student develop pattern recognition and how can it be assessed? The student learns medicine "bottom up" and will eventually practice it "top down." Professional behavior needs to be defined by the Curriculum Committee and then appropriate assessment methodology can be developed. A publication entitled "Professional Behavior" from the American Board of Internal Medicine can serve as a background document.
Knowledge assessment needs refinement in several ways: (1) course/clerkships that use NBME shelf examinations must ensure that questions correspond with the course's or clerkship's objectives; (2) basic science recall in the clinical setting should be assessed in PBE interstation examinations.
The discussion of this report was somewhat abbreviated as some members had to leave. The following points were made: (1) the debriefings prove to be constructive and effective. However, Margaret Duerson questioned the validity of exclusive student assessments of courses and clerkships. Are they becoming popularity contests (she made the same objection in regard to faculty evaluation). Jeaninne Webb argued that student evaluations are valid provided they are complemented by student performance data; (2) courses and clerkships must spell out their objectives so that achieving those objectives can be assessed; (3) the Curriculum Committee should evaluate the course and clerkship directors' performance with appropriate feedback.
Parker Small will begin the development of the final report. As a first step, it was suggested that he prepare a listing of recommendations for discussion by the subcommittee members. In particular, emphasis must be on motivation and resources for implementation of any recommendation in the final report. Establishment of priorities is essential.