Attendees: Course and Clerkship Directors, medical students
Dr. Rarey gave a review of the April 12 retreat. He indicated that the work was ongoing and dealt with Objective 7 which reads: "A uniform competency-based evaluation system must be established" as specified by the Curriculum Committee. Dr. Rarey then reviewed the agenda for the meeting.
Presentation by Dr. Kellner
Dr. Kellner asked that everyone give a little to get a little and indicated that we need to come up with an evaluation system that will work. The evaluation system, simply stated, must improve learning. In his speech to the group, he stated (to students) that what you on a course or clerkship ends up on your grade. He also said that, as students, you need to be productive, keeping in mind that we have an intrinsic conundrum; behavior is a continuum, and evaluation is categorical.
The components for a good evaluation system are:
Here is how the flow works:
What form might a form take that would fill that component?
The behaviors which fit their performance:
We have been getting feedback and making changes to the form. Delete the ones you don't use. This form can be very individualized between the evaluator and course and clerkship director. See how this relates to the other form.
Formative Feedback
Giving feedback is the best thing an evaluator has to do.
Erin Dunbar came up with a student-initiated formative feedback. Students would have these forms during the course or clerkship that they would fill out. This form appeals very much to faculty. The realities are the thing we fear that if we come in with feedback lower, everyone gets very defensive and starts backpedaling. This way we are starting out with your (student's) self-assessment. This looks like something we are proposing and faculty have responded quite positively to it.
Summative feedback
At the end, the scoring would be done.
How might this work in reality? The form will describe behavior which is for example, no good, ok, better, best (as in the overheads presented), which for calculations might equal: 0, 1,2,3. How does this translate into a final grade is where we run into a problem. An "A" grade might become: F or D; C or C+, B, B+, A. Think in terms of the concept, the idea of a form, a behavioral descriptor. Would it be useful or fair. (Open discussion).
Student: I like that performance is a continuum but grade has to be categorized. I'd like to see 8 categories. More categories might help.
Dr. Stair: We are trying to get away from slotting the student into a grade. It's frustrating to evaluate people. The math system you had there made sense to me.
Dr. Small: Instead of having 4-6 categories you would want the faculty to draw a line on that continuum.
Student: The more you have the more you can differentiate between them.
Question: Is Medicine liked because of the four-point descriptor or because of the moderator involved in the session?
Dr. Rarey: Medicine had a facilitator and students felt there was some objectivity.
Dr. Nadeau: In perspective, Neurology had a five-point scale without descriptors. Going by COM targets and 25% of people got As. I had to set the boundary at 4.75. Grade inflation of that magnitude does no one any favors. Medicine format is the most effective format.
Another student agreed with having more categories and thought that four categories would be an acceptable range and going in the right direction.
Dr. Rarey: How would you determine self-evaluation?
Student: Having these descriptors is really good.
Dr. Nadeau: A four--point scale is adequate to define what you all do, but it will differ from clerkship to clerkship.
June/2000