Evaluation Subcommittee Meeting -February 12, 1997

Present:

Alguire, Duerson, Duff, Lucas, Okun, Romrell, Rooks, Stevens, Small, Suter, VanSusteren, Watson, Webb

1. Parker Small gave a brief overview of where, in his opinion, we are and where he proposed to be going. In short, after completing the review of concept maps and matrices task forces will be formed to carry out specific assignments for creating the individual components of the subcommittee's report. Further, copies of previous evaluation committee reports (Harris Committee and Duerson Committee) would be made available. It was again emphasized that we are accumulating large numbers of evaluation data which are widely distributed. What is missing, however, are systematic and planned procedures for data analysis, utilization and followup.

2. In reviewing the concept maps and matrices it was noted that criteria and data sources mentioned either need to be matched/coordinated. This should be done by the task forces.

Course/Clerkship Concept Map should indicate that all criteria must be addressed by evaluations (for change see attachment). There is a separate matrix for courses and clerkships respectively. The annotations on the clerkship matrix prepared by Pat Alguire were found particularly helpful. Obviously, there are some overlaps of course/clerkship evaluations and faculty evaluations, as the quality of instruction by the faculty is a major variable influencing the quality of courses and clerkships. Nevertheless, separate evaluations data are needed and obtainable. For faculty evaluation, emphasis is on individual performance, while for course/clerkship evaluation the performance of the faculty as a team is more important.

An interesting discussion followed a comment by Larry Rooks expressing his concerns about who or what organization would be responsible for the evaluation system as a whole, the Curriculum Committee or who? Bob Watson explained that it was COMEC's task to implement evaluations as directed by the Curriculum Committee. Still to be agreed on is who (individual or group) will critically analyze the data, identify deficiencies wherever they are and report them to the Curriculum Committee and the involved individuals (faculty, course/clerkship directors) for corrective action. Considering the amount of data generated, procedures for their analysis and interpretation must be developed and responsibilities assigned.

Regarding the Curriculum Evaluation concept map it was noted that it should indicate that faculty, students and graduates are assessed for satisfaction with the curriculum, as well as for the quality of their respective performance. As far as graduates are concerned, ultimately we should consider an assessment of their performance as practicing physicians after completing their residencies. The difficulty of that task was acknowledged (see attached revised concept map).

3. Task Force Assignments

The following assignments were made:

Student evaluations:

• Margaret Duerson with Jeaninne Webb, Michael Okun and Tim Lucas

Faculty evaluations:

• Patrick Duff with Jeaninne Webb and ?

Course evaluations:

• Lynn Romrell/Parker Small, Ed Meyer, and Ben Stevens

Clerkship evaluations:

• Patrick Alguire and John Meuleman

Curriculum evaluation:

• Manny Suter, Dick Reynolds and Larry Rooks

The goal is to generate specific recommendations including required resources and implementation plans. The charge to the task forces is as follows:

1. Review concept map and matrix, revise as necessary with particular attention to the criteria. Annotations similar to those created by Pat Alguire on the clerkship meeting would be particularly helpful.
2. Fill in the matrix cells indicating "effectiveness/cost" of a particular data source on a 0-4 scale. For example on the student matrix the cell for "triple jump" and "identify and learn new knowledge" might be 4/3. "4" means the triple jump is an outstanding evaluation method for this criterion and "3" means it requires a great deal of faculty effort.
3. Review existing evaluation instruments for the identified criteria as to their appropriateness and effectiveness.
4. Identify, suggest, or create additional instruments needed to assess adequately the criteria not covered in 3 above.
5. The chairs of the task forces will meet at 7:30 am on February 26 in CG-78 to coordinate the work and establish a time line.

The following items are attached:

• Educational Program Evaluation System concept map
• Student evaluation concept map and matrix
• Faculty evaluation concept map and matrix
• Course/clerkship concept maps, and course and clerkship matrices
• Curriculum evaluation concept map and matrix
• Ocie Harris Evaluation Report of 1987
• Margaret Duerson Evaluation Report of October 7, 1994

Have fun!

The next meeting of the subcommittee will be announced later.