Exhibit 6
In 1997, the Evaluation Subcommittee was charged by the Curriculum Committee to design and implement a comprehensive evaluation system that included:
Accordingly, the evaluation subcommittee has crafted four respective evaluation formats to be implemented July 1, 1999. The formats are to evaluate:
The formats are based upon the concepts presented in the document, "University of Florida College of Medicine Curriculum Renewal Plan, B. An Evaluation System: Evaluation of Student Achievement, Faculty Instruction, Courses and Clerkships" approved by the Curriculum Committee in June 1997.
The goals of addressing student evaluation are threefold:
These goals can be achieved successfully by accomplishing the following seven objectives. Such objectives are not totally new; some objectives were proposed in reports of past task forces of the curriculum committee. **
**Subcommittee on evaluation/competencies report, M. Duerson, chair, 9/7/94
Final report of the committee for review of student evaluation, O. Harris, chair, 4/87
Report of Task Force on Student Evaluation, M. Duerson, chair, 3/25/97
Revised Report of task force on student evaluation, M. Duerson, chair, 4/9/97
Subcommittee on competencies and evaluation, M. Duerson, chair, 10/1/93
1. Measurable, learning course/clerkship objectives must be given to students at the outset of each course/clerkship, and must relate to graduation competencies. The objectives will serve as explicit criteria to measure mastery of student's knowledge, behaviors and skills.
2. Faculty development will be provided selectively to assist faculty in writing measurable learning objectives and to encourage faculty to use innovative criterion-referenced methods of assessment.
1. Examinations must be based upon measurable learning objectives.
2. The use of performance-based formats, other than multiple choice questions (MCQs) must be increased. Use of performance-based evaluations is strongly encouraged in all courses, but is required in all clerkships.
3. Faculty development will be provided selectively to faculty to use innovative methods in assessment.
1. Written competency-based formative feedback must be given to a student by course faculty at least once during the course or clerkship. Ideally this should be accomplished mid-way or earlier during accrues/clerkship.
2. The inclusion of student self-assessment is strongly encouraged during the formative feedback process.
3. Faculty development will be provided selectively to enhance the faculty's ability to provide effective feedback.
1. There will be two summative feedback systems utilized for evaluation of medical students:
|
PBE 1
|
PBE 2
|
PBE 3A
|
PBE 3B
|
| Diagnostic Skills | Diagnostic Skills | Diagnostic Skills | Diagnostic Skills |
| History | History | History | History |
| Communication | Communication | Communication | Communication |
| Professionalism | Professionalism | Professionalism | Professionalism |
| Problem-solving |
2. Faculty development will be provided selectively to faculty to use a variety of summative evaluation techniques.
Evaluation of students' professional behavior and human relationship competencies must be conducted during all clerkships, and is strongly encouraged for basic science courses.
Mission-based budgeting seeks to allocate educational funds equitably, support desired educational outcomes, promote innovation, and reward outstanding teachers, courses and clerkships. To accomplish this, instruction must be evaluated by mission outcome, as well as by activity. Courses and clerkships are evaluated quantitatively through the Faculty Effort Reports you submit. The qualitative component is by evaluation of your annual report, composed of the syllabus and your written report, and numerical data from the College of Medicine education Center (COMEC). Course and clerkship evaluation is performed by the Evaluation Subcommittee of the Curriculum Committee chaired by Kyle E. Rarey, Ph.D. Recommendations of the committee are forwarded to the Curriculum Committee and to the Senior Associate Dean of Educational Affairs. Recommendations for providing resources to courses and clerkships are then forwarded to them by COMEC.
I. The course or clerkship syllabus given to the students
II. The written report prepared by the director with accompanying appendixes
III. Data supplied by the Office of Medical Education (OME)
In order to facilitate evaluation, course/clerkship directors should be sure to include all material requested in the exact form and organization requested. Each section of the syllabus and report should be clearly delineated and identified to correspond with these instructions.
Each section of the syllabus and report are evaluated longitudinally by members of the subcommittee. Four parameters are used to assess quality of a course or clerkship syllabus and four parameters to assess the written report. A reviewer will assess the same section in all reports, i.e. like an NIH grant evaluation. Therefore, each section should “stand alone.” For example: if a deficiency is identified in the section on student evaluation of instruction and a strategy for improvement implemented, the improvement should appear in that section in addition to the separate section on plans for improvement. Do not refer a reviewer to another section.
An example of an exemplary submission (course
report/syllabus,
clerkship report/syllabus) and the evaluation
form used by the subcommittee is provided. It is strongly recommended that
these documents be reviewed during preparation of your materials.
a. Department chair with contact information
b. Course/clerkship director with contact information
c. Course/clerkship administrative assistant/secretary with contact information
a. Length, hours, units of study as appropriate
b. Sites for instruction
c. Schedule for a typical week
a. Course/clerkship
b. Department - if there is one
1. The competency categories to be addressed
2. The measurable learning objectives for each competency category
3. The learning/instructional activity used to achieve each learning objective
4. The evaluation methods used to assess student achievement of each learning objective.
1. Identify the internal and external measures used to assess student's knowledge and skills. Representative forms should be included.
2. Describe the system for giving formative feedback. Representative forms should included, but alone are not sufficient.
3. Describe the system for determining summative feedback. Representative forms should be included.
1. Describe how students performed relative to the course/clerkship learning objectives
|
Year
|
A+
|
A
|
B+
|
B
|
C+
|
C
|
D or F
|
| current |
1. Describe student performance on the NBME Subject Examination, if given, including comparison with national norms
2. Submit representative samples of assessment instruments/forms in the appendix of this report
1. Provide a synopsis/summary of student on-line evaluations of the course/clerkship. This should include analysis and comment by the Director of strengths and areas needing improvement. Numerical data alone is not sufficient.
2. Provide a synopsis/summary of the student debriefings. This should include analysis and comment by the director of student concerns. Minutes alone are not sufficient.
3. Provide a synopsis/summary of student evaluation of the Director, and administrative support, if applicable.
1. Provide a synopsis/summary of student evaluation (on-line and debriefings) of the overall instruction by faculty, residents and staff, as applicable.
2. Critique the overall teaching performance of faculty, residents, and staff from the Director's perspective. Identify and comment on strengths and areas needing improvement.
3. Describe the program or plans for faculty development.
1. Discuss the timing and appropriateness of the course/clerkship to the curriculum's general professional education program including content and instructional methods.
2. Discuss the strengths and weaknesses of the course/clerkship and describe plans for improvement. Include a description of present and/or future innovative programs.
3. Discuss the long-term plans for continued development and new directions for the course/clerkship.
4. Describe the resources needed to improve the course/clerkship.
Course & Clerkship section updated 1/29/031. NBME subject examinations
2. USMLE Steps 2 and Steps 2, if applicable.
Evaluation of faculty instruction is based upon following premises:
a. Effective teaching skills can be defined. Characteristics of good teaching that can be evaluated are reasonably consistent in the literature. These include:
1) being well prepared for class
2) motivating students
3) effective communication skills
4) demonstrating comprehensive knowledge
5) treating students with respect
6) organized presentationb. Sources to assess teaching effectiveness can be identified. The most commonly identified evaluators of teaching effectiveness are students, peers, self, and college administration, with each group having its unique role. Faculty self-evaluation, when used with student and peer evaluation, is helpful in faculty growth and development. This self assessment must be consistent with assignments and the goals and objectives for faculty development agreed upon with the departmental chair.
c . Some, but not all, faculty evaluation systems change behavior. Faculty development and evaluation are related processes. The former aspires to improve faculty performance and the latter aims to make judgment regarding worth. The processes can be integrated,and can be a powerful technique in changing behavior. On the other hand, there is little evidence that faculty evaluation alone improves instruction. As with any feedback system, faculty evaluation conducted early in the course of instruction favored instructional improvement because it allows faculty members the time and opportunity to make modifications.
Over the past five years, there have evolved standard methods and forms for student and peer evaluation of the faculty through College- and University-Wide initiatives. All teaching units should conform with the standards and use the required forms. Compliance will allow course directors to perform their responsibilities more efficiently, department chairs to have more meaningful assessments of faculty achievement, and the administration to more accurately assess the effectiveness of the overall teaching programs.
For the Curriculum Committee to assess the quality of instruction by the teaching faculty within the courses/clerkships, course directors are responsible that the following objective is addressed.
Standard forms for evaluating teaching faculty by students (Exhibits 2-5) should be used for student assessment (they are available in hard copy and online in Internet-based versions). The forms are available in four formats: Basic Science Teaching, Clinical Teaching, Graduate Medical Education, and a short form for Lectures/Discussion Groups. The latter form should be used only when a faculty member has a minor role in teaching activities. These data will be included in the annual course report.
The primary mission of the COM is to educate medical students in humanistic, scientific and practical principles of medicine in keeping with the emerging needs of society. Therefore, an effective, dynamic curriculum that serves its students and supports its teaching faculty must be continually reviewed to determine its quality and how students perform within it.
The following are the general goals of the curriculum evaluation system:
1. To ensure that the curriculum is "appropriate" for the education and development of competent physicians.2. To assess whether the measurement tools are "appropriate" to gather data about the effectiveness of the programs3. To update deans, teaching faculty and students as to the effectiveness of the educational program
Curriculum planning at the UF COM is guided by a set of principles, which were adopted by the Curriculum and Executive Committees. These 15 principles describe which characteristics that the overall curriculum design should possess. The principles, therefore, should be the yardsticks to measure the success of the overall program. A specific evaluation plan is designed to assess the degree of accomplishment of each principle.
These principles are listed below:
The curriculum is intended to provide a set of knowledge, skills and behaviors that encompass a broad overview of the practice of medicine and is relevant to becoming a competent caring physician regardless of subsequent specialty choice.
Evaluation Issue:
• Is there "appropriate" content in the curriculum for a general professional education?• Determine whether the general professional education program is having the desired impact on students
Evaluation plan:
• Solicit opinions of graduates as to how well the curriculum prepared them for the general practice of medicine• Review periodically the college's-adopted list of competencies to ensure a general professional education of each student• Annual survey of UFCOM students performance in residency programs
The general professional education is defined by a set of competencies expected to be "mastered" by the students.
Evaluation Issues:
• Does the curriculum content and structure, and the student evaluation system, adequately include the use of competencies?
Evaluation Plan:
• Determine whether all competencies are addressed in the curriculum (i.e., that at least one course teaches it and evaluates students to be sure they are competent in the area).• Review competency categories to determine where each objective fits into the curriculum .• Course and clerkship reports should be analyzed for the extent to which education and evaluation is competency based.• Review descriptors to be sure appropriate descriptors are being used for competencies.• Assess competency list every 3-5 years to update/modify it.
The curriculum committee has adopted the clinical presentation model as the primary mechanism for integrating basic and clinical science content
Evaluation plan:
• An annual narrative report should be provided by the evaluation subcommittee to describe progress in implementing the clinical presentation model.• Course and clerkship reports should be analyzed for the extent to which they use the clinical presentation model.
Evaluation Issue:
Does the curriculum require students to learn independently (and if so, specify where)?
Evaluation plan:
• Annually evaluate student's perception of their academic environment as a promoter of independent learning. Specific items in the annual class survey should assess student independent learning.• Course and clerkship reports should include information about activities to promote independent learning.
Evaluation plan:
• Utilize student self and peer evaluation.
Evaluation plan:
• Assess how and where in the curriculum this topic is being formally addressed.
Evaluation plan:
• Decide upon and monitor the amount of inpatient versus outpatient clinical training site.• Decide upon and monitor the amount of primary care versus specialty care training.
Evaluation plan:
• Provide a narrative description of the progress made toward implementing the Informatics plan adopted by the curriculum committee.• Evaluate students use of informatics.• Determine what course and clerkships utilize formal informatics activities.• Urban vs rural; underserved vs others
• Is the competency system having the desired outcome in terms of student performance?• Have (for each clinical rotation/basic science course) an additional test, besides the shelf exams, that includes questions designed to see if students are competent in OUR competency areas, e.g., domestic violence, etc.• Using the data from the competency tracking system, each class should be annually assessed to ensure that the students as a group are making adequate progress in each of the competency categories.• Using data from our Performance Based Exam Series, each class should be annually assessed to ensure that the students as a group are making adequate progress in the competency categories assessed by those exams.
Evaluation plan:
• Assess the level of student educational scholarship, e.g., medical student research and publications.
Evaluation plan:
• Determine the quality of activities which span colleges, departments, specialties, and provider types.
Evaluation plan:
• Insert items, addressing these issues, in the annual class surveys.• Institute a critical incident report program
A set of graduation competencies (see attachment 1) that define the general professional education has been adopted. Course objectives and course student evaluation systems must now be aligned with competency attainment. This process will allow us to be in compliance with LCME standards and prepare us for our upcoming LCME site visit in the year 2000.
The following steps describe the process we must complete in order to implement this system in July of 1999.
Competency: A knowledge, skill or behavior which must be demonstrated by medical students in order to achieve the primary goal of the medical education program-the general medical professional education.
Competency Category: A set of competencies grouped around a common theme.
Learning Objective: A competency or competency category utilized in a course as a statement of educational intent. For practical purposes learning objective and a competency are the same. A competency becomes a learning objective when it is utilized to express educational intent in a course.
1. Each course director will select specific competency categories, which their course is capable of teaching and assessing
2. Learning objectives that are contained in the course syllabus will written in the following format:a. Learning objectives
b. Learning activity associated with objective
c. Method of assessment (see page or example)3. All clinical clerkships must include learning objectives based on professional behavior competencies
4. All courses must include learning objectives based on core discipline competencies.
Clinical clerkship core discipline learning objectives contain two components:
A. Competency derived objectives unique to the discipline.
Basic science course core discipline learning objectives should include two components:
A.Competency based learning objectives related to clinical presentations
B.Competency based learning objectives unique to the discipline.
1. Student evaluation will be based on competency attainment in the categories or specific competencies selected by the course director.
2. Each competency or category will be weighted in order to determine the student's final grade in the course (see example pages ).
3. A standard competency assessment scale will be utilized in all courses (see attachment).
4. COMEC will prepare a customized student evaluation sheet based on the course selections. These forms will be used for both formative and summative feedback.
6. Competencies will be summed across all evaluators and all courses, thereby, allowing the generation of a student competency development profile.
7. Additional competencies may be added if desired, but not additional competency categories.
8. Equivalent competencies may be substituted for a graduation competency. In pediatrics for instance, the Diagnostic Competency, "Demonstrates ability to perform a comprehensive physical examination" might be replaced by "Demonstrates ability to perform a comprehensive examination of the newborn."