The following guidance is produced by the Faculty Board Office and draws on material produced by the Board and experiences of examiners. It is intended as a helpful check list; comments and additions would be welcome. The guidance relates to both Tripos (MVST)and also 2nd MB and / 2nd Vet MB.
The Faculty office acts as Examinations Office for the MVST, and Senior Examiners are welcome to contact Chad Pillinger in the office (tel (7)66905; e-mail: firstname.lastname@example.org) for advice.
The Chairs and Assistant Chairs for MVST in 2016-17 are:
MVST IA: Dr Jennifer Nichols, PDN (Chair), (Assistant Chair TBC)
MVST IB: Dr Christof Schwiening, PDN D(Chair), (Assistant Chair TBC)
|Michaelmas Term||Appointment of Examiners
Senior Examiners duties
Deadline for form and conduct changes to be agreed by FBB
Done by GB on recommendation of FBB (Note 1)
|Early Lent Term||First Meeting of all Examiners||Convened by Chair of Examiners (Note 3)|
|Early-Mid Lent Term||Call for questions
First meeting of Subject Examiners
|Done by Senior Examiners (Note 4)|
|Lent Term/early Easter Vacation||Subject Examiners draft, first version of papers.
Draft sent to External for comments
|See (Notes 4 and 5)|
|Late Easter Vacation||Final drafts of papers, taking into account EE's comments
Papers printed either in Departments or by Reprographics
|Early Easter Term||Examiners arrange attendance and invigilation during exams, and finalise marking arrangements||See (Note 7) - Examination arrangements|
|Exam Period||Subject examiners meet to finalise subject markbook
Final Tripos Examiners meeting
Results submitted to Faculty Office
(Note 7) (b)
|Post Exams||Senior Examiner deals with complaints/appeals
Senior Examiner compiles report
Note 1 - Appointment of Examiners
Examiners are appointed by the General Board on the nomination of the Faculty Board of Biology. In the case of the 2nd MB/2nd Vet MB nominations, Examiners also need to be approved by the Faculty Board of Clinical Medicine/Faculty Board of Veterinary Medicine. A standard nomination form is available from Annie Mackay in the Faculty office (email: email@example.com ). Forms are circulated over summer. The timetable for appointment of examiners is set out in Ordinances:
Chairmen and Assistants: by the end of the Easter term
Seniors, Internal and External Examiners: by 30 September
Assessors: 4 weeks before the examination.
Once an appointment has been made by the General Board, the Board of Examinations write to the examiner concerned who is asked to confirm their acceptance using the card provided.
In 2012, the General Board reviewed policies, procedures and documentation for external examining, and released the following advice;
Care should be taken to avoid nominations where conflicts of interest (either professional or personal) may arise. External examiners should not be appointed if they are covered by any of the following categories:
- near relative of a member of staff or student involved with the programme of study
- an examiner on a cognate course in the institution
- anyone closely associated with the sponsorship of students on the course
- anyone closely associated with placements or training
- anyone required to assess colleagues who are recruited as students to the programme of study
- anyone in a position to influence significantly the future of students on the programme of study
- anyone who has been directly involved as an external member of a validation panel for the programme
Should any concerns arise about possible conflicts of interest, advice or requests for permission under exceptional circumstances should be sought directly from the Education Section.
The General Board's full guidance notes on the appointment of External Examiners can be found here.
Note 2 - Nomination of and Timetable for Appointment of Examiners
There is a standard nomination form available from Annie Mackay in the Faculty office (email: firstname.lastname@example.org). Forms are circulated over the summer. The timetable for appointment of examiners is set out in Ordinances:
Chairmen and Assistants: by the end of the Easter term
Seniors, Internal and External Examiners: by 30 September
Assessors: 4 weeks before the examination.
Preparation: Duties and responsibilities of Senior Examiners
Senior examiners are accountable for both the process and the conduct of the examinations for which they are responsible.
In preparation, Senior Examiners should acquaint themselves with the relevant Ordinances, including the regulations for the particular examination for which they are responsible and should:
- Liaise with Course Organisers and/or the Head of Department to ensure all published information about the exams is up-to-date and correct. If there is to be a change to the form and conduct of the exam, a Form and Conduct Notice needs to be approved by the Faculty Board and published by the end of the Michaelmas term. The Education Section of the General Board has issued guidance on Form and Conduct Notices on their website. (The Faculty publishes current MVST Form and Conduct Notices)
- In consultation with the Course Organiser, ensure that sufficient and appropriate Examiners have been appointed by the Faculty Board, and that if there is to be a change of Senior Examiner at the end of the year, determine which one of the appointed examiners will be the next Senior Examiner. This person should be designated as Deputy Senior Examiner.
- Ensure that an External Examiner has been appointed.
- Produce a specimen paper. If one does not already exist, produce a specimen paper. Specimen papers should closely reflect the examination layout in terms of the type, number and distribution of questions, without giving too many clues as to the actual questions which will appear in the examination.
It is recommended that the following records be maintained by the Senior Examiner to pass on for the next year:
- Names of Senior Examiner, Examiners and External Examiner (with address)
- Copies of any Form and Conduct Notice published in the Reporter for the Examination concerned. Any advice issued by the examiners to candidates in the form of sample questions, information placed on notice boards about what the exam will consist of, advice about when and where to attend, etc.
- Copies of agenda and minutes of subject Examiners' meetings; copies of correspondence, if any, between Examiners and those teaching the course soliciting questions; copies of correspondence with the External Examiner.
- Copy of exam paper and practical exam paper, together with any marking schemes or any other advice issued to examiners.
- Copy of the complete internal subject mark-book (or computer print-out) thereof, in accordance with the appropriate Examinations Data Retention Policy.
- Copy of External Examiner's report (which is formally sent to the Vice-Chancellor, but the External Examiner should be asked to send a copy to the Senior Examiner), which should be passed to the Course Organisers.
- Copies of comments of individual internal examiners on the way that particular questions marked by them were answered - prepared in a way that will be useful as guidance for those teaching that part of the course.
- A statistical summary (how many + percent failed to qualify where relevant, obtained various grades, ideally a cumulative histogram).
Note 3 - Examiners Meetings
There will be two meetings of MVST Senior Examiners (one in late Michaelmas or early Lent) and the Final Meeting in June. All Senior Examiners should attend both meetings, and must be present at the Final Meeting in June to sign the Class List, unless special permission has been granted by the Vice Chancellor for absence.
The Senior Examiner is responsible for organising meetings of Examiners in their own subject. All Examiners, including the External Examiner, for the subject must be present when the marks of candidates in the subject are approved, and should sign a paper copy of the markbook. This also applies for the 2nd MB/2nd Vet MB.
Note 4 - Dealing with Internal /External Examiners
The Senior Examiner should ensure that all internal Tripos examiners receive copies of the Faculty Board guidelines Guidance for marking written Tripos answers . It is important that examiners appreciate that the guidance is to assist them in forming a judgement on the quality of individual essays with the Tripos. For the Medical and Veterinary Sciences Tripos, a candidate's class is more a measure of their relative standing with their peers than an absolute measure of ability. Marks in individual subjects are often moderated to allow them to be combined fairly with others in the Tripos.
The External Examiner should be sent:
(a) Information about the role of an External Examiner in the particular subject concerned, including times and dates of meetings, deadlines for submission of marks etc.
(b) the General Board's Guidance on the arrangements for External Examiners. In the case of External Examiners for the 2nd MB/2nd Vet MB, copies of the Guidance for External Examiners in Second MB/Second Vet MB Subjects issued by MVST I Committee.
(c) Details about the course and examination, (for example, relevant extracts from the Lecture List University Guide to Courses, course syllabus and synopses, bibliographies, lists of prescribed topics texts, departmental booklets)
(d) For Tripos examinations, marking schemes and classing conventions, which you should receive from the Faculty office for MVST.
(e) A copy of the previous External Examiner's final report.
Note 5 - Setting and Preparing the Paper(s)
It is usual practice for the senior examiner in each subject to circulate lecturers with a call for suggestions of questions for the exams. These questions are used by the examiners to assemble the examination papers
The aims and objectives should be clearly stated in the course handbooks and relevant lecture and practical notes, which are available from the course organiser, and the exam assessment should be checked and mapped onto these aims and objectives. External Examiners should normally be given the opportunity to suggest questions and comment on, change/insert/remove questions, and to formally approve the exam paper.
The Senior Examiner will need to provide the Board of Examinations with camera ready copy of the examination paper. The Reprographics Centre at the Old Schools will contact Senior Examiners with the timetable for this. Where appropriate, practical paper questions should be checked with Classroom staff to ensure the correct materials are provided for the exam.
All those concerned with the production of exam questions should be reminded that this should be done under conditions of the strictest security, and computers and printers should be secure from unauthorised access.
All Examiners and Assessors who are also involved in College teaching should be careful in supervisions leading up to the exam period. The Senior Examiners should ensure that all Examiners and Assessors are issued with a copy of the Examiners and Assessors Code of Conduct which has been approved by the Faculty Board.
Arrangements for Examinations which use Multiple Choice Questions (MCQs)
The MVST Part I Committee has agreed that the core syllabus for 2nd MB/Vet MB consists of the lecture material as summarised in the handouts. Examiners are, therefore, advised to set MCQs using course handouts. It should be possible for students to answer any MCQ using either the facts or methods described in the handouts.
The devising, compiling and setting of such questions can be a time consuming process but there is a considerable body of evidence that MCQs, properly devised, are effective at testing understanding of concepts as well as factual knowledge. The problem is that the process of devising good questions does take more time than setting short note or essay-type questions, which underlines the need to start the process of building up a bank of suitable questions in good time. Examiners intending to use MCQs are advised to begin to engage with the business of question setting as soon as possible, and are encouraged to build up a bank of questions that can be adapted and/or revised in future years.
There are several web sites available with further advice on the devising and setting of MCQs - here are three:
How to Write Tests. University of Lethbridge, Faculty of Education
Constructing Written Test Questions for the Basic and Clinical Sciences. US National Board of Medical Examiners
Extended Matching Questions. Association of American Veterinary Medical Colleges
MCQs can and should be machine marked, which means considerable time savings for examiners in the busy exam period.
Machine marking involves the use of standardised "mark sheets", which can be put through an optical mark reader to give automatic entry of the marks into the list. These mark sheets can be used not only for "one right answer" format, but also for "steeplechase", and short answer format questions.
Cambridge Assessment have agreed that they will make available to examiners advice on devising appropriate mark sheets for different layouts and formats of questions, and optical mark reader facilities for machine marking these answer sheets. The "turn round" for this machine marking is about two days. Details of Cambridge Assessment Guidelines on MCQs are available here.
Senior Examiners should consult Jo Ireland at Cambridge Assessment (tel: 553853) about preparation of answer sheets as early as possible, ideally before Christmas, as she will need to schedule production of answer sheets with her other work.
In most cases it will simply be a case of updating last year's mark sheet, but for some examinations there may be a change in format. Mark sheets should be produced in parallel with production of the question paper to ensure that they are compatible. Reprographics will produce the required number of answer sheets and question papers.
Note 6 - Arrangements for printing of exam papers
Senior Examiners should submit a camera ready copy of the examination paper AND answer sheet to the Reprographics Centre at the Old Schools by the end of April (or at least two weeks before Second MB and Second Vet MB exams conducted at the end of Lent Term). Reprographics will produce the required number of answer sheets and question papers.
After the examination, the Senior Examiner should collect the answer sheets from the exam hall (or arrange for them to be delivered to their department). The Senior Examiner should check the answer sheets to make sure all the candidates are accounted for and that candidate numbers have been entered on each sheet. The Senior Examiner should then get the answer sheets delivered to Cambridge Assessment for processing.
Marks will be returned by Cambridge Assessment to the Senior Examiner for incorporation into the markbook.
It would be helpful if Senior Examiners could adopt a standard format for their answer sheets as far as possible. In particular, the header for all examinations should include only the following information
- Tripos (MVST)
- Part (Part IA or IB)
- Subject (for example Biology of Disease)
- Date of the examination
- Paper and section number (for example, Paper I Section I)
- Candidate number
There should be no box for candidates to indicate their gender.
The Faculty Board have agreed that ALL examination papers, including MCQ papers, should be published and they are, therefore, included in the bound volumes of papers produced for College libraries.
Note 7 - Examination Arrangements
Many of the details of exam arrangements are dealt with by the Board of Exams, who will circulate examiners with general guidance. There are, however, some matters which are dealt with by the Faculty Office, acting as the exams office for Biology, and by the examiners themselves. These are outlined in sections (a), (b) and (c) below.
(a) List of Candidates and Markbooks
Lists of candidates, in the form of a markbook, are supplied by the Faculty Office.
Senior Examiners will be given administrative instructions for dealing with marks at the first Examiners' meeting.
(b) Invigilation and Arrangements during Exam Period and Marking Arrangements
Senior Examiners should make arrangements for invigilation of practicals and the starting of theory papers at all venues. Detailed instructions are issued to Senior Examiners by the Board of Examinations in April. As candidates take time to read short answer and MCQ questions, Examiners should be available by phone to answer any questions that arise for the whole duration of any section of the exam containing this type of question.
Senior Examiners will also need to:
- Arrange collection of scripts, and their checking and distribution to examiners, in liaison with the Board of Examinations. Examiners may also make special arrangements to collect scripts completed in College from the Board of Exams. This is highly recommended as there can sometimes be delays getting these scripts delivered to departments.
- Arrange the entry of question marks into the subject markbook. Candidates should record in the cover sheet which questions they have attempted, and examiners are advised to check scripts as they are received and to record the details of any that appear to have material missing, or to have been damaged. This will help in determining at what stage material might have been lost. They should report any apparent loss of material to the Board of Exams as soon as possible, so that searches of the exam room can be arranged. Senior Examiners should report to the Chairman of Examiners on any apparent loss of answers; if material is not found the Chairman will decide whether this could have affected the overall class of the candidate concerned.
- Process marks, and present them to the internal and external examiners; chair discussions on the marks, the distributions of the marks, scaling etc.
- Deliver electronic and signed hard-copy final marks, and Grade Roster for CamSIS data, for both the Second MB/Second Vet MB and MVST IA and IB to the Faculty Office. The Faculty Office will arrange for Second MB lists to be forwarded on the Student Records and to Colleges. After the Final Meeting of Examiners, copies of the mark book for MVST are distributed by the Faculty Office to Colleges so that students can be told their marks. Marks are also posted on CamSIS for students to view.
Second MB/Second Vet MB
Senior Examiners must supply the Faculty Office with lists of all successful and unsuccessful candidates for the Second MB and Second Vet MB. These lists should be hard copy, signed, and provided as soon as possible. Senior Examiners should also provide the electronic (Excel) markbook and Grade Roster of marks for entry onto CamSIS. The total mark should be presented as a percentage in both the markbook and Grade Roster. The Grade Roster includes space to make it clear in what proportion the two sections are combined and for a pass mark to be given, and this information must be included. All required data will be passed by the Faculty Office to Student Records for entry onto CamSIS, and marks will be circulated to Colleges.
Detailed instructions on completion of the Grade Roster will be supplied to all Senior Examiners in advance of the examination period.
The Medical Education Committee have agreed that the DOMVE should act as Chairman of Examiners for 2nd MB/2nd Vet MB examinations, and Senior Examiners will consult him for advice.
(c) Other Guidance
Examiners must not discuss any aspect of the exam with students.
University policy is that any complaints or appeals from students must go through the relevant college authority (usually Tutor or Director of Studies) to the Chair of Examiners, who will pass the matter to the relevant Senior Examiner for action.
Students should be aware that they should not attempt to contact Examiners directly. It has been found that they have, on occasions, been known do do this. Examiners should pass any e-mail or letters received from candidates direct to the Chairman of Examiners, who will issue a standard reply advising students to raise any queries through their Director of Studies.
Guidance to Examiners on Double Marking
Senior Examiners should consult the Faculty Board's guidelines on double marking.
The Faculty Board recommend that Examiners follow the guidance issued by the Education Committee regarding writing on scripts. This advice is based on two principles: objectivity of marking (where double marking is used) and the administrative complications of retaining and disseminating comments on scripts to students if requested to do so under the Data Protection Act 1988. Examiners are recommended to document their marks and comments separately from scripts, but where they think it is sensible and prudent for marks and comments to be made on scripts. Examiners should take due care that this data can be retrieved easily if a request is made. It is the responsibility of the Examiners, or the relevant department, to ensure that there are resources and procedures in place to deal with any such requests as they arise. The return of written answers in exam scripts is exempt; only examiners' comments and marks need be returned.
Note 8 - Post Exam Activity
After the examinations, all Senior Examiners will need to prepare a report for the Faculty Board on the examination, its components, student performances, the marks given etc, and any matters of exam organisation and teaching/learning of the material in the course that need attention, for the Faculty Board and feedback to Teaching Committee and Course Organiser's meetings.
Reports should state prominently the number of candidates, together with the mean and standard deviation of the marks that are reported to the Chairman of Examiners and the percentages of candidates falling within each of the notional subject classes. Senior Examiners are also encouraged to report on rescaling, if any, applied to the marks.
Senior Examiner's reports should be sent to the Secretary of the Faculty Board, copied to the Course Organiser, within one month of the examination, as there is very little time to make changes in a course following an exam. The report should also be sent to the person who will be Senior Examiner in the following year.
External Examiners are asked to report to the Vice Chancellor. Copies of their reports are fed to the Faculty Office and to Heads of Department by the Education Section of the General Board. A response should be made in accordance with Faculty Board Guidelines
The Faculty Board has agreed that these will be published on the web and are available here.
Retention of markbooks, scripts and other paperwork
Senior Examiners are responsible for destroying confidential waste and computer records in accordance with the Examinations Data Retention Policy for MVST. Interim marks, marks for individual questions and other such data should be retained for one month after publication of the class list, in case an appeal is lodged. If an appeal is lodged, the data for that student should be retained until the outcome of the appeal is known. Scripts should be retained for six months and may then be destroyed. The Board of Exams will store scripts and arrange for their disposal. Please pack scripts in envelopes not black sacks.
Under the University's review procedure for examinations, representations regarding the conduct of an examination should be lodged with the Registrary before the class list is published, and within three days after the end of the examination. The representations are referred to the Chairman of Examiners for consideration by the Examiners at their final meeting.
Representations concerning the conduct of an exam arising after the list of successful candidates is signed should be lodged with the Registrary within one month of publication of the class list.
The representations must be considered by the Chairman and at least two other Examiners, and a response made within a month.
If the candidate is dissatisfied with the response from the Chairman, the candidate may apply to the Registrary for the matter to be referred to the Examinations Review Committee. An application will not normally be considered unless it is received within three months of the date on which the Chairman's response was made.
No representations under the procedure will be considered unless they relate to the conduct of the examination. The procedure applies to the Tripos only.