NST Senior Examiner Guidelines

The following guidance has been issued by the Faculty Board Office and draws on material produced by the Board and experiences of examiners. It is intended as a helpful check list; comments and additions would be welcome. The guidance relates to examinations in the Natural Sciences Tripos that fall under the jurisdiction of the Faculty Board of Biology.

The Secretary of the NST Management Committee, Jane Clare, acts as the Examinations Officer for the NST and Senior Examiners are welcome to contact her for advice (tel (3)35090; e-mail: jc478@cam.ac.uk) or, Chad Pillinger (tel (7)66905; e-mail cp316@cam.ac.uk) about Faculty Board of Biology examinations policies

The Chairmen and Assistants for NST in 2018-19 are:

• NST IA Chair: Dr Dee Scadden
• NST IA Assistant Chair: Dr Stewart Sage
• NST IB, II and III Chair: Dr Kate Plaisted-Grant
• NST IB, II and III Assistant Chair: Prof Jim Haseloff

Appointment of Examiners

Examiners are appointed by the General Board on the nomination of the Faculty Board of Biology. Once an appointment has been made by the General Board, the Student Registry write to the examiner concerned who is asked to confirm their acceptance.

The Faculty of Biology uses a Moodle site to collate information relating to the appointment of examiners. To gain access to the site or for more information regarding the process, please contact FacBiol@admin.cam.ac.uk. The timetable for appointment of examiners is set out in Ordinances:

Chairmen and Assistants: by the end of the Easter term

Seniors Examiners: by 1^st day of Michaelmas term

Internal and External Examiners: by 9^th November

Assessors: 4 weeks before the examination.

The General Board reviewed policies, procedures and documentation for external examining, and released the following advice;

Care should be taken to avoid nominations where conflicts of interest (either professional or personal) may arise.  External examiners should not be appointed if they are covered by any of the following categories:
- near relative of a member of staff or student involved with the programme of study 
- an examiner on a cognate course in the institution
- anyone closely associated with the sponsorship of students on the course
- anyone closely associated with placements or training
- anyone required to assess colleagues who are recruited as students to the programme of study
- anyone in a position to influence significantly the future of students on the programme of study
- anyone who has been directly involved as an external member of a validation panel for the programme

Should any concerns arise about possible conflicts of interest, advice or requests for permission under exceptional circumstances should be sought directly from the Education and Student Policy Section.

The General Board's full guidance notes on the appointment of External Examiners can be found here.

Preparation

In preparation, Senior Examiners should acquaint themselves with the relevant Ordinances, including the regulations for the particular examination for which they are responsible and should:

1. Liase with Course Organisers and/or the Head of Department to ensure all published information about the exams is up-to-date and correct. If there is to be a change to the form and conduct of the exam, a Form and Conduct Notice needs to be approved by the Faculty Board, and NST Management Committee, and published by the end of the Michaelmas term. The Education Section of the General Board have issued guidance on Form and Conduct Notices on their website. (The Faculty publishes current NST biology Part II Form and Conduct Notices).

    

2. In consultation with the Course Organiser, ensure that sufficient and appropriate Examiners have been appointed by the Faculty Board, and that if there is to be a change of Senior Examiner at the end of the year, determine which one of the appointed examiners will be the next Senior Examiner. This person should be designated as Deputy Senior Examiner.

    

3. Ensure that, where appropriate, an External Examiner has been appointed. There is no requirement for Part IA and IB courses of the NST to have External Examiners.

    

4. If one is not already in existence, produce a specimen paper. Specimen papers should closely reflect the examination layout in terms of the type, number and distribution of questions, without giving too many clues as to the actual questions which will appear in the examination.

Paperwork

It is recommended that the following records be maintained by the Senior Examiner to pass on for the next year:

1. Names of Senior Examiner, Examiners and External Examiner (with address)

    

2. Copies of any Form and Conduct Notice published in the Reporter for the Examination concerned. Any advice issued by the examiners to candidates in the form of sample questions, information placed on notice boards about what the exam will consist of, advice about when and where to attend, etc.

    

3. Copies of agenda and minutes of subject Examiners' meetings; copies of correspondence, if any, between Examiners and those teaching the course soliciting questions; copies of correspondence with the External Examiner.

    

4. Copy of exam paper and practical exam paper, together with any marking schemes or any other advice issued to examiners.

    

5. Copy of the complete internal subject mark-book (or computer print-out) thereof, in accordance with the appropriate Examinations Data Retention Policy.

    

6. Copy of External Examiner's report (which is formally sent to the Vice-Chancellor, but the External Examiner should be asked to send a copy to the Senior Examiner), which should be passed to the Course Organisers.

    

7. Copies of comments of individual internal examiners on the way that particular questions marked by them were answered - prepared in a way that will be useful as guidance for those teaching that part of the course.

    

8. A statistical summary (how many + percent failed to qualify where relevant, obtained various grades, ideally a cumulative histogram).

Dealing with Internal/External Examiners

The Senior Examiner should ensure that all internal Tripos examiners receive copies of the Faculty Board guidelines Guidance for marking written Tripos answers . It is important that examiners appreciate that the guidance is to assist them in forming a judgement on the quality of individual essays with the Tripos. For Parts IA and IB of the Natural Sciences Tripos, a candidate's class is more a measure of their relative standing with their peers than an absolute measure of ability. Marks in individual subjects are often moderated to ease comparison across subjects and to aid students and Directors of Studies in determining future courses of study.

Information about the way that marks for individual subjects are combined in the NST is available from the examinations section of the NST website.

The External Examiner should be sent:

(a) Information about the role of an External Examiner in the particular subject concerned, including times and dates of meetings, deadlines for submission of marks etc.

(b) the General Board's Guidance on the arrangements for External Examiners.

(c) Details about the course and examination, (for example, relevant extracts from the Lecture List University Guide to Courses, course syllabus and synopses, bibliographies, lists of prescribed topics texts, departmental booklets)

(d) For Tripos examinations, marking schemes and classing conventions, which you should receive from the Chairman of Examiners.

(e) A copy of the previous External Examiner's final report.

Examiners Meetings

For Parts IA and IB there will be two meetings of Senior Examiners - one in late Michaelmas or early Lent and the Final Meeting in June. All Senior Examiners should attend both meetings, and must be present at the Final Meeting in June to sign the Class List, unless special permission has been granted by the Vice Chancellor for absence.

The Senior Examiner is responsible for organising meetings of Examiners in their own subject. All Examiners, including the External Examiner, for the subject must be present when the marks of candidates in the subject are approved, and should sign a paper copy of the markbook.

For Part II the Senior Examiner is responsible for organising meetings of Examiners.  As for Part IA and IB there should be a minimum of two meetings.  The first should set the marking and classing criteria and discuss any issues arising from the previous years' examination, as well as reminding Examiners of guidance available to them.  The final meeting following the examination will set the class boundaries and produce a final class list which must be signed by all Examiners, including the External Examiners.   Attendance of the the final meeting is mandatory unless there are grave circumstances. 

Setting and Preparing the Paper(s)

See also code of conduct for examiners and assessors who supervisor in Easter Term

The aims and objectives should be clearly stated in the course handbooks and relevant lecture and practical notes, which are available from the course organiser, and the exam should be checked and mapped onto them. External Examiners should normally be given the opportunity to suggest questions and comment on, change/insert/remove questions, and to formally approve the exam paper.

The Senior Examiner will need to provide camera ready copy of the examination paper. The Reprographics Centre at the Old Schools will contact Senior Examiners with the timetable for this. Where appropriate, practical paper questions should be checked with Classroom staff to ensure the correct materials are provided for the exam.

All those concerned with the production of exam questions should be reminded that this should be done under conditions of the strictest security, and computers and printers should be secure from unauthorised access.

It would be useful, when submitting exam questions for NST IA/1B, if bullet-point answer guides are submitted at the same time. These should give the major points that might be expected in a good answer.

All Examiners and Assessors who are also involved in College teaching should be careful in supervisions leading up to the exam period. The Senior Examiners should ensure that all Examiners and Assessors are issued with a copy of the Examiners and Assessors Code of Conduct which has been approved by the Faculty Board.

Arrangements for Examinations which use Multiple Choice Question (MCQs)

The devising, compiling and setting of such questions can be a time consuming process but there is a considerable body of evidence that MCQs, properly devised, are effective at testing understanding of concepts as well as factual knowledge. The problem is that the process of devising good questions does take more time than setting short note or essay-type questions, which underlines the need to start the process of building up a bank of suitable questions in good time. Examiners intending to use MCQs are advised to begin to engage with the business of question setting as soon as possible, and are encouraged to build up a bank of questions that can be adapted and/or revised in future years.

There are several web sites available with further advice on the devising and setting of MCQs - here are three:

How to Write Tests. University of Lethbridge, Faculty of Education

Constructing Written Test Questions for the Basic and Clinical Sciences. US National Board of Medical Examiners

Extended Matching Questions. Association of American Veterinary Medical Colleges

MCQs can and should be machine marked, which means considerable time savings for examiners in the busy exam period.

Machine marking involves the use of standardised "mark sheets", which can be put through an optical mark reader to give automatic entry of the marks into the list. These mark sheets can be used not only for "one right answer" format, but also for "steeplechase", and short answer format questions.

UCLES have agreed that they will make available to examiners advice on devising appropriate mark sheets for different layouts and formats of questions, and optical mark reader facilities for machine marking these answer sheets. The "turn round" for this machine marking is about two days.

Senior Examiners should consult Jo Ireland at Cambridge Assessment (tel: 553853) about preparation of answer sheets as early as possible, ideally before Christmas, as she will need to schedule production of answer sheets with her other work.

In most cases it will simply be a case of updating last year's mark sheet, but for some examinations there may be a change in format. Mark sheets should be produced in parallel with production of the question paper to ensure that they are compatible. Reprographics will produce the required number of answer sheets and question papers.

Senior Examiners should submit a camera ready copy of the examination paper AND answer sheet to the Reprographics Centre at the Old Schools by the end of April. Reprographics will produce the required number of answer sheets and question papers.

After the examination, the Senior Examiner should collect the answer sheets from the exam hall (or arrange for them to be delivered to their department). The Senior Examiner should check the answer sheets to make sure all the candidates are accounted for and that candidate numbers have been entered on each sheet. The Senior Examiner should then get the answer sheets delivered to UCLES for processing.

Marks will be returned by UCLES to the Senior Examiner for incorporation into the markbook.

It would be helpful if Senior Examiners could adopt a standard format for their answer sheets as far as possible. In particular, the header for all examinations should include only the following information

• Tripos (NST)
• Part (Part IA or IB)
• Subject (for example Biology of Cells)
• Date of the examination
• Paper and section number (for example, Paper I Section I)
• Candidate number
• Instructions

There should be no box for candidates to indicate their gender.

The Board have agreed that ALL examination papers, including MCQ papers, should be published and they are therefore included in the bound volumes of papers produced for College libraries.  Exceptions to this must be agreed via the Faculty Board of Biology, who if they agree can make a request to the Education Section.

List of Candidates and Markbooks

For NST lists of candidates in a set file format are supplied by the Student Registry.

Senior Examiners will be given administrative instructions for dealing with marks at the first Examiners' meeting.

Invigilation and Arrangements during Exam Period

Senior Examiners should make arrangements for invigilation of practicals and the starting of theory papers at all venues. Detailed instructions are issued to Senior Examiners by the Student Registry in April. As candidates take time to read short answer and MCQ questions, Examiners should be available by phone to answer any questions that arise for the whole duration of any section of the exam containing this type of question.

Senior Examiners will also need to:

1. Arrange collection of scripts, and their checking and distribution to examiners, in liaison with the Student Registry. Examiners may also make special arrangements to collect scripts completed in College from the Student Registry. This is highly recommended as there can sometimes be delays getting these scripts delivered to departments.
2. Arrange the entry of question marks into the subject markbook. Candidates should record in the cover sheet which questions they have attempted, and examiners are advised to check scripts as they are received and to record the details of any that appear to have material missing, or to have been damaged. This will help in determining at what stage material might have been lost. They should report any apparent loss of material to the Student Registry as soon as possible, so that searches of the exam room can be arranged. Senior Examiners should report to the Chairman of Examiners on any apparent loss of answers; if material is not found the Chairman will decide whether this could have affected the overall class of the candidate concerned.
3. Process marks, and present them to the internal and external examiners; chair discussions on the marks, the distributions of the marks, scaling etc.
4. Deliver electronic and signed hard-copy final marks to Ms Jane Clare, Education Section. After the Final Meeting of Examiners, copies of the mark book are distributed by Ms Clare to Colleges so that students can be told their marks.

Examiners should not discuss any aspect of the exam with students.

Students should be aware that they should not attempt to contact Examiners directly. It has been found that they have, on occasions, been known to do this. Examiners should pass any e-mail or letters received from candidates direct to the Chair of Examiners, who will issue a standard reply advising students to raise any queries through their Director of Studies.

Guidance to Examiners on Double Marking

Senior Examiners should consult the Faculty Board's guidelines on double marking.

NST Part II BBS: Handling of Dissertation Marks and Oral Exams

The Faculty Board has approved the following guidelines for handling of NST Part II BBS Disseration marks:

(a) Subject examiners should consider whether it would be appropriate for their subject for dissertations to be submitted anonymously. The guiding principle in handling dissertations should be that every effort must be made not to compromise the anonymity of candidates for the whole examination, even if it is not appropriate to handle dissertations anonymously (for example where all candidates are given an oral exam).

(b) It is recommended that dissertations should be double marked, and that where they are double marked, each examiner should mark independently. The Senior Examiner for the parent single subject should determine marks for individual dissertations based on the marks submitted by examiners or assessors. If there is a wide discrepancy between the two sets of marks for an individual dissertation, and the difference cannot be reconciled by the two examiners, the Senior Examiner should arrange for remarking and/or an oral exam.

(c) If dissertations are not submitted anonymously, after marking a named assistant should encode the dissertation marks and enter them into the markbook. The Faculty Office will supply named assistants with a markbook showing name and number so that this can be done.

(d) From 2015 it was agreed that no BBS student should be examined viva voce for their dissertation.

Examiners Reports

After the examinations, all Senior Examiners will need to prepare a report for the Faculty Board on the examination, its components, student performances, the marks given etc, and any matters of exam organisation and teaching/learning of the material in the course that need attention, for the Faculty Board and feedback to Teaching Committee and Course Organiser's meetings.

Reports should state prominently the number of candidates, together with the mean and standard deviation of the marks that are reported to the Chairman of Examiners and the percentages of candidates falling within each of the notional subject classes. Senior Examiners are also encouraged to report on rescaling, if any, applied to the marks.

Senior Examiner's reports should be sent to the Secretary of the Faculty Board, copied to the Course Organiser, within one month of the examination, as there is very little time to make changes in a course following an exam. The report should also be sent to the person who will be Senior Examiner in the following year.

External Examiners are asked to report to the Vice Chancellor. Copies of their reports are fed to the Faculty Office and to Heads of Department by the Education Section of the General Board. The Senior Examiner should liaise with the Course Organiser (for interdepartmental courses) or Head of Department (or other designated person) over an appropriate response to the External Examiner's report and prepare recommendations for any changes to the examination next year, to put to the Faculty Board. Recommendations should be made via the Biological Sciences Committee. Copies of any response made to the External Examiner's report should be sent to the Education Section of the General Board and to the Faculty Office.

The Faculty Board has agreed that the reports will be published on the web. They are available here.

Retention of markbooks, scripts and other paperwork

Senior Examiners are responsible for destroying confidential waste and computer records in accordance with the appropriate Examinations Data Retention Policy for the NST. Interim marks, marks for individual questions and other such data should be retained for one month after publication of the class list, in case an appeal is lodged. If an appeal is lodged, the data for that student should be retained until the outcome of the appeal is known. Scripts should be retained for six months and may then be destroyed. The Student Registry will store scripts and arrange for their disposal. Please pack scripts in envelopes not black sacks.

Annotating Scripts

The Faculty Board recommend that Examiners follow the guidance issued by the Education Committee regarding writing on scripts. This is advice is based on two principles: objectivity of marking (where double marking is used) and the administrative complications of retaining and disseminating comments on scripts to students if requested to do so under the Data Protection Act 1988. Examiners are recommended to document their marks and comments separately from scripts, but where they think it sensible and prudent for marks and comments to be made on scripts, Examiners should take due care that this data can be retrieved easily if a request is made. It is the responsibility of the Examiners, or the relevant department, to ensure that there are resources and procedures in place to deal with any such requests as they arise. The return of written answers in exam scripts is exempt; only examiners' comments and marks need be returned.

Appeals Procedure

Details of the University's review procedure can be found here.

No representations under the procedure will be considered unless they relate to the conduct of the examination. The procedure applies to the Tripos only.

Changes to Regulations

Due Notice must be given to all parties about any proposed changes to course delivery or assessment so that no candidate is disadvantaged.  The Education and Student Policy Section have provided guidance on changes to Regulations.