Some Reliability Issues for Simultaneous Interpreting Assessment

Shao-chuan Fred Wu
Postgraduate Programme in Translation and Interpreting
School of Modern Languages
University of Newcastle upon Tyne, UK

The test design and assessment criteria for evaluating interpreters' performance are still being described as intuitive (Campbell, 2003). In their review on the research work about translation and interpreting assessment, Campbell and Hale found that "there exist a number of knowledge gaps" (2003: 217). These gaps mainly concern the reliability and validity issues in the assessments, especially within the context of educational measurement. After a comprehensive discussion on the foundations of interpreter performance assessment, Sawyer (2004) also suggested nine steps toward a valid and reliable interpreting assessment, including exam procedures, test specifications, exam materials, assessment criteria, and raters training, etc. (2004: 128). Studies on conference interpreting quality assessment in the past decades had provided useful guidelines for interpreter training and assessment. What constitute a good interpreter can now be defined more clearly, and research literatures on the criteria for the entrance/exit exams of interpreting courses receive the most attention. However, it is not until recent years have the researchers' attentions been drawn to the assessment issues of the student interpreters during the training process (Hatim & Mason 1997, Gile 2001 in Sawyer, 2004: 93-94}. The design and administration of the in-training assessments are still mainly relied on the experiences of individual trainers, and inconsistencies of the exam procedures exist among many education institutions. It is even more important to look into the behaviours of jury examiners in interpreting exams because any well-thought exam criteria, procedures and materials will be of little value in test reliability if the examiners don't use them in a consistent manner. Understanding how the examiners use the criteria will also help develop a more practical and reliable assessment instruments.

This paper attempts to review and look into some reliability issues of the interpreting assessment, especially simultaneous interpreting assessment, within the context of educational measurement. Some interpreting assessment tools and procedures will be reviewed. A preliminary report on the findings of a study on the criteria used by the interpreting examiners and how the criteria are being used will also be presented.