The inter-rater and intra-rater reliability of a new Canadian oral examination format in anesthesia is fair to good.

Purpose: In response to the Royal College's request to improve the validity and reliability of oral examinations, the Examination Board in anesthesia proposed a structured oral examination format. Prior to its introduction, we studied this format in two residency programs to determine reliability of the examiners.

Methods: Twenty faculty and 26 residents from two Canadian residency programs participated (Sites A and B).