Simulator assessment of innate endovascular aptitude versus empirically correct performance.

Objective: Prior work has established that performance on an endovascular simulator incorporating tactile feedback (haptics) correlates with previous endovascular experience and can be improved by training. This study was designed to test the ability to define and measure innate endovascular aptitude and empirically correct performance and to determine whether these are two different things.

Methods: Subjects ranging in endovascular skill level from novice to expert were surveyed to determine video game experience and skill, endovascular level of training, and endovascular experience. They were then tested by using a standard protocol requiring timed advancement of a catheter and wire sequentially into each of three vessels arising from a simulated type I arch. Recorded trials were independently and blindly scored by two experienced endovascular faculty members by using a modification of a previously validated scale (Modified Reznick Scale; MRS). Summed scores were analyzed by frequency analysis and categorized as satisfactory and unsatisfactory on the basis of a clear bimodal distribution. Categorical outcome, time to task completion, and other variables were analyzed by means of linear regression, analysis of variance, and Welch modified two-sample t tests, as indicated.

Results: A total of 61 subjects were enrolled: 42% students, 8% technicians, 19% surgeons, 13% cardiologists, and 18% radiologists. Of these, 62% were considered novices and 30% experts on the basis of previous experience; 56% of subjects worked in an endovascular-related occupation. MRS scores were highly correlated between raters (P < .0001) and showed a clear bimodal distribution, with subjects in any endovascular occupation (including technicians) scoring significantly better than all others (P < .0001). Hours of video games played per week were correlated highly with completion times (P < .001) and MRS scores (P < .001). Measures of formal training (number of endovascular cases and occupation) correlated highly with completion times (all P < .03) and MRS scores (all P < .008). In comparing completion times vs MRS scores, three groups were apparent: unskilled-inexperienced, skilled-inexperienced, and skilled-experienced, corresponding primarily to senior subjects without endovascular experience, younger subjects without endovascular experience, and formally trained endovascular physicians, respectively. Those judged intermediate in aptitude reduced times to the lowest possible level before improving their MRS scores.

Conclusions: Although inherently subjective, the MRS yields reproducible scores that correlate with endovascular experience and formal training. Experts and novices with extensive video game experience achieve short completion times, whereas high MRS scores are achieved only by formally trained subjects. Innate endovascular aptitude and empirically correct performance may be two separate things, and aptitude may be acquirable through (or identified by) extensive nonmedical video game experience.