Objective Evaluation of Motor Skills for Orthopedic Residents Using a Motion Tracking Drill System: Outcomes of an ABOS Approved Surgical Skills Training Program.

Background: Orthopedics is a motor skills-demanding surgical specialty requiring surgical skills training outside of the operating room. Unfortunately, limited quantitative techniques exist to determine the effectiveness of these surgical skills training programs. Using a variety of drill, surgeon, and specimen mounted sensors, we evaluated orthopedic surgery residents during a surgical skills training course approved by the American Board of Orthopaedic Surgeons (ABOS). This evaluation consisted of quantitative measures of various kinematic and kinetic parameters with the goal of relating these to clinically-significant outcomes.

Methods: Seven experienced surgeons and 22 surgical residents participated in this study, each performing 5 surgical drilling trials, pre- and post-training. Utilizing arm and tool kinematics, applied force, tool and bone vibration, and drill RPM were measured using a combination of force, acceleration, and optical tracking sensors. Post hoc screw pullout testing and resident survey data were also evaluated. Overall, 25 measured parameters were expressed as scalars and their covariance calculated.

Results: Non-trivial direct correlations whose magnitude exceeded 0.5 were: maximum penetration distance with applied force, drill toggle with drill roll angle, and drill RPM with force. Surgeons applying a high drill RPM also yielded a large force which in turn gave an increase in tendency for over-penetration. As a whole, the differences between experienced and novice surgeons measured in these trials were not statistically significant. However, when looking at specific performance criterion individually (maintaining steady force, minimizing over-penetration, minimizing both the major and minor axis diameters, minimizing toggle and drill vibration), experienced surgeons tended to outperform their novice counterparts.

Conclusions: Objective assessment of surgical skills using sensor based technologies may help elucidate differences between novice and experienced surgeons for improved out-of-the-OR training methodologies.