Multi-Camera AR Guidance System for Surgical Instrument Handling and Assembly: Investigating Workload and Efficiency

Problem
The paper addresses the high cognitive demands placed on scrub nurses during surgical procedures, particularly when handling unfamiliar instruments. This gap in capability is critical, as existing methods often rely on paper manuals, which can lead to increased workload and inefficiencies. The authors present a preprint work that proposes a novel augmented reality (AR) guidance system that does not require additional markers, aiming to improve the handling and assembly of surgical instruments.

Method
The proposed system integrates multi-camera 6D pose estimation with AR visualization on a head-mounted display. The 6D pose estimation network is trained exclusively on synthetic data, which is intended to enhance generalizability and applicability in real-world surgical settings. The system utilizes known objects for camera calibration and pose estimation, allowing for accurate tracking without the need for physical markers. The AR interface provides tooltip localization cues and step-wise assembly animations, enabling scrub nurses to navigate through the assembly process. Interaction is facilitated through gaze-based selection and a foot pedal, allowing users to switch between assembly steps seamlessly during surgery.

Results
In a technical evaluation, the AR guidance system demonstrated superior performance compared to state-of-the-art 6D pose estimation methods. A user study involving 29 scrub nurses was conducted in a simulated knee arthroplasty environment, comparing the AR system against traditional paper manuals. The results indicated that AR guidance significantly reduced perceived workload and objectively decreased task completion time by 21.3%, equating to a reduction of 4.76 minutes. Notably, scrub nurses with less experience using the instrument set showed greater benefits from the AR system. Error frequencies remained comparable between the AR and manual conditions, suggesting that the system does not compromise accuracy. Qualitative feedback from participants highlighted improvements in process clarity, reduced information overload, and a sense of increased independence during the procedure.

Limitations
The authors acknowledge that the study is limited by its small sample size and the specific context of knee arthroplasty simulation, which may not generalize to all surgical scenarios. Additionally, the reliance on synthetic data for training the pose estimation network may introduce limitations in real-world applicability, as the system’s performance could vary with different instrument sets or surgical environments. The lack of long-term evaluation of the system’s impact on surgical outcomes is also noted.

Why it matters
This work has significant implications for the integration of AR technologies in surgical settings, particularly for enhancing the performance of scrub nurses who may be less familiar with specific instruments. By reducing cognitive load and improving efficiency, the proposed system could lead to better surgical outcomes and training processes. The findings contribute to the growing body of literature on AR applications in healthcare, as discussed in related works on AR in surgical training and instrument handling, and are available on arXiv.