NARA – Nakamir Augmented Reality Assistant for Training, Rehearsal and Guidance

C. Leuze
Nakamir, California, United States

Poster stand number: T151

Keywords: Augmented reality, Training, IVAS, Medical, Software

NARA is an augmented reality (AR) based training platform that allows instructors and experts to quickly create custom training content and students to consume that training content also in far-forward environments. NARA records an instructor while they interact with a real-world object while wearing a Hololens 2 AR headset. The instructor can create virtual 3D annotations during the interaction, explain the interaction using voice and record a 3D rendering of the real-world object. From this recording, NARA then creates a 3D virtual assistant that can be viewed by the student. The student can follow along the scenario while the virtual assistant interacts with the real-world object. This allows the student to follow the assistant’s hands or gaze, view annotations the instructor placed and hear audible instructions explaining the scenario. The student can also view the recording virtually by viewing the assistant interacting with a 3D virtual rendering of the real-world object. We use computer vision to ensure that the virtual assistant is shown at the correct location with respect to the real-world object. The software is agnostic to the use case and works e.g. for medical training mannequins but also for objects like an engine or large equipment.