Unity 3d game engine

The training software was developed in the Unity3D game engine (Version 2021.3LTS), using the Pico XR SDK (version 1.2.4). The Pico Neo 2 Eye VR headset was used for development and training. It provides stand-alone functionality, meaning that no connection to a computer or any external tracking devices is necessary. The device features a 75 Hz display refresh rate and the VF per eye is stated to be 101° [30 ] according to the developer’s specifications, though independent measurements have shown a VF per eye of 89° both horizontally and vertically [21 (link)]. The built-in eye tracker of the Pico Neo 2 Eye has a refresh rate of 90 Hz and an accuracy of 0.5° according to the device specifications, with an ideal eye-tracking range of 25° horizontally and 20° vertically [30 ]. The use of VR is possible while wearing glasses or contact lenses.

The goal of the task consisted of pairing—without any order restriction—a set of 3 randomly placed red squares with 3 randomly placed gray squares using a computer mouse device (Figure 7). The game was custom developed using the Unity 3D game engine (Unity Technologies). To complete the task, the participant had (1) to select a red square by pressing the left mouse button (the square becomes green after selection) and (2) select an available gray square by pressing the left mouse button. The right and wheel buttons were deactivated. Audio feedback was provided with “Very Good!” whenever the participant paired all squares.

Participants wore an HTC Vive head-mounted display (HMD; New Taipei City, Taiwan) equipped with a Tobii eye tracker (Tobii, Stockholm, Sweden) and held an HTC Vive controller in their dominant hand. The two 1080 × 1200 px OLED screens have a refresh rate of 90 Hz and a combined field of view of approximately 100° (horizontally) × 110° (vertically). The integrated Tobii eye tracker (Tobii, Stockholm, Sweden) recorded eye movements binocularly with a refresh rate of 120 Hz and a spatial accuracy below 1.1° within a 20° window centered in the viewports. The experiment was implemented in C# in the Unity 3D game engine (version 2017.3; Unity Technologies, San Francisco, CA, US) using SteamVR (version 1.10.26; Valve Corporation, Bellevue, WA, US) and Tobii eye-tracking software libraries (version 2.13.3, Tobii, Stockholm, Sweden) on a computer operated with Windows 10.

To compare agents’ with humans’ wayfinding performance (see Study 1), we replicated the desktop VR experiment setup and conducted two Monte-Carlo-type simulation experiments: (1) with visibility-based cognitive agents and (2) agents without visibility that rely solely on a ‘direct routing’ algorithm (i.e., the A* algorithm) to calculate the shortest path from the entrance to each destination. For each simulation experiment, a total of 1600 Monte-Carlo-type samples were taken. The agent’s initial heading was used as the random variable. The experimental framework used to develop and execute the simulation processing was based on the Unity3D game engine (Unity Technologies). We used Unity3D to render, control, record, and replay agents’ movements in the virtual environment. Trajectories were recorded by logging agents’ positions and orientations every 0.2 s. The simulations were executed using ETH Zurich’s Euler computing cluster through singularity-based containerization. To compare agents’ and humans’ wayfinding behaviors, two of the wayfinding behavioral measures used to quantify human behavior were also calculated for agents: (1)‘Time to Escalator’ and (2) ‘Average Cosine Similarity’.

The AR instructions were designed as a step-by-step guide that included the same text, pictures, and videos as the conventional instructions. In addition, simple 3D models were displayed in two steps (Fig. 3). It was developed for the Microsoft HoloLens 2 (Microsoft, Redmond, Washington) using the Unity 3D Game Engine (Unity Technologies, San Francisco, California). The application can be controlled both by hand gestures and voice commands. Using the outstretched index finger, the user can interact with the interface simply by moving the finger “through” the projected button in the same way one would press a physical button. Audio as well as visual feedback indicates that a button was successfully pressed. Voice commands work either by reading out the name of a particular button or by focusing the eyes on the button and saying “select”.