Comparison of deviceless methods for distant object manipulation in mixed reality

Mixed Reality (MR) applications based on the current generation of Head-mounted displays (HMDs) support a deviceless manipulation of virtual objects based on finger tracking and control of the 3D transformation with gestures. However, when the object manipulation is performed at some distance, and when the transform includes scaling, it is not apparent how to remap the hand motions over the degrees of freedom of the object. The most popular software toolkits used for XR development provide specific solutions, but there are still usability issues and a need for clear guidelines for the interaction design. In this work, we compare three different solutions for the devices and remote translation, rotation, and scaling of virtual objects in a real environment in two user studies aimed at assessing the usability of the methods for different tasks. The tasks are a fast and rough docking of virtual cubes on a tangible shelf from varying distances and the accurate placement of cubes on the shelf in a fixed amount of time. The outcomes of these studies show that the usability of the manipulation methods is strongly affected by the use of separate or integrated control of the degrees of freedom, by the use of the hands in a symmetric or specialized way, by the visual feedback, and by the previous experience of the users and provides valuable guidelines for the design of interactive mixed environments.