Plane Detection with Hololens

Abstract

One of the main problems in mixed (MR) and augmented reality (AR) is the placement of virtual content. In non-AR apps, UI-elements are mostly placed in the same location of a fixed window. In AR apps, on the other hand, virtual content can be placed in the user’s surroundings. The difficulty is that the space around a user constantly changes in contrast to a fixed window. A general automatic method for content placement in AR is therefore of great use. In this thesis, we present a method to find suitable locations for content placement in AR. First, a plane detection algorithm is used to find planar surfaces in the three-dimensional world space. Found planes are further divided into small sections and analysed to decide how suitable a location is for content placement. Position based criteria like distance-size ratio, the user’s field of view, viewing angle and occlusion are used to rate locations (sections) according to their visibility to the user. Criteria based on the colour and structure of virtual content and the real world are used to find locations where content stands out from the background and is legible. Object detection is used to find locations that might be of interest to the user. The result of this thesis is a complete method that combines plane detection and the above criteria to find suitable locations for virtual content placement in MR or AR.