Using Watershed Transform for Vision-based Two-Hand Occlusion in an Interactive AR Environment

Main Article Content

Peng Peng Leim, Guat Yew Tan, Kah Pin Ng, Miin Huey Ang


To achieve a natural interaction in augmented reality environment, we have suggested to use markerless vision-based two-handed gestures for the interaction; with an outstretched hand and a pointing hand used as virtual object registration plane and pointing device respectively. However, two-handed interaction always causes mutual occlusion which jeopardizes the hand gesture recognition. In this paper, we present a solution for two-hand occlusion by using watershed transform. The main idea is to start from a two-hand occlusion image in binary format, then form a grey-scale image based on the distance of each non-object pixel to object pixel. The watershed algorithm is applied to the negation of the grey scaled image to form watershed lines which separate the two hands. Fingertips are then identified and each hand is recognized based on the number of fingertips on each hand. The outstretched hand is assumed to contain 5 fingertips and the pointing device contains less than 5 fingertips. An example of applying our result in hand and virtual object interaction is displayed at the end of the paper.

Article Details