Hand shape recognition is one of the most important techniques used for human-computer interaction. However, it often takes great efforts for developers to customize their hand shape recognizers. In this paper, we propose a novel method that enables a user to create a hand shape recognizer from simple sketches. The user only needs to produce a “stick- figure” of a hand shape, and our method will generate a hand shape recognizer automatically. We propose the Hand Boltzmann Machine (HBM), a generative model built upon unsupervised learning, to represent the hand shape space of a binary image, and formulate the user provided sketches as an initial guidance for sampling to generate realistic hand shape samples. Such samples are then used to train a hand shape recognizer. We evaluate our method and compare it with the other state-of-the- art models in three aspects, namely i) its capability of handling different sketch input, ii) its classification accuracy, and iii) its ability to handle occlusions. Experimental results demonstrate the great potential of our method for real world applications.
My presentation is also available online. It is a bit huge (19.3MB) and generated by Keynote.
The data set is available upon email. Please contact me at xlz...@cs.hku.hk
