Task recognition with Hidden Markov Models:

The preceding section shortly described a generic object recognition system which is the basis of our computer vision system for the recognition of user tasks. As mentioned above, the recognition system is used for the recognition of image patches which correspond to appearances of a hand, a portion of an arm or any part of the background. In order to use the recognition system we define a library of 30 images (grouped into images corresponding to the same action and chosen arbitrarily from the Patrol data). Each of the images are split into 4x4 sub-images which are used as image patch database. In the experiment below we define three different image groups, one of each action. When applied to the incoming video stream from the camera, the system calculates 3 groups$\times$16 = 48 probabilities at 10Hz. This probability vector is then used as feature vector for a set of HMM which have been trained to recognize different tasks of the user.