People location tracker using multiple cameras

Problem: Determine the position of people in in-door environments in real-time.

Why is it important?

  1. For human-computer applications (games, automatic control)
  2. Automatic activity recognition in the home setting
  3. Medical applications

The system consists of the following components

  1. Six cameras located at a 9 feet height in the room
  2. Personal Digital Assistant to train the system (ipaq)
  3. Pattern Recognition algorithm to detect the location of persons

Cameras

Location of the cameras inside the living room

 

Diagram of the conections and the distribution of the system

Algorithm

For each cluster tracker camera

  1. calculate frame by frame difference
  2. dialate the image by one pixel
  3. form Clusters from the image (label, bounding box, number of pixels, average color and velocity)
  4. merge clusters that are similar (dist,color,size, and velocity)
  5. Calculate largest cluster (with more area in bounding box)

For the control computer

  1. Get the largest cluster for each cluster tracker camera
  2. calculate the center of the bounding box and form feature vector

  3. converted the (x,y) coordinate of the bbox to a single value 2D->1D (y*width+x) The features are the coordinates of the bbox for each camera (six features). I got 100 examples for the eight locations in the room. (800x6 matrix)

  4. apply linear discriminant analysis to the feature vector

  5. use a support vector machine with a Gaussian kernel to classify the point

Graphical explanation of the steps involved in extracting the bounding box of a person

 

Results for the largest cluster of all the cameras in the sytem

Training of the system

In order to train the system, a PDA application in java was built so that a person inside the room could be able to specify his location. For example, if the person wants the system to recognize the couch, he just goes to the couch, selects the option "couch" from the PDA application and then hits the send button. The person needs to start moving aroung the area to be recognized, in this case, the couch. the system them starts recording 100 examples of the positions of this person from the six cameras to build a feature matric 200x12. After all the examples of all the locations to be recognized inside the room have been recorded, the control computer translates the examples to the LDA space and then use these examples to train a support vector machine with a gaussian kernel. In this case, the system was trained to recognize the following locations: couch, TV, right shelf, left shelf, drawer, desk, mirror and pictures. In total, this system is able to recognize a person at eight different locations inside the room.

 

PDA and java program used to train the system

Processing the data

These are the results of applying LDA to the training data

LDA results for the training set

These are the results of using different kernels, dimentions and parameters for the support vector machine used in the classification step

 

Video of the performance of the system

<a href="../videos/locator.rm"> Play Using Real Player Application</a>