Event Generator

The event generator in our system is responsible for mapping the tracks produced by the tracker onto a set of pre-determined discrete events. These events form the basis for the syntactic constraints that the parser enforces on the input stream. The events in our system are not object-centered, but rather are formulated in terms of tracker states. While identity of an object is unknown to the tracker, it remains the same throughout the detected object trajectory. The tracker can "lose" an object and then "find" it again later due to occlusion or sudden change of lighting, in which case the identity of the object is not preserved. Reasoning about object identity in such situations is deferred to the parser which can enforce contextual information. In this reasoning, exact configuration of the object trajectories is not important. Only the endpoints of the tracks influence how the identity is computed.

Every time the tracker reports beginning or the end of a track, the event generator produces an event. This set of primitive tracker states, such as object-lost, object-found, forms the alphabet of interactions, presented to the parser in form of a grammar.

In order to generate events, the event generator is given a simple environment map. Based on this map the event generator determines if the events occurred in ``special locations'', where the objects tend to enter or leave the scene. Besides that, if the tracker does not have sufficient degree of confidence about the object's class, multiple candidate events are generated, one per candidate class with likelihoods assigned to each according to what's reported by the tracker. The events are generated according to the following rules:

.

If the track began in an area where objects tend to enter the scene, car-enter and person-enter events are generated. The events are marked with the corresponding likelihoods to account for errors in classification. For instance if a beginning of a person track is reported by the tracker and the likelihood of that event is 0.7, a person-enter event with likelihood 0.7 is posted to the parser. Along with it, a complementary event car-enter is posted in the same time slot, with the likelihood of 0.3.

.

If the track did not begin in one of the "entry" areas, car-found and person-found events are generated.

.

If the track ended in one of the "exit" areas, car-exit and person-exit events are produced.

.

If the track did not end in one of the "exit" areas, car-lost and person-lost events are posted.

.

If an object's velocity dropped below a certain threshold, an object-stopped event is generated.

Figure 2: Illustration of a process of mapping tracks onto discrete events. The tracker reports beginning and the end of the track. In this example, the beginning of the track corresponds to an object entering the scene. At that point the class label of the class cannot be determined. This results in generation of two concurrent events - one per class (cars and persons) with probability of the label being 0.5.

The process of generating events is illustrated in figure 2. Note that some of the track endpoints are mapped onto a pair of concurrent events, which accounts for classification errors. The parser will select one or the other, depending on which one results in the overall parse with maximum probability. Typically, at the beginning of each track, the tracker has not observed the object long enough to be certain about its class membership. Therefore, x-enter and x-found events have likelihoods close to 0.5. In contrast, by the time the object disappears or is lost, there is enough data to make more accurate classification decision. Consequently, class likelihoods of x-exit and x-lost events are typically higher than those of x-enter and x-found.