A scenario in videos is a set of objects (including people) interacting over space and time, e.g., a patient making a cup of tea at home, which is recognisable and meaningful. Video frames can be represented as object maps, which are spatially interrelated objects. Object maps change between frames. Scenario recognition in videos is to make sense of the sequences of object maps. It can be used for detecting normal and abnormal situations in applications such as assistive living, or more specifically, smart homes.

The predominant techniques for scenario recognition are probabilistic. Dynamic Bayesian Networks (DBN) is a probabilistic technique that has been widely used in computer vision for scenario recognition. Hidden Markov Models (HHM) is another popular probabilistic technique for scenario recognition. DBN and HMM are both numerical techniques, and training is required. It is difficult to have a training video archive that covers all appearances of a scenario, and the temporal variability is a challenge.

At the University of Ulster, extensive work has been conducted on sensor-based scenario/activity recognition for assistive living. In this project we plan to extend this line of work to a new arena that is based on videos – video-based scenario recognition. We plan to address the challenges in video-based scenario recognition by taking an ontology based approach augmented by DBN, HMM and the neighbourhood counting methodology.

Reference