Detecting events in videos using semantic analytics of subtitles

Abstract

Recently, television broadcasters such as the NRK and TV2 channels, have begun offering live internet television and movie archive along with their regular schedule, much like the known video archives such as Youtube and Vimeo. The amount of all television offered reduces the ability of the user to get an overview of the programs that are available at any given time, making the user will probably miss important events. Regular indexing engines for recommending does not generally work on media since it is hard to index media data. Tags and keywords describing a media le does only describe the whole le, making it difficult to use them for indexing and recommendation of specific scenes within the media. This thesis presents a text event detection system for discovering interesting events based on video subtitles. By performing textual analytics, our system is able to discover events that are not discoverable through regular syntactic search. We have, based on related work, extended algorithms used for discovering semantic relationships between different words. Also, we have experimented with several algorithm for capturing the essence of each sentence, relating the prominent sense of each sense towards the events. Our experiments illustrates how we can increase the accuracy of the algorithm used by performing a context exploration based on event keywords. The results shows that our system improved the base algorithm of the prototype by including more relevance methods like consecutive sentence similarity and similarity based on sentence internals.