Style Similarity Measure

Jpetiot/ janvier 7, 2005/ Applications

Context 

Based on a similarity matrix, we define a generic measure for video documents, to identify style similarity. We consider that similarity in style relies on the occurrence of common elements between the compared documents, from a production point of view. Those common elements – we call them production invariants – can be characterized by a combination of audiovisual characteristics. They can be highlighted by the fact that the dominant color (corresponding to a given set, a given lightning) evolves in the same way along two different documents from a low-level point of view, or the fact that a same commercial is repeated at different moment in a TV program at a higher level point of view. Measuring the degree of style similarity between two compared documents relies then, on the ability to quantify the occurrence of these common elements.

Overview

We can argue that the general order of the events in videos has a strong influence on the similarity. Therefore we look for the diagonal on which we can observe the highest similarity coefficients to establish a similarity measure. Let σk be the sum of coefficients along the kth diagonal normalized by the number of coefficients on this diagonal. Let dim be the number of rows (or columns) of the similarity matrix

We defined a vector Wk of weights which allows to give more importance to coefficients located around the kth diagonal than to the other.

We compute Mfk which is the sum of all the weighted coefficients Sk obtained for a similarity matrix produced for a given feature f.

As far as there may be several features used as an input for this processing, there may be several similarity matrix, and so, several Mfk coefficients. Finally, we compute the overall similarity coefficient Mf as the maximum of all the Mfk coefficients obtained for each similarity matrix.

 We can argue that the general order of the events in videos has a strong influence on the similarity. Therefore we look for the diagonal on which we can observe the highest similarity coefficients to establish a similarity measure. Letσ k be the sum of coefficients along the kth diagonal normalized by the number of coefficients on this diagonal. Let dim be the number of rows (or columns) of the similarity matrix in this formula.
 We defined a vector Wk of weights which allows to give more importance to coefficients located around the kth diagonal than to the other.We compute Mfk which is the sum of all the weighted coefficients Sk obtained for a similarity matrix produced for a given feature f.
 As far as there may be several features used as an input for this processing, there may be several similarity matrix, and so, several Mfk coefficients. Finally, we compute the overall similarity coefficient Mf as the maximum of all the Mfk coefficients obtained for each similarity matrix.

Applications

 Distance between documentsIn this experience, we took 10 recordings out of the TREC Video 2004 database: 9 TV News programs from CNN (number 1 to 9) and 1 from ABC (number 11) and we added 1 TV game program to that collection (number 10). For each program, we computed its mean similarity with all the documents of the collection.  The diagram clearly show that the TV game program is an outsider in this collection while the ABC program is slightly different from the CNN ones. 
   Production invariants detection
On the previous graphic, we can see that document N°5 is the most similar to the entire collection. We have computed the similarity between this document and all of the other CNN TV news programs. For each minimal sequence of tmin duration of this document (see Similarity matrix), we summed the similarity coefficients with the other recordings. We obtained the results given by the green histogram. If we now choose a threshold to identify sequences which are the most similar with the other documents, we then obtain excerpts such as anchors, sport introduction, headlines or weather forecasts.  (keyframes from the TREC Video 2004 database) 

Contributors

Main publications

Share this Post