Automatic trailer generation for movies using convolutional neural network.

In this paper, a new method for producing movie trailers is presented. In the proposed method, the problem is divided into two sub-problems: "genre identification" and "genre-based trailer production". To solve the first sub-problem, the poster image and subtitle text processing strategy has been used in which, a convolutional neural network (CNN) model has been used to extract features related to the movie genre from its poster image. In addition to these features, the content features of the movie are described in the form of a TF-IDF vector, which is extracted through the pre-processed text of the subtitle. Using a classification and regression tree, a combination of extracted features is classified to identify the genre of the movie. After determining the genre of the movie, a CNN model is defined for each movie genre, and this model is trained based on the key sequences of movies of a particular genre. For this purpose, at the beginning of the second phase of the proposed method, the film is divided into its constituent scenes, and then two categories of visual and textual features are used to describe the characteristics of the scenes. These features are combined to determine the key scenes based on the CNN model corresponding to the movie genre. Finally, the movie trailer is created based on the time constraint specified by the user. The performance of the proposed method has been investigated from two aspects of genre recognition accuracy and trailer production quality. Based on the results, the proposed method can achieve 83.39% accuracy in detecting movie genres, which is at least 1% higher than compared methods. On the other hand, the 56.69% precision of the proposed method in trailer generation shows an improvement of at least 8% compared to the compared methods.