This post is part of a series about Artificial Intelligence (AI) and its potential role in science communication. In this third post, I describe how the computer, Watson, helped create a movie trailer.
Before we get to the Watson movie trailer, let’s first think about how movie trailers are made. Movie trailers are designed to convince people to go see a particular movie. Superficially, trailers appear to be a condensed version of the film, but good trailers are carefully designed to raise expectations and to appeal to the viewer’s emotions. Most trailers follow a typical formula, modified for the genre such as Action/Adventure, Comedy, Drama/Thriller, or Horror. Many trailers begin by introducing the characters and the setting of the film. Next to appear are the obstacles that change that world and set the characters on a new course. This may be followed by increasingly exciting, funny, or tension-filled scenes to ramp up the viewer’s desire to find out what happens. The specifics—selection of clips, the way they are cut (rapid-fire or slow-reveal), the fonts used for text titles, narration, music, and other choices—differ among movie genres.
All, however, are built more or less the same way by the trailer editor. The original movie is first watched carefully and deconstructed to reveal its basic components, visual and audio. The process then slices the movie audio and video further into segments that can then be rearranged to build the trailer. Next comes the choice of the best elements to use. Is the acting superb? The cinematography? The story? Editors often select those elements that highlight the merits of the film or the ones that have the most emotional impact on a viewer.
Not surprisingly, the AI-enhanced trailer of the movie Morgan was created in much the same way as a regular trailer. The first step, however, was to train Watson to understand what a movie trailer is and what features of a movie are used in movie trailers. The IBM team did this through machine learning and Watson APIs (Application Programming Interfaces, i.e., programming instructions). Basically, each of 100 movie trailers was dissected into component scenes, which were then subjected to the following analysis: (1) Visual (identification of people, objects, and environment), (2) Audio (narrator and character voices, music), and (3) Composition (scene location, framing, lighting). Each scene was tagged with one of 24 emotions (based on visual and audio analysis) and further categorized at to type of shot and other features.
Once Watson was trained, it was fed the full-length movie, Morgan. Based on its knowledge of what makes up a movie trailer—particularly a suspenseful one, Watson then selected ten segments as the best candidates for a trailer. These ten turned out to be scenes belonging to two broad categories of emotion: tenderness or suspense. Because the system was not taught to be a movie editor, a human editor was brought in to finish the trailer. The human editor ordered the segments suggested by Watson and also added titles and music. [see reference below for additional details]
Here’s the trailer that resulted, along with some explanations of how it was done (direct link to video):
As you saw, the end result looks and sounds like a typical movie trailer. The big question is if this cognitive movie trailer does what a good trailer should: make us want to see the movie.
If you like science fiction films that explore questions about human engineering or artificial intelligence, then this trailer might appeal. The trailer does convey through the ten selected scenes that Morgan is an engineered creation that goes rogue—a story we’ve heard before. However, we are left in the dark about what exactly Morgan’s problem is (other than being locked up) and how the humans will deal with it. Many trailers fail by showing too much of the story. For example, the official Morgan trailer shows a lot more of the movie, which made the story sound similar to another movie, Ex Machina (an engineered human-like entity is confined in a futuristic laboratory, tested for flaws, goes amok, kills or maims one or more people, and escapes into the world). But by limiting what’s revealed, the Watson-enhanced trailer makes us think that maybe this story will differ from previous movies and be worth seeing.
I thought the computer-selected segments were interesting in that they not only conveyed a range of emotions (happiness, tenderness, suspense, fear), but many did so in a subtle way (a smile, a hand gesture, a slight gasp, a head turn). No scenes seemed to be selected from the latter part of the movie, which would have given too much of the story away. I don’t know if this was a result of the Watson system ranking scenes near the end lower than those from the beginning and middle.
In the end, I think the Watson-enhanced trailer is pretty good and perhaps better in some ways than the official trailer created entirely by a human.
For more information about the making of the Morgan movie trailer, see this article: Smith, J.R. 2016. IBM research takes Watson to Hollywood with first “cognitive movie trailer”. Think <https://www.ibm.com/blogs/think/2016/08/31/cognitive-movie-trailer/>
This post is part of a series about Artificial Intelligence (AI) and its potential role in science communication. In the next post (part 4), I’ll talk a bit about what AI means for human creativity.