Tag Archives: Morgan

What Does Artificial Intelligence Say About Human Creativity?

Posted on by

pexels-photo-largeThis post is part of a series about Artificial Intelligence (AI) and its potential role in science communication. In this post (part 4), I talk about creativity and how it relates to AI.

In the previous posts, I’ve been talking about the computer Watson and how it helped create a trailer for the movie Morgan. Is this “cognitive movie trailer” evidence of AI creativity or the potential to mimic human creativity? In other words, can a human be replaced by a machine—in this case a trailer editor who uses skill and imagination to create something new?

Let’s first consider what creativity is. The dictionary defines creativity as the ability to make new things or think of new ideas. But is it a trait only exhibited by humans? Is it an attribute that some people have and others don’t? Is it an occasional mental state that we enter? Can one learn to be more creative? I’m not sure of the answers to all these questions, but perhaps it’s more helpful to ask what creativity is not. It’s not problem solving, which is a process whereby a “rule” or “algorithm” is applied to solve a problem. Being able to understand and apply a rule is different from discovering the rule.

In the case of the computer Watson, we can see that understanding what a movie trailer is and identifying the best scenes from the movie Morgan to use fall into the realm of problem solving and not creativity. A human stepped in to do the actual film editing, which additionally suggests that the “creative” aspect of putting together the trailer could only be done by a person with the requisite editing skills and imagination to sequence the clips and add other components such as music and text. However, I don’t think a human was essential to do the editing, once the scenes were selected.

A movie trailer template could have provided a guide with plascreenshot_imovie13ceholders for media and text, much the way iMovie trailers are created. In this screenshot, you can see an iMovie trailer template, which guides the choice of video clips and text. Scenes are suggested, as are text titles that form a story. Such a template could have been used along with the ten selected scenes from Morgan to produce a finished trailer. However, such an ability by an AI could not be called creative. Although some decision-making would be involved in selecting which scene to go into each placeholder, those steps would be guided by a set of rules—in other words, problem-solving, not creativity. Also, templates would produce an assembly-line of movie trailers that all follow the same format—rather than a unique trailer with sequences, pacing, music, and other features individually selected by the editor using his or her knowledge, skill, and imagination.

I think we are a long way from machines that think and create like humans. However, we are at a point where AI can be used to enhance human skills and help us perform tasks involving vast amounts of information. Artificial intelligence systems are already at work aiding, for example, analysis of medical images, detection of suspicious charges to our credit cards, or automated telephone customer service. The real question is not whether AI can replicate human thinking or creativity but how AI can help humans create new things or think of new ideas faster and more efficiently.

This post is part of a series about Artificial Intelligence (AI) and its potential role in science communication. In the next and final post (part 5), I’ll discuss how AI might help scientists be better communicators.

How Did Artificial Intelligence (AI) Help Create a Movie Trailer?

Posted on by

This post is part of a series about Artificial Intelligence (AI) and its potential role in science communication. In this third post, I describe how the computer, Watson, helped create a movie trailer.

Before we get to the Watson movie trailer, let’s first think about how movie trailers are made. Movie trailers are designed to convince people to go see a particular movie. Superficially, trailers appear to be a condensed version of the film, but good trailers are carefully designed to raise expectations and to appeal to the viewer’s emotions. Most trailers follow a typical formula, modified for the genre such as Action/Adventure, Comedy, Drama/Thriller, or Horror. Many trailers begin by introducing the characters and the setting of the film. Next to appear are the obstacles that change that world and set the characters on a new course. This may be followed by increasingly exciting, funny, or tension-filled scenes to ramp up the viewer’s desire to find out what happens. The specifics—selection of clips, the way they are cut (rapid-fire or slow-reveal), the fonts used for text titles, narration, music, and other choices—differ among movie genres.

All, however, are built more or less the same way by the trailer editor. The original movie is first watched carefully and deconstructed to reveal its basic components, visual and audio. The process then slices the movie audio and video further into segments that can then be rearranged to build the trailer. Next comes the choice of the best elements to use. Is the acting superb? The cinematography? The story? Editors often select those elements that highlight the merits of the film or the ones that have the most emotional impact on a viewer.

Not surprisingly, the AI-enhanced trailer of the movie Morgan was created in much the same way as a regular trailer. The first step, however, was to train Watson to understand what a movie trailer is and what features of a movie are used in movie trailers. The IBM team did this through machine learning and Watson APIs (Application Programming Interfaces, i.e., programming instructions). Basically, each of 100 movie trailers was dissected into component scenes, which were then subjected to the following analysis: (1) Visual (identification of people, objects, and environment), (2) Audio (narrator and character voices, music), and (3) Composition (scene location, framing, lighting). Each scene was tagged with one of 24 emotions (based on visual and audio analysis) and further categorized at to type of shot and other features.

Once Watson was trained, it was fed the full-length movie, Morgan. Based on its knowledge of what makes up a movie trailer—particularly a suspenseful one, Watson then selected ten segments as the best candidates for a trailer. These ten turned out to be scenes belonging to two broad categories of emotion: tenderness or suspense. Because the system was not taught to be a movie editor, a human editor was brought in to finish the trailer. The human editor ordered the segments suggested by Watson and also added titles and music. [see reference below for additional details]

Here’s the trailer that resulted, along with some explanations of how it was done (direct link to video):

As you saw, the end result looks and sounds like a typical movie trailer. The big question is if this cognitive movie trailer does what a good trailer should: make us want to see the movie.

If you like science fiction films that explore questions about human engineering or artificial intelligence, then this trailer might appeal. The trailer does convey through the ten selected scenes that Morgan is an engineered creation that goes rogue—a story we’ve heard before. However, we are left in the dark about what exactly Morgan’s problem is (other than being locked up) and how the humans will deal with it. Many trailers fail by showing too much of the story. For example, the official Morgan trailer shows a lot more of the movie, which made the story sound similar to another movie, Ex Machina (an engineered human-like entity is confined in a futuristic laboratory, tested for flaws, goes amok, kills or maims one or more people, and escapes into the world). But by limiting what’s revealed, the Watson-enhanced trailer makes us think that maybe this story will differ from previous movies and be worth seeing.

I thought the computer-selected segments were interesting in that they not only conveyed a range of emotions (happiness, tenderness, suspense, fear), but many did so in a subtle way (a smile, a hand gesture, a slight gasp, a head turn). No scenes seemed to be selected from the latter part of the movie, which would have given too much of the story away. I don’t know if this was a result of the Watson system ranking scenes near the end lower than those from the beginning and middle.

In the end, I think the Watson-enhanced trailer is pretty good and perhaps better in some ways than the official trailer created entirely by a human.

For more information about the making of the Morgan movie trailer, see this article: Smith, J.R. 2016. IBM research takes Watson to Hollywood with first “cognitive movie trailer”. Think <https://www.ibm.com/blogs/think/2016/08/31/cognitive-movie-trailer/>

This post is part of a series about Artificial Intelligence (AI) and its potential role in science communication. In the next post (part 4), I’ll talk a bit about what AI means for human creativity.

What is Watson and What Does It Have to Do with Videos?

Posted on by

This post is part of a series about Artificial Intelligence (AI) and its potential role in science communication. In this second post (part 2), I describe Watson, a computer that was trained to assist in the making of a movie trailer.

artificial-intelligence-elon-musk-hawkingIn the previous post (part 1), I explained that IBM’s computer system, Watson, was used to help a Hollywood film studio make a trailer for the movie, Morgan. But what is Watson? According to the IBM website, Watson is “a technology platform that uses natural language processing and machine learning to reveal insights from large amounts of unstructured data”. Translating that into everyday language: Watson is a computer that can answer tricky questions like the ones posed on the gameshow Jeopardy!. In 2011, Watson beat two reigning champions, providing answers to Jeopardy! clues—example: even a broken one of these on your wall is right twice a day; correct reply: what is a clock?—and winning $1,000,000 (which was donated to two charities).

Actually, Watson is a cluster of computers (90 servers and 2880 processor cores) running something called DeepQA software. Despite its performance on Jeopardy!, Watson does not “think” like a human and arrives at an answer to a question differently. Tons of information from various sources have been input, providing Watson with an enormous information base to analyze. For the game show, Watson used more than 100 algorithms to come up with a set of reasonable answers to a question. It then ranked those answers and searched its information database for any evidence in support of each answer. The answer with the most evidence was given the highest confidence. When the confidence was not high enough during the Jeopardy! game, though, Watson did not risk losing money by offering an answer.

Despite fears that AI will eliminate jobs or go rogue and destroy humankind, as depicted in the Terminator series, the system is viewed by developers as a way to augment human intelligence and to reduce the time spent on tasks involving large amounts of information. IBM prefers the term Augmented Intelligence (systems that enhance and scale human intelligence) to Artificial Intelligence (systems that replicate human intelligence). There are many ways in which AI can augment information-intensive fields such as medicine, telecommunications, weather forecasting, and financial services. Since the Jeopardy! match, Watson has been used to create cognitive apps and computing tools for businesses and healthcare professionals.

It’s not difficult, then, to imagine AI systems aiding scientific research and especially the communication of those findings in a more efficient way. More and more people are getting their information, particularly about science, in the form of video, but many science professionals have little time or incentive to devote to learning and using new communication tools. A system that can reduce the time involved in making a video and simultaneously enhance the quality could greatly improve communication of science and its importance to society. The first cognitive movie trailer, aided by the computer, Watson, is a “proof of concept” in this regard.

For more information about Watson and preparation for the Jeopardy! gameshow, see this article: Ferrucci, D. et al. 2010. Building Watson: An overview of the DeepQA process. Association for the Advancement of Artificial Intelligence pp. 59-79.

This post is part of a series about Artificial Intelligence (AI) and its potential role in science communication. In the next post (part 3), I’ll describe how Watson helped create a movie trailer.

Science Communication, Artificial Intelligence, and Hollywood

Posted on by

This is the first post in a series about Artificial Intelligence (AI) and how it might help scientists be better communicators. In this post, I introduce the topic.

Consider this futuristic scenario:

4246476627_f40c638984_oA scientist is working on a grant proposal and must create a three-minute video synopsis of what she plans to do with the funding and how her research will benefit society. This video synopsis is one of the required components of proposals submitted to government funding agencies. She logs onto a platform in the Cloud and uploads video clips showing her and her team working in the laboratory and talking on camera about the potential applications of the proposed research. An AI (Artificial Intelligence) system analyzes all of the uploaded information, as well as millions of images, animations, and video clips in the public domain. Within minutes, the AI system has identified the key components necessary to address the stated goals of the funding opportunity and has produced a draft video of the required length that is both intellectually and emotionally stimulating. The scientist takes the draft video file and makes a few edits based on her knowledge of the field and potential reviewers. She renders the final video and attaches it to her application package, which she submits to the funding agency. Her proposal is funded, and the funding agency uses her video synopsis on their website to inform the public about the research they are supporting and how it may affect them.

Far-fetched? Perhaps not. Recently, I was watching an episode of GPS in which Fareed Zakaria interviewed the CEO of IBM, Ginni Rometti, and my ears perked up when they talked about an AI helping a film editor cut a movie trailer, reducing the time required from weeks to a day. The movie studio, 20th Century Fox, recently collaborated with IBM Research and its computer Watson to produce the first computer-generated movie trailer for the science fiction film Morgan, which is about, appropriately enough, an artificially enhanced human.

Watson was trained to “understand” what movie trailers are and then to select key scenes from the full-length movie to create a trailer that would appeal to movie-goers. A similar approach could be applied to scientific information to produce a video proposal that resonates with peer reviewers and panelists, as in the hypothetical example above….or a video abstract to inform the scientific community about a recent journal article. The idea here is that a busy scientist may one day be able to use AI to rapidly scan a vast storehouse of data—much faster and more thoroughly than a human—and then to suggest the best material and design for an information product such as a video.

AI is being considered as a way to enhance many activities involving the analysis of large amounts of data—such as in the medical or legal fields. Using AI to create movie trailers or science videos may seem to be a trivial goal compared to making a more accurate medical diagnosis; however, when you consider how important it is for science professionals to be good communicators, the idea seems worthwhile. In the coming posts, I’ll explore this topic further and provide a bit more detail about how IBM’s Watson was used to create a movie trailer.

This post is part of a series about Artificial Intelligence (AI) and its potential role in science communication. In the next post (part 2), I’ll provide more information about Watson, the computer.