Creating hit programs at Netflix and Amazon: It’s not just about data

By | January 12, 2016

wernickeThere is a newly released TED talk on ted.com titled “How to use data to make a hit TV show.” The speaker is Sebastian Wernicke, a data scientist (and former animator) who makes an interesting comparison about the rival political dramas House of Cards (Netflix) and Alpha House (Amazon). The video, which was shot last year at TEDx Cambridge, is embedded below. I am really interested in what Wernicke had to say, as it touches on how audience viewing data can be used in different ways to shape video programming. There is a lean media angle here, and I wanted to see how big data was being used by these companies.

Wernicke’s hypothesis goes something like this: The two streaming video giants are using big data in different ways to identify factors that will make a hit TV show (as measured on the IMDB audience ratings curve). The companies will do things such as measuring how long certain demographic groups watch certain episodes of certain shows with certain characteristics, where they are mostly likely to abandon a series, etc. But their approaches diverge, as described in this Observer article:

A look at exactly how the two companies used the data provides good insight. When Amazon set out to make a data-driven show, the company held a competition. They evaluated a bunch of show ideas, selected eight of them and then created a pilot episode for each and made them available online for free. Millions watched the free episodes, and the company used data (such as how many people watched each show, how long they watched and what parts they skipped) to create a show they hoped would be destined for greatness. After crunching millions of data points, the results said they should create a sitcom about four Republican U.S. senators. Alpha House was born.

Around the same time, Netflix was brewing up something similar. But instead of using a competition, the company looked at the data they already had about viewing on their platform (ratings, viewing history, etc). They used that data to discover small bits and pieces about what viewers like and took a leap of faith to create a drama about a single U.S. senator.

Wernicke suggests that the different approaches explained why one of the shows (House of Cards) rated 9+ on the IMDB scale, while Alpha House barely broke the average (~7.6) and has struggled. He said:

“Whenever you are solving a complex problem, you are essentially doing two things. The first is you are taking the problem apart into its bits and pieces so you can deeply analyze those bits and pieces. Then you put them back together again so you can come to your conclusion. …

The crucial thing is data and data analysis is only good for the first part. … it’s not suited to put those pieces back together again and then to come to a conclusion. There’s another tool to do that and we all have it. And that tool is the brain … especially if it’s the brain of an expert.

That’s why I believe Netflix was so successful. They used data and brains where they belong in the process. They used data to understand things about their audience that they otherwise would not have been able to understand at that depth. But then the decision to take all of these bits and pieces and put them back together again and make a show like House of Cards, that was nowhere in the data. Ted Serandos and his team made that decision to license that show. Which also means they were taking a very big personal risk with that decision.

And Amazon on the other hand, they did it the wrong way around. They used data all the way around to drive their decision-making. First they held their competition of TV ideas, then when they selected Alpha House to make as a show. Which of course was a very safe decision for them because they could always point at the data and say, ‘this is what the data tells us.’ But it didn’t lead to the exceptional results that they were hoping for.

So data is of course a massively useful tool to make better decisions, but I believe things go wrong when data is starting to drive those decisions. No matter how powerful, data is just a tool.”

I am conflicted about Wernicke’s presentation. I agree that data is a tool, and is suited to analytical tasks. And decision-making driven purely by data can go wrong in so many ways. The once high-flying social games producer Zynga is one example of a company that relied too heavily on data (“What ended up happening is people were maybe exceptionally focused on the data and didn’t spend enough time looking at the qualitative gameplay”) before falling back to earth.

The problem I have is not with Wernicke’s conclusions about data analysis and its place in media development, but rather the examples he uses. In my opinion, the success/failure of Netflix and Amazon television programming can’t only be explained by the two data-driven approaches used during the development process. There were also creative decisions taking place, even at Amazon, ranging from casting to scripting to the shoots. Then there was the marketing and positioning of the shows, not to mention other factors–demographic differences between the Netflix and Amazon audiences, the user interface, the fact that Netflix has a different recommendation algorithm, etc. How did the creative decisions and other factors affect the ratings of the two programs?

Let’s look at the problem from a counterfactual point of view. What if Kevin Spacey had not been cast as the lead in House of Cards? What if Alpha House had a better writing team? What if one of the shows had been launched in a different month? Surely the ratings would be different. If Amazon topped Netflix, would that invalidate Netflix’s approach to using data?

Another angle: If the Amazon approach is so bad, why is Man In The High Castle (which is also the product of a programming bake-off) so good? Conversely, why have some Netflix shows stumbled? Ted Serandos gave the green light to Marco Polo, but it has been savaged by critics and its IMDB rating is 8.1 — above average, but not in the same league as House of Cards.

In summary, I agree with Wernicke’s ideas about the role of data in the production of media content. But the examples used in his presentation leave out a lot of details relating to the creative decisions that were made at both companies. I hope Wernicke can address some of these factors in a future video (or maybe he did in his original presentation, as the segment below is only 12 minutes long).

Feel free to leave your comments below.

Sebastian Wernicke: How to use data to make a hit TV show

Help the author spread the word!Share on Facebook0Share on Google+0Tweet about this on TwitterShare on LinkedIn0Email this to someone

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload the CAPTCHA.