David Simon and Nate Silver on Big Data and The Future of Storytelling

In 2008, Nate Silver became a household name when his website FiveThirtyEight delivered eerily accurate predictions about the outcome of the presidential race. His results flummoxed traditional pollsters and analysts because his results came from exhaustive analysis of data on every possible "metric" related to voter behavior; suddenly, traditional opinion polling was, if not obsolete, highly suspect. 6 years later, his website is owned by Disney, and big data is the word on everyone in Hollywood's lips. Yesterday at the Tribeca Film Festival, Silver, House of Cards creator Beau Willimon, The Wire and Treme's David Simon, veteran journalist Anne Thompson and moderator John Hockenberry sat down for a conversation about if storytelling may become a matter for statisticians rather than screenwriters. Click through to hear what they had to say!

Today, a new category of media consumer is emerging: the binge viewer. Rather than wait every week for their favorite show, this subgroup of viewers (which contains lots of people I know) instead waits until they have a solid week to devote to watching an entire season, or even series. With Netflix's original series House of Cards, what was once an anecdotal phenomenon is now a behavior that is actively catered to: House of Cards is binge viewing by design, with the entire second season released in one day.

As technology improves, so does data gathering technology, and on sites like Netflix, subscribers actively self-report their preferences; while this is designed to enhance their viewing experience, it could also be argued that these consumers are, in essence, working for Netflix without compensation. It was information gathered via this self-reporting that lead the company to discover that 10% of Sopranos viewers also liked Two and a Half Men. This correlation led the company to put the wildly popular Charlie Sheen sitcom in the suggested column for fans of The Sopranos. As it turned out, the number was more like 20%, and Netflix made some more money (as did struggling actor Charlie Sheen).

This anecdote is indicative of the so-called correlative effects of big data; with petabytes of information flying around the world 24 hours a day, people in the civilized world are tracked and quantified to a greater degree than ever, and more so everyday. It doesn't take much to realize that the reason a site like Facebook is free isn't because Mark Zuckerberg is a nice guy (he may be, though that movie made him look kinda creepy). It's free because in exchange for the social networking you enjoy with your friends (sharing photos, tending digital gardens on Farmville, etc.), you are providing Facebook with data about yourself, your consumer habits, favorite movies, music, and TV shows. And almost everyone with a Gmail account has had the creepy experience of finding their ads just a little too personalized.

Where data was once causal, i.e., consisting mostly of observations of a phenomenon, followed by a search for rational causes, we are now entering a correlative age, where data is swept up and then run through complex algorithms that determine probable future behavior. There is so much information, humans are unable to process it. And this torrent of information is big business, so big, in fact, that just a few days ago, Amazon signed a deal with HBO for exclusive streaming rights to their programming. According to Forbes:

[Amazon] has also struggled to come up with compelling original shows. The site uses the clever tactic of having viewers vote on pilots to decide which shows will get picked up as series. In theory, that should create a lot of fan loyalty and get viewers to become evangelists for the shows that ultimately make it to the streaming service. But so far, Amazon has failed to produce anything buzz-worthy. Most people probably can’t even name one Amazon original show.

Asked whether or not the idea of the opinions of millions of viewers changed the way he approached to the story of Kevin Spacey's Washington insider, Willimon said that, to him, the fact that he could plan a full two seasons only meant a greater degree of creative control; without worrying that he would be cancelled from week to week, he had the freedom to plant easter eggs in the first hours and not come round to them until the end of the season.

Both Willimon and Simon were adamant that the new cable and streaming model is far superior to the TV industry in which they both toiled for years. Simon, who also worked as a journalist in Baltimore for years, said half-jokingly that he "doesn't care at all" about what the audience wants, comparing them to a child who wants more candy, or, in this case, MORE OMAR:

Video is no longer available: www.youtube.com/watch?v=h2us0OaFUR8

Simon also made the point that The Wire happened to come along at just the right time, when HBO was starting its original programming and was willing to spend $24 million on a season of The Wire. Today, he doubts he could convince them to greenlight the show, saying he has failed, for the most part, over 110 hours of Sunday night programming. He also revealed that for 9 years he has been working on a project with Ed Burns, a secret history of the C.I.A. But, he was quick to acknowledge that it would be prohibitively expensive, with CGI and locations around the world. Anne Thompson suggested he could try casting movie stars, since TV is no longer sitting at the kid's table.

Despite the craze for this newest version of the Nielson Ratings, Silver was quick to observe that, "Statistics are history." Even though new algorithms can, to a degree, predict what someone will want to watch, and conceivably allow a development executive to realize that some extremely niche demographic is being underserved (and then fill that void with content), the situation is still the same as the caveat at the end of every investment commercial: "Past results are no guarantee of future results. "

Storytelling is a human art, and humans are anything but rational; for the time being, at least, big data will not be dictating what you binge view while home sick with the flu. Admittedly, the creators of The Wire and House of Cards are in an enviable position, but the takeaway, for indie filmmakers, is that story is still king, and, as Simon said, quoting screenwriter William Goldman's famous adage about Hollywood: "Nobody knows anything!" 

Do you think Netflix asking for your favorite movies is thoughtful, or intrusive? As an independent filmmaker, how beholden do you feel to the market? Every filmmaker knows that rom-coms gross more than art films (usually), but has this ever given you pause, or caused you to change the way you told a story? Or do you think big data is nothing but hot air? Let us know, in the comments! 

Link: Tribeca Film Festival

You Might Also Like

Your Comment


First off, I've been told that story is different than story. Audience genre preference changes but the story is a much more deeper thing. More of what your trying to say like theme. Second I think that as soon as you see a trend(unless it's a personal prediction, some people have that ability) then it's already too late to be part of the trend because it will soon be over anyways.

April 25, 2014 at 7:01PM, Edited September 4, 8:56AM


Nicely written piece. Oh, to be a fly in that room.

April 25, 2014 at 10:24PM, Edited September 4, 8:56AM

Carlos D

I could be misunderstanding, but what happens to the notion of discovering a series or genre you would not otherwise know you would like until you saw it ? Does limiting margins by statistics kill the creativity to do the unexpected ?

April 26, 2014 at 7:33AM, Edited September 4, 8:56AM


a very good point

April 26, 2014 at 3:27PM, Edited September 4, 8:56AM


Reaching eyeballs has always been the mountain that follows the marathon of making an independent IP/film. My hope is by exploring/consulting data science before the climb, that I and my team will be able to identify better/more efficacious route(s) to the top and thus, use our limited resources as wisely as possible.

April 27, 2014 at 5:37PM, Edited September 4, 8:56AM


I think that CONTENT that COMPELS is king and always has been. I can be broken down into genre and type etc yes but the actual stickiness of the storyline and plot it's self is the winner. Who does NOT look at a terrible accident on the side of the road...humans WANT to know the ANSWER to QUESTIONS. A good story gives them a QUESTION and reveals an ANSWER at the end. If very simplistic terms could sum up a "sticky" story it would be give them a burning question and don't answer it till the last page/frame. Just watch JJ at this. he knows the secret all too well.

April 28, 2014 at 9:49AM, Edited September 4, 8:56AM


The prediction value of a model based on big data for story telling to me is absurd. The value is in the relations between data points, very much like the relation between visual identity and your shot list, which aggregate to a cine style in the long run. Facebook is not in the prediction business, nor should film makers in my sense.

May 5, 2014 at 8:17AM, Edited September 4, 8:56AM


Thanks for this

July 25, 2020 at 3:07AM