Aristotle, ancient Greek philosopher and Alexander the Great's tutor, was a huge fan of theater and author of the Poetics, a critical work that has also long served as a guide for playwrights (at least those interested in making Aristotle's sort of play); in his work, the philosopher and amateur critic discussed what he felt were the essential elements of any successful drama. Firstly, for Aristotle, the highest form of drama was the tragedy, and by tragedy he meant:

...the imitation of an action that is serious and also, as having magnitude, complete in itself; in appropriate and pleasurable language;...in a dramatic rather than narrative form; with incidents arousing pity and fear, wherewith to accomplish a catharsis of these emotions.


Foundations of a twist

The concepts of peripeteia and discoveryare the ancient foundations of the twist as we know it today. Peripeteia is the change of a character's situation from one state to or another, one that comes from out of the blue, and is unexpected. Conversely, with discovery, what happens to the character is a change from ignorance to knowledge. That is, and in our case, the character comes to an apprehension of how the they are the cause of the situation they find themselves in.

Today, the idea of the twist has changed, but a modern film with a particularly Aristotelian twist is the book and film adaptation of Psychoin which Norman Bates, the murderer, is actually the protagonist. He has no idea that he is the one behind the murders of Marion Crane, Arbogast, or his attempted murder of Lila. Norman, has, until then, been, ostensibly covering for his mother, though the audience is pretty sure that there is not a senior citizen running around with a butcher knife. 

The crucial thing is that Norman doesn't know. In order to cope with what he's done, he has split his personality so that his mother is separate from him. It's only at the end that his world begins to break down. 

Ingredients for a twist

Hitchcock (and novelist Robert Bloch) made use of a particular type of twist, though there are several others which take their inspiration from Aristotle but have a distinctly modern sensibility. Each, though, is dependent on ignorance. Either ours, or the characters.

A big thing to keep in mind is this: a twist, like any ending, must be inevitable and unexpected. That is, even though the audience is surprised by it, they realize that things couldn't have been any other way. And, in order to pull this off successfully, clever placed breadcrumbs in the form of foreshadowing have to have been placed throughout the story, so that looking back, it becomes obvious that we were being led around by the nose towards a foregone conclusion, and so when the ending does arrive, the audience will have a certain sense of satisfaction, having been subconsciously following your foreshadowing the whole time, their conscious minds too distracted by the events.

Here's a list of just a few ways to pull the rug out from under your audience:

Change a character's identity

In Fight ClubTyler Durden is revealed to not exist, to be, like Mrs. Bates for Norman, a figment of the protagonist's imagination. Except, here we are in the head of Norman, as it were, instead of watching the story from the relatively objective, outside perspectives of, respectively, Marion, Arbogast, and Lila. In Fight Club, we viscerally experience the dissociation that Psycho uses to terrify. But, every story is unique, in the same way that every story has been told. That's deep. 

Reveal a change in motivation

In Se7enfor instance, the killer is, in the last half hour, revealed to have a completely other reason for leading Detectives Somerset and Mills out to the desert; instead of turning himself in from a guilty conscience and desire to reveal more victims, he is planning an elaborate trap.

Scales fall from character's eyes

In this sort of twist, the character realizes that not one thing, but everything, their entire worldview, has been changed by some revelation: the fabric of reality itself has been somehow altered. But this can entail no more than a shift within the character's social group -- whatever constitutes their world -- in this sort of ending (and all these endings feature elements of either peripeteia and/or discovery.) See: The Matrix (and lots and lots of other sci-fi).

You get the idea. And twists don't have to come at the end of movies, either. Sometimes it's better to have a series of small twists that lead inexorably towards one giant one, and other times it should come completely out of nowhere to turn the character's life upside down. And sometimes, as in Oedipus Rex, Oedipus sets in motion the events which lead to his discovery -- the prophecy which has stricken his city with plague concerns Oedipus himself, who unwittingly fulfilled the prophecy by killing his father (whom he had not met and did not recognize) and marrying his mother (ditto).

Aristotle considered this to be the most tragic story for the fact that Oedipus himself was a king brought low by his own character, the same character that had caused him to murder his father so many years before, setting the tragic plot into motion. Because remember, action is character.