The Best Stories Deliver What You Don't Expect

The Souldancer 10th Anniversary campaign is counting down its last six days on Kickstarter. If you've been waiting to claim the Special Edition of Souldancer or the exclusive hardcover of the new bridge novel The Other Side of Fear, now is the time to pledge before it's too late.

A few years back, “subverting expectations” was all the rage in writing circles. The idea was that readers who've consumed thousands of stories know all the tropes and can tell where a plot is going long before the characters catch up. So the solution is to surprise them.

Unfortunately, that simple advice has produced some of history's worst storytelling.

How many times have you read an oldpub story from the past quarter century in which the hero suddenly picks up the Idiot Ball? Or the villain's last-minute win comes out of left field because the writer withheld crucial information? Or a sequel devotes the first act to nullifying the first installment's hard-earned victory?

True, those stories did subvert the audience's expectations, in that they expected a satisfying story.

Here's what Gen Y oldpub writers missed because they were, for the most part, misanthropic hacks that hated their readers: The best surprises come not from proving the audience wrong, but from proving them right for the wrong reason.

What does that mean? 

Suppose you're reading a murder mystery. For 250 pages, every clue points toward three suspects. Each had motive, means, and opportunity. The inspector slowly eliminates possibilities until the solution feels close enough to touch.

Then, in chapter twenty-five, the detective announces that the killer was Bob.

Who is Bob, you ask?

The victim's secret identical twin, naturally. He has never appeared before, nobody mentioned him, and nothing in the previous 250 pages so much as implied his existence.

You didn't see that one coming!

Did the author surprise you? Yes, he did; he also wasted your time.

The dirty secret no one admits is that surprise by itself adds nothing. Anyone can produce an unexpected event by withholding information, violating established rules, or making a character act irrationally.

An effective revelation achieves the much more difficult feat of changing what readers thought they knew.

Think about the great revelations you've been treated to in books, movies, and games. It's a good bet that the ones which stuck with you provoked two reactions in rapid succession:

“What?”

Followed immediately by: “Of course!”

Therein lies the butter zone.

The writer showed you the evidence. You saw the clues. You catalogued every relevant fact.

But you misunderstood what those facts meant.

If the hero's former nemesis really did pull a face turn and join forces with him, but for reasons nobody suspected, or the seemingly out-of-place detail from Chapter Three actually explained everything after all, the reader's interest in the big reveal is magnified.

To break it down even more directly:

So the litmus test is whether or not a big revelation uncovers an internally consistent story behind the story.

To understand what that means, imagine reading a story twice. On the first pass, the writer gets you looking in a certain direction, so you gloss over snatches of dialogue, background details, and slightly odd character behavior.

Then you get to the reveal.

And instead of closing the book, start the story over.

Now you understand that character's pause before answering, a key character's absence from a crucial scene, or a piece of apparent window dressing that kept turning up.

A strong twist therefore produces two different stories from the same text.

The first depends on what the audience assumes; the second depends on what the author actually wrote.

That's why some stories get better after you know the ending.

A cheap twist has the opposite problem: Once the shock is gone, you've got nothing left.

That's why audience expectations are best thought of as tools, not obstacles.

Genres offer writers a vast storehouse of assumptions. Put a lone gunslinger in a dusty frontier town, or an eccentric detective and a corpse in a locked room, and readers instantly understand half the story's plot and all of its promises.

Those expectations serve the author as narrative shorthand.

More importantly, they give you strings to pull.

For example, if readers expect the mysterious old wizard to become the hero's mentor, you can use that assumption to hide another role in plain sight.

The writer doesn't need to lie; readers will gladly mislead themselves. And when you finally reveal the truth, no one can say it's not fair, because the story never broke its promises. The deception was all on the part of the audience, who mistook familiarity for certainty.

And unlike a crude rug-pull, it leaves the reader satisfied.

Writers sometimes obsess over avoiding predictability to the point of forgetting why readers form expectations to begin with.

Expectations are evidence that communication is taking place. If readers expect the hero to confront the villain, odds are you've established a conflict worth resolving. 

Much like splitting atoms, breaking expectations can unleash a lot of energy. The catch is, you've got to provide an even better alternative payoff. Otherwise, all you've done is trade satisfaction for novelty.

And novelty has a limited shelf life.

That's the real problem with the directive to “subverting expectations.” It focuses the writer's attention on defeating the audience. Great storytellers pursue a far more difficult but rewarding goal. They teach the audience what to expect, let them draw the obvious conclusion, then reveal that they had all the pieces of a puzzle they put together wrong.

So showing them the original picture printed on the box brings not frustration, but cathartic relief.

When I wrote my second novel Souldancer, I knew readers would bring assumptions from Nethereal. So I used those assumptions and gradually revealed the larger story that stands behind the readers' received understandings.

That gambit paid off. The novel went on to win the first Dragon Award for Best Horror Novel, and ten years later, readers are still getting hooked on it.

Now Souldancer is back in a definitive 10th Anniversary edition, alongside the new Soul Cycle novella The Other Side of Fear. And the project has entered its final six days on Kickstarter.

If you've been waiting to join the campaign, now is the time to back the Souldancer 10th Anniversary Edition and help propel the Soul Cycle into its second decade in style.

I'm also making this week's regular post public for the campaign's final stretch. Posts like this are normally reserved for paid members. If you'd like to keep getting them after the campaign ends, you can upgrade to a paid membership. 

And readers who want The Other Side of Fear as it's being serialized can upgrade to Margrave or higher for the weekly preview chapters.