Do Fictional Stories Flow Better Than True Stories?

What makes a story “flow”? We often use the term casually to describe a narrative where one event seems to lead naturally to the next. But can that feeling be measured computationally?

In our new ACL paper, “Fiction Flows: A Replication and Reinterpretation of Narrative Sequentiality,” co-authored with Sil Hamilton, Haiqi Zhou, and Federico Pianzola, we revisit an influential claim in computational narrative research: that imagined stories exhibit more narrative flow than recalled autobiographical stories. The original idea was that imagined stories draw more heavily on shared cultural scripts — familiar patterns of how events unfold — while memories contain more idiosyncratic details that interrupt predictability. 

To test this, we replicated the original measure of “sequentiality,” which uses language models to estimate how much easier it is to predict the next sentence of a story once the prior context is known. We used the original data which came from an online experiment where people were asked to tell a story, either from memory or when given a “summary” produced by someone else’s memory. The problem with the original measure is that it includes the story summary in its calculations. The resulting score can thus reflect not only how well a sentence follows from the prior context, but also how much it diverges from the summary. In other words, imagined stories may appear to “flow” better not because their events are more sequential, but because they move farther away from the topic prompt than recalled stories do.

When you remove the summary guess what happens? The result disappears.  

This suggests that the earlier finding may not have been measuring narrative flow so much as the mismatch between a story and its summary, i.e. effects of the online experiment.

The story changes, however, when we move to published books away from crowd-worker produced mini-stories. Using passages from contemporary fiction and non-fiction, we find that fiction does show a consistent sequentiality advantage over comparable non-fiction genres. Fictional sentences are often more surprising in isolation, but they benefit more from narrative context: once you know what has come before, the next sentence becomes more predictable.

Now we have a new problem: is LLM-based sequentiality indeed measuring “flow,” or something else? When we tested a suite of measures to capture traditional understandings of discourse sequentiality we found little correlation between the LLM scores and those more traditional metrics.

This leaves us with a few insights:

•  Replication matters. The original study simply does not replicate under stricter conditions. Even when an effect reproduces, its interpretation may change once we examine the assumptions built into the measurement.
• Fiction appears more sequential. From the point of view of language models, fictional sentences look more sequential. This tells us a lot about the way fictional storytelling embraces simplicity and predictability, something I talk a lot about in my new book.
• We still don’t know what LLM’s are measuring when we measure sequentiality. Stay tuned for future work that tries to get to the bottom of this!