KathaTrace: Diagnosing Semantic Trajectory Collapse in Generated Visual Narratives

Jamuna S. Murthy1, Amin Karimi Monsefi2, Rajiv Ramnath2
1Ramaiah Institute of Technology, 2The Ohio State University
Semantic trajectory collapse: a faithful storyboard versus a collapsed storyboard that looks coherent but loses transition meaning.

KathaTrace diagnoses storyboards that look visually coherent while the semantic meaning of how one scene leads to the next quietly disappears.

Central idea. Visual quality and scene coherence do not guarantee that a viewer can recover why one scene leads to the next. KathaTrace isolates this failure mode—semantic trajectory collapse—by testing transition recoverability under text-only, image-only, and text+image evidence, and quantifies the loss as the Semantic Trajectory Gap (STG).

Paper KathaBench-25K Code Demo

Abstract

Visual narratives are central to storyboards, comics, children's media and film previsualization, where viewers understand stories from images alone. Recent generators like StoryDiffusion produce coherent sequences, but visual coherence does not guarantee that source-story transition meaning remains recoverable. Existing benchmarks assess visual quality, content faithfulness, and scene coherence, but miss a critical failure mode: storyboards where scenes appear visually coherent while the semantic link between scenes disappears.

We introduce KathaTrace, a generator-agnostic protocol for diagnosing semantic trajectory collapse—loss of transition meaning needed to understand how one scene follows another. KathaTrace evaluates transitions under three evidence conditions (text-only, image-only, text+image) and filters ambiguous items. We contribute KathaBench-25K, with 5,000 narratives from classical collections (Aesop, Panchatantra, Kathasaritsagara), 20,000 transitions, and 28,712 recoverability questions.

We define Semantic Trajectory Gap (STG) as text-only minus image-only recoverability, measuring transition meaning lost during visualization. Human validation yields Fleiss' κ = 0.845. Experiments across state-of-the-art generators show substantial STG (23.5 ± 1.3). Semantic Compass, an actionability probe, uses KathaTrace signals for post-generation repair, improving storyboard selection.

Paper

The full manuscript includes the task formulation, KathaBench-25K construction protocol, semantic trajectory representation, recoverability questions and evidence conditions, validity filtering, the Semantic Trajectory Gap definition, contrastive variants, judge aggregation, Semantic Compass, main results, ablations, and extended qualitative analysis.

Open Paper

Core Idea: Trajectory Meaning, Not Just Scene Quality

KathaTrace starts from a simple observation: a storyboard can satisfy every standard quality check—sharp images, consistent characters, smooth visual continuity—and still fail at the one thing visual narratives exist to do, which is to let a viewer follow the story from images alone. A transition between two scenes can look fine while the causal, intentional, or emotional link connecting them silently disappears.

Rather than scoring images in isolation, KathaTrace evaluates each scene-to-scene transition under three evidence conditions: text-only (can a reader recover the transition from narration alone?), image-only (can a viewer recover it from the generated images alone?), and text+image (does combined evidence help?). The drop from text-only to image-only recoverability is the Semantic Trajectory Gap—a direct measurement of meaning lost during visualization.

Text-only evidence

Establishes the upper bound on transition recoverability using only the source narrative, before any image generation occurs.

Image-only evidence

Measures what a viewer can actually recover about the transition from the generated storyboard alone, exposing semantic trajectory collapse.

Semantic Trajectory Gap (STG)

Text-only minus image-only recoverability: a generator-agnostic score for how much transition meaning a visualization pipeline loses.

What Is New?

1. A failure mode existing benchmarks miss

Prior visual narrative benchmarks score image quality, content faithfulness, and local scene coherence. None directly test whether transition meaning—why scene N leads to scene N+1—survives visualization.

2. A generator-agnostic diagnostic protocol

KathaTrace evaluates transitions under three evidence conditions and filters ambiguous items via validity filtering, producing a protocol that applies to any story visualization system without retraining.

3. KathaBench-25K

5,000 narratives from Aesop, Panchatantra, and Kathasaritsagara, with 25,000 scenes, 20,000 transitions, 28,712 recoverability questions, and 10,000 contrastive variants, validated with Fleiss' κ = 0.845.

4. Semantic Compass

An actionability probe that turns KathaTrace's diagnostic signals into a post-generation repair and selection mechanism, improving which storyboard candidates get kept.

Method Overview

Each source story is decomposed into scenes and scene-to-scene transitions. For every transition, KathaTrace generates recoverability questions targeting the semantic link between scenes (action, causal, intentional, emotional, and consequence dimensions), then poses those questions under text-only, image-only, and text+image evidence. Validity filtering removes ambiguous items before judge aggregation computes per-dimension recoverability and the overall Semantic Trajectory Gap.

KathaTrace framework: scenes decomposed into transitions, evaluated under text-only, image-only, and text+image evidence to compute STG and drive Semantic Compass.
KathaTrace framework. Scenes and transitions are extracted from each narrative, evaluated under three evidence conditions, and aggregated into the Semantic Trajectory Gap and per-dimension diagnostics that feed the Semantic Compass repair signal.

A Benchmark Failure Case

Case study where a storyboard omits the rescue transition, losing the reciprocity moral of the story.
Case study: a generated storyboard omits the rescue transition entirely. The images remain visually coherent, but the reciprocity moral that depends on that transition becomes unrecoverable from images alone.

KathaBench-25K

KathaBench-25K draws on three classical narrative collections chosen for their dense use of causal, intentional, and moral transitions: Aesop's Fables, the Panchatantra, and the Kathasaritsagara ("Ocean of Story"). Each story is annotated into scenes and transitions, with recoverability questions and contrastive variants generated and validated for use across text-only, image-only, and text+image evaluation.

5,000 source narratives
25,000 annotated scenes
20,000 scene-to-scene transitions
28,712 recoverability questions
10,000 contrastive variants
κ = 0.845 Fleiss' kappa, human validation

Representative Story Examples

KathaTrace draws from three classical story sources in the `data/Stories` folder — Aesop's Fables, the Panchatantra, and The Ocean of Story. Below are all nine demo storyboards, three from each source, each flipping automatically through its five scenes.

Aesop’s Fables

The Cat and the Birds

The Cat and the Birds — scene 1 The Cat and the Birds — scene 2 The Cat and the Birds — scene 3 The Cat and the Birds — scene 4 The Cat and the Birds — scene 5
A compact fable where the cat's intent and the moral consequence hinge on a single visual transition. Beware deceptive appearances.
Aesop’s Fables

The Peacock and the Crane

The Peacock and the Crane — scene 1 The Peacock and the Crane — scene 2 The Peacock and the Crane — scene 3 The Peacock and the Crane — scene 4 The Peacock and the Crane — scene 5
A vain display of plumage meets a quiet act of flight, testing whether ability survives the comparison. Ability matters more than beauty.
Aesop’s Fables

The Wolf and the Lamb

The Wolf and the Lamb — scene 1 The Wolf and the Lamb — scene 2 The Wolf and the Lamb — scene 3 The Wolf and the Lamb — scene 4 The Wolf and the Lamb — scene 5
A false accusation collapses under scrutiny, yet the outcome depends on power rather than truth. Power can ignore justice.
Panchatantra

The Monkey and the Wedge

The Monkey and the Wedge — scene 1 The Monkey and the Wedge — scene 2 The Monkey and the Wedge — scene 3 The Monkey and the Wedge — scene 4 The Monkey and the Wedge — scene 5
A strategic tale where action and causality must survive scene-to-scene visualization. Do not meddle without understanding.
Panchatantra

The Jackal and the Drum

The Jackal and the Drum — scene 1 The Jackal and the Drum — scene 2 The Jackal and the Drum — scene 3 The Jackal and the Drum — scene 4 The Jackal and the Drum — scene 5
A frightening sound on an empty battlefield turns out to be nothing once investigated closely. Inquiry overcomes fear.
Panchatantra

The Fall and Rise of a Merchant

The Fall and Rise of a Merchant — scene 1 The Fall and Rise of a Merchant — scene 2 The Fall and Rise of a Merchant — scene 3 The Fall and Rise of a Merchant — scene 4 The Fall and Rise of a Merchant — scene 5
A merchant's fortune collapses and is rebuilt slowly through patient, honest labor. Humility restores honor.
The Ocean of Story

The Disguise of Vāsavadattā

The Disguise of Vāsavadattā — scene 1 The Disguise of Vāsavadattā — scene 2 The Disguise of Vāsavadattā — scene 3 The Disguise of Vāsavadattā — scene 4 The Disguise of Vāsavadattā — scene 5
A richly layered narrative that highlights intentional and emotional continuity across transitions. Perseverance protects future success.
The Ocean of Story

The Fall of Sunda and Upasunda

The Fall of Sunda and Upasunda — scene 1 The Fall of Sunda and Upasunda — scene 2 The Fall of Sunda and Upasunda — scene 3 The Fall of Sunda and Upasunda — scene 4 The Fall of Sunda and Upasunda — scene 5
Two inseparable brothers fall to rivalry once jealousy over a single desire divides their shared strength. Jealousy destroys strength.
The Ocean of Story

The Stratagem of King Puṇyasena

The Stratagem of King Puṇyasena — scene 1 The Stratagem of King Puṇyasena — scene 2 The Stratagem of King Puṇyasena — scene 3 The Stratagem of King Puṇyasena — scene 4 The Stratagem of King Puṇyasena — scene 5
A besieged king feigns retreat, luring his stronger enemy into a trap sprung from cunning, not force. Strategy can defeat force.

Diagnostic Case Studies

The Semantic Trajectory Gap decomposes into per-dimension failures. These case studies isolate each dimension: action failure, causal failure, intentional failure, emotional failure, and consequence failure, using well-known fables where the moral hinges on a specific transition type.

Action failure — "The Monkey and the Wedge"

Action-failure case study from The Monkey and the Wedge.
The physical action that drives the outcome is lost between generated scenes.

Causal failure — "The Boy Who Cried Wolf"

Causal-failure case study from The Boy Who Cried Wolf.
The causal chain linking repeated deception to its consequence becomes unrecoverable from images alone.

Intentional failure — "The Cat and the Birds"

Intentional-failure case study from The Cat and the Birds.
The cat's underlying intention—central to the moral—does not survive visualization.

Emotional failure — "Sunda and Upasunda"

Emotional-failure case study from Sunda and Upasunda.
The emotional escalation that motivates the climax is flattened across transitions.

Consequence failure — "The Goose That Laid the Golden Eggs"

Consequence-failure case study from The Goose That Laid the Golden Eggs.
The link between the greedy action and its irreversible consequence disappears between scenes.

Boundary and Hard Cases

Not every transition loss is equally diagnosable. These boundary cases stress-test KathaTrace's validity filtering on transitions that are inherently hard to visualize: symbolic sacrifice, hidden intention, and long-range reciprocity spanning many scenes.

Symbolic sacrifice — "King Shibi and the Dove"

Boundary case: symbolic sacrifice in King Shibi and the Dove.
A symbolic act of sacrifice that resists literal visual depiction.

Hidden intention — "The Monkey and the Crocodile"

Boundary case: hidden intention in The Monkey and the Crocodile.
An intention deliberately concealed from other characters, and from the viewer.

Long-range reciprocity — "The Elephants and the Mice"

Boundary case: long-range reciprocity in The Elephants and the Mice.
A reciprocal payoff that depends on a transition many scenes earlier in the story.

Extensibility Beyond Storyboards

KathaTrace is evaluated primarily on storyboard-style narrative generation, but the transition-level recoverability protocol is not tied to any single visual format. The same evidence-condition methodology extends to comics, illustrated pages, and video keyframes, wherever a sequence of images is meant to carry a story's meaning.

Cross-format extensibility overview spanning comics, illustrated pages, and video keyframes.
Cross-format extensibility: the same transition-recoverability protocol applies across comics, illustrated pages, and video keyframes.

Comic-style pilot — reciprocity at the lunch table

Comic-style pilot example illustrating a lunch-table reciprocity transition.

A modern comic-style retelling tests the protocol outside classical fable settings.

Comic-style pilot — spare-bicycle reciprocity

Comic-style pilot example illustrating a spare-bicycle reciprocity transition.

A second comic pilot confirms the protocol generalizes across everyday reciprocity scenarios.

Results

Across state-of-the-art story visualization generators, KathaTrace reveals a substantial and consistent Semantic Trajectory Gap, confirming that semantic trajectory collapse is a widespread failure mode rather than an artifact of any single model. The evaluator-baseline ablation shows that a naive recoverability evaluator overstates how much transition meaning survives visualization, underscoring the need for KathaTrace's evidence-condition protocol and validity filtering.

Semantic Compass, which consumes per-dimension KathaTrace diagnostics as an actionability signal, improves storyboard selection over generation pipelines that ignore trajectory-level feedback.

Method STG ↓ Text-only Recov. Image-only Recov. Text+Image Recov.
Naive Evaluator Baseline 28.9 ± 1.5 91.2 62.3 88.7
StoryDiffusion 25.1 ± 1.4 92.0 66.9 90.1
Other SOTA Generators (avg.) 24.6 ± 1.3 91.6 67.0 89.8
KathaTrace protocol (mean over generators) 23.5 ± 1.3 91.8 68.3 90.4

STG is text-only minus image-only recoverability (lower is better — less meaning lost during visualization). Figures are illustrative aggregates reported across the KathaBench-25K test split.

Related Links

KathaTrace builds on advances in story visualization, narrative consistency evaluation, and VLM-as-judge methodology. Unlike benchmarks that score images for quality, faithfulness, or local coherence, KathaTrace focuses on whether the meaning of a transition between scenes survives visualization at all.

Story visualization generators: StoryDiffusion, StoryBooth, DreamStory, StoryGPT-V, Make-A-Story, CMOTA, ViSTA, and Story2Board.

Story/narrative benchmarks: LogiStory / LogicTale, VinaBench, ViStoryBench, StoryBench, and MSVBench.

Text–image alignment / faithfulness metrics: TIFA, DSG, and VQAScore.

Unlike these benchmarks, which score image quality, faithfulness, or local scene coherence, KathaTrace specifically targets whether the meaning of a transition between scenes—why one scene leads to the next—survives visualization at all.

Frequently Asked Questions

Why is another benchmark needed?

Existing benchmarks primarily evaluate visual quality, image-text alignment, consistency, or generic question answering. KathaTrace instead evaluates transition recoverability: whether the semantic changes connecting consecutive events remain recoverable from the generated storyboard. This isolates narrative meaning preservation rather than visual plausibility, enabling diagnosis of failures that remain undetected by existing fidelity metrics.

Why is recoverability a better evaluation objective?

The goal of storyboard generation is not merely to produce visually coherent images, but to preserve the narrative conveyed by the source story. Recoverability directly measures whether an independent evaluator can reconstruct the intended transition semantics from the generated storyboard. This operationalizes semantic preservation independently of image aesthetics or language similarity.

Why are transitions the evaluation unit instead of scenes?

Narrative meaning primarily resides in changes between events, rather than in isolated scenes. Individual scenes often remain visually plausible even when the causal, emotional, motivational, or consequential transition connecting them has been lost. Evaluating transitions therefore targets the semantic structure that distinguishes a coherent narrative from a sequence of unrelated images.

Why does STG measure semantic loss?

Text-only recoverability approximates the semantic information available in the source narrative, while image-only recoverability measures how much of that information remains accessible after storyboard generation. Their difference therefore estimates the semantic information lost during visual generation. The metric is further validated through human correlation, evaluator agreement, robustness analyses, and multiple baseline comparisons.

How do we know the annotations are reliable?

All annotations follow a fixed schema with predefined transition fields, accepted-answer sets, ambiguity filtering, and multi-stage human validation. The released benchmark additionally includes validation metadata, inter-annotator agreement statistics, provenance information, and quality-control procedures, allowing users to inspect annotation reliability rather than treating annotations as opaque labels.

Could the benchmark simply be measuring image quality?

Multiple controls suggest otherwise. Planner-only improvements substantially reduce STG even when the renderer is fixed, whereas changing renderers alone produces smaller improvements. Furthermore, semantic contrastive experiments demonstrate that visually similar but semantically incorrect transitions still receive substantially worse recoverability scores, indicating that STG measures semantic preservation rather than visual appearance alone.

Why use VLM judges instead of humans?

Large-scale evaluation requires tens of thousands of recoverability decisions, making fully human evaluation impractical. KathaTrace therefore uses structured VLM judges calibrated against human evaluations. Human studies, judge agreement experiments, and multi-judge analyses demonstrate that the observed trends remain stable across judge families while maintaining scalability and reproducibility.

Is the benchmark practical for the community?

The released benchmark supports multiple evaluation budgets. Besides the full benchmark, calibrated subsets (e.g., Human-Gold-1K and Strict-Gold-400) enable rapid experimentation while preserving the same evaluation protocol. Since annotations and evaluation metadata are precomputed, users only generate storyboards and execute the released evaluation pipeline.

What scientific insight does the benchmark provide?

KathaTrace reveals which transition types are systematically lost during storyboard generation, distinguishes planning failures from rendering failures, quantifies ambiguity across narrative categories, localizes transition-level errors, and identifies narrative structures that remain challenging despite visually coherent outputs. These analyses provide actionable understanding rather than only leaderboard rankings.

Why will the community adopt this benchmark?

The benchmark addresses an evaluation dimension not covered by existing metrics while remaining reproducible and extensible. It provides standardized annotations, fixed evaluation protocols, released prompts, accepted-answer sets, confidence intervals, and scalable evaluation subsets. Consequently, it enables consistent comparison across methods while diagnosing semantic failures that traditional visual-fidelity metrics overlook.

BibTeX

@article{murthy2026kathatrace,
  author    = {Murthy, Jamuna S. and Monsefi, Amin Karimi and Ramnath, Rajiv},
  title     = {KathaTrace: Diagnosing Semantic Trajectory Collapse in Generated Visual Narratives},
  year      = {2026},
}