Controlla GitHub
Research demo · graph-defined controls · checkpoints coming soon

Controlla: graph-guided control beyond one attribute.

A demo interface for Controlla, a graph-constrained latent-geometry framework for controllable generation. Emotion is the primary multimodal testbed, while pose and lighting illustrate how the same framework can support other graph-defined control factors.

Open demo View control modes Paper / links
Emotion Graph Pose Graph Lighting Graph Identity Factor Graph Traversal
Demo UI ready
Drop reference image here
or replace with your image path
Optional text / audio / attribute cue
for multimodal control
Graph typeEmotion / Pose / Light
Identity strength0.85
TraversalGeodesic
Guidance scale7.5
Seed42
Output size1024 × 1024
This is a static demo-ready interface. Later, connect the button to a Hugging Face Space, Gradio app, FastAPI backend, Flask endpoint, or hosted checkpoint API.
Graph traversal preview

Generated output preview

Add your preview image as assets/demo-output.jpg, or connect this panel to live inference. The same interface can demonstrate emotion, pose, and lighting control.

Plug-in graph controls

Controlla is not tied to one attribute type. A control factor becomes compatible when its semantic states can be represented as a graph and aligned with the learned attribute factor.

Emotion control example

Emotion control

Traverse affective states such as neutral, happy, sad, angry, fearful, surprised, disgusted, or contemptuous while preserving reference identity and multimodal consistency.

Pose control example

Pose control

Replace the affect graph with a pose graph to support ordered pose traversal, such as frontal to left-facing or frontal to right-facing changes, while keeping identity stable.

Lighting control example

Lighting control

Use a lighting graph to guide structured illumination changes, such as neutral, warm, side-lit, low-key, or cinematic lighting, while preserving content and identity.

Why Controlla?

Instead of treating control only as inference-time prompting, Controlla frames controllability as a property of structured latent geometry.

◈

Graph-defined control

Semantic attributes are organized using graph priors, allowing emotion, pose, lighting, or other control spaces to be represented as structured traversal problems.

◎

Identity preservation

The identity factor is held stable while the attribute factor moves along a graph-consistent path, reducing identity drift during semantic transformation.

✦

Multimodal conditioning

The framework supports reference images, text prompts, emotion labels, and optional audio cues, with extensibility to other control signals.

AffectHuman-43K benchmark

Controlla is evaluated primarily on AffectHuman-43K, an emotion-aligned controlled generation benchmark. Affect is used as a demanding testbed because it requires identity preservation, semantic accuracy, smooth traversal, and multimodal consistency.

View dataset
42K+multimodal samples
8emotion categories
0%cross-split identity leakage
4input fields: image, reference, text, audio

Paper, code, dataset, and checkpoints

Add or update public links as the release becomes ready. This section is designed to support a paper PDF, GitHub repository, dataset page, checkpoint release, and hosted inference demo.

📄

Paper

Link the public PDF or arXiv version here when it is ready for release.

⌘

Code

Link implementation code, graph construction scripts, evaluation scripts, and reproducibility instructions.

⚙

Checkpoints

Link model weights or a hosted inference demo after checkpoint release is approved.

FAQ

What is Controlla?

Controlla is a graph-constrained latent-geometry framework for controllable generation. It learns identity and attribute factors, then guides semantic change through graph-consistent latent traversal.

Is Controlla only for emotion editing?

No. Emotion is the primary multimodal evaluation testbed. The same framework can support other control factors, such as pose or lighting, when those factors can be represented through a graph.

Why include pose and lighting?

Pose and lighting show that the method is not restricted to affective labels. They illustrate the plug-in nature of the graph prior: replace the affect graph with a task-specific graph and preserve identity while traversing the new attribute space.

How is Controlla different from prompt-based editing?

Prompt-based editing usually imposes control externally at inference time. Controlla instead tries to shape the latent representation itself so that semantic changes follow structured, graph-consistent trajectories.

Why is graph geometry useful?

Graph geometry defines how semantic states are related. This helps the model avoid arbitrary interpolation, abrupt transitions, identity drift, and inconsistent intermediate edits.

What inputs does the demo support?

The demo is designed around reference images, text prompts, emotion labels, optional audio cues, and graph-defined attribute controls such as emotion, pose, and lighting.

What is AffectHuman-43K?

AffectHuman-43K is a leakage-aware multimodal benchmark for reference-grounded affective control. It contains image, reference-image, text, and audio fields and uses identity-disjoint splits.

Is this a live demo?

The current page is a demo-ready static interface. Live inference can be connected later through a Hugging Face Space, Gradio app, FastAPI backend, Flask app, or hosted checkpoint API.

Where will code, dataset, and checkpoints be released?

Links can be added to the paper, GitHub repository, Hugging Face dataset, and checkpoint release once the public release is approved.

What are the current limitations?

The main limitations include identity drift under extreme attribute changes, ambiguity in conflicting multimodal cues, artifacts under difficult poses or lighting, and the need for stronger live inference deployment before public interactive use.