01 / Key message
Obliviate: Erasing Concepts from Autoregressive Image Generation Models
Guidance-based concept erasure for autoregressive image generators, trained across complete visual-token trajectories.
Aligned visual prefix
02 / Method
How it works
Generate a trajectory
A frozen teacher produces a visual-token rollout conditioned on the concept selected for removal.
Align both branches
Conditional and pseudo-unconditional predictions share the same visual prefix, keeping their comparison stable.
Shift distributions
Trajectory-wide KL supervision moves student probability toward safer continuations without discarding scene semantics.
03 / Abstract
Abstract
Autoregressive image generators are becoming central to unified multimodal systems, yet most concept-erasure research has focused on diffusion models. Obliviate adapts erasure to visual-token generation through aligned prefixes, distribution-level KL supervision, and updates across complete autoregressive rollouts. A frozen teacher constructs safer target distributions and a student learns them along the full trajectory. Evaluation spans Liquid, Emu3-Gen, and Janus-Pro, covering explicit content, graphic violence, and brand removal; on the defensive RAB benchmark, nudity detection falls from 91.58 to 3.15 while overall model utility is maintained.
04 / Contributions
What this adds
- 01
Autoregressive erasure
Addresses concept removal directly in modern visual-token generators rather than translating assumptions from diffusion models.
- 02
Aligned visual prefixes
Conditions both teacher branches on the same evolving image context to create a stable and meaningful target signal.
- 03
Full-trajectory supervision
Uses KL divergence over visual-token distributions across complete rollouts instead of isolated token updates.
- 04
Broad evaluation
Studies explicit content, graphic violence, branding, and artistic style across three autoregressive generators.
05 / Citation
Citation
Shakibania, H., Grebe, J. H., Braun, T., Aktemur, E., Aslani, S., Yiğit, M. G., & Rohrbach, M. (2026). Obliviate: Erasing Concepts from Autoregressive Image Generation Models. Accepted at ECCV 2026. arXiv:2606.28643.
BibTeX
@misc{shakibania2026obliviate,
title = {Obliviate: Erasing Concepts from Autoregressive Image Generation Models},
author = {Hossein Shakibania and Jonas Henry Grebe and Tobias Braun and Ege Aktemur and Saleh Aslani and Mehmet Görkem Yiğit and Marcus Rohrbach},
year = {2026},
eprint = {2606.28643},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2606.28643}
}