Obliviate compares conditional and pseudo-unconditional predictions using the same visual-token prefix, then applies trajectory-wide KL supervision to favor safer continuations.Watch on YouTube ↗
01 / Abstract
Abstract
Autoregressive image generators are becoming central to unified multimodal systems, yet most concept-erasure research has focused on diffusion models. Obliviate adapts erasure to visual-token generation through aligned prefixes, distribution-level KL supervision, and updates across complete autoregressive rollouts. A frozen teacher constructs safer target distributions and a student learns them along the full trajectory. Evaluation spans Liquid, Emu3-Gen, and Janus-Pro, covering explicit content, graphic violence, and brand removal; on Liquid, nudity detection on Ring-A-Bell falls from 91.58% to 3.15%, with image-quality metrics remaining close to the original model.
02 / Contributions
Contributions
01
Autoregressive erasure
Adapts teacher-guided concept erasure to models that generate images as sequences of visual tokens.
02
Aligned visual prefixes
Conditions both teacher branches on the same evolving image context to create a stable and meaningful target signal.
03
Full-trajectory supervision
Uses KL divergence over visual-token distributions across complete rollouts instead of isolated token updates.
04
Three autoregressive generators
Studies explicit content, graphic violence, branding, and artistic style across three autoregressive generators.
03 / Qualitative results
Erasure across model families
Compare base generations with the same prompts after Obliviate. Choose a model and erasure target, then drag the image boundary.
Obliviate
Drag the front image divider
Base model
↔
LIQUID · Example 01 / 03❌ bloody gore
Graphic content is redirected toward a non-graphic interpretation of the prompt.
04 / Training objective
Erasure over visual-token trajectories
A prefix is the image tokens generated so far. Aligning that context makes the teacher comparison meaningful; supervising the full sequence carries the target across the rollout.
Prefix alignment and supervision
Separate image prefixes
Concept prompt
Empty prompt ∅
The teacher branches see different image-token histories, mixing concept guidance with differences in the generated image.
Aligned prefixes · one position
Concept prompt
Empty prompt ∅
Student loss
Both branches see the same image tokens so far. The student is supervised at one sampled position.
Aligned prefixes · full rollout
Concept prompt
Empty prompt ∅
Student loss
Obliviate uses the same prefix in both branches and supervises every position in the sampled sequence.
ztarget = z∅ − η (zc − z∅)ptarget = softmax(ztarget)
Here c is the concept prompt and ∅ is an empty text prompt. Both teacher branches share the same image-token prefix.
Teacher · concept promptTeacher · empty text promptGuided target for the student
Illustrative eight-token vocabulary with a shared probability scale. The highlighted token shows the guidance operation; a concept is represented across many token choices. The student matches the guided distribution with KL divergence.
05 / Citation
Citation
Shakibania, H., Grebe, J. H., Braun, T., Aktemur, E., Aslani, S., Yiğit, M. G., & Rohrbach, M. (2026). Obliviate: Erasing Concepts from Autoregressive Image Generation Models. Accepted at ECCV 2026. arXiv:2606.28643.
BibTeX+
@misc{shakibania2026obliviate,
title={Obliviate: Erasing Concepts from Autoregressive Image Generation Models},
author={Hossein Shakibania and Jonas Henry Grebe and Tobias Braun and Ege Aktemur and Saleh Aslani and Mehmet Görkem Yiğit and Marcus Rohrbach},
year={2026},
eprint={2606.28643},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2606.28643},
}