Protected questions steer AI solvers toward secret wrong answers. Repeated target matches across an assignment form a detectable fingerprint of sustained blind copying.
01 / Abstract
Abstract
As multimodal assistants solve more educational exercises, detecting copied answers after submission becomes increasingly unreliable. This work explores a preventive alternative: add subtle, task-preserving perturbations to the visual parts of multiple-choice questions so AI solvers are steered toward designated incorrect answers. Across an assignment, those controlled errors form a statistical fingerprint. The method optimizes against accessible surrogate models, calibrates transfer through repeated black-box queries, and assembles questions whose answer patterns support likelihood-ratio testing across Claude, Gemini, and GPT assistants. The study establishes feasibility under a defined sustained-copying threat model while making educator judgment and the method’s limitations explicit.
02 / Contributions
Contributions
01
Perturbing visual exercises
Adds subtle changes to visual multiple-choice questions to steer AI solvers toward designated incorrect answers.
02
Controlled error fingerprint
Uses target-specific visual perturbations to create a pattern that blind AI copying reproduces across an assignment.
03
Calibrated detection
Combines black-box response calibration with assistant-specific statistical tests and clearly stated student-model assumptions.
03 / Protecting visual exercises
Protecting visual exercises
Subtle changes to a question’s image aim to steer the AI answer toward a secret incorrect option while preserving the task for students.
For company B, find the missing amounts.
Company B detail
A$63,020Intended AI target
B$58,410
C$71,320
D$77,490Correct answer
Protected input from the paper’s MMMU example. D is correct; the perturbation targets A. Paper, Figure 6 ↗
The intended change
Conceptual sets, not measured capability boundaries. The perturbation aims to preserve the exercise for students.
04 / Assignment construction
Constructing a protected assignment
Each question has a secret wrong-answer target. Repeated AI queries calibrate its reliability; detection combines the target matches across an assignment.
1
Choose a target
DCorrect→ASecret target
Select a visual question that students and the tested AI can answer, then assign an incorrect option.
2
Optimize the image
x + δ→A
Optimize a subtle image perturbation against accessible surrogate models to elicit the target.
Compare the answer pattern with the calibrated AI and genuine-student models, then flag unusual matches for educator review.
Target matches across an assignment
Matches the secret target
Illustrative answer patterns for eight questions
Question
01
02
03
04
05
06
07
08
Secret targets
A
D
A
C
B
D
A
C
Genuine answers
D
A
D
B
D
B
C
C, matches target
Copied AI answers
A, matches target
D, matches target
B
C, matches target
B, matches target
D, matches target
A, matches target
C, matches target
One matching answer can occur by chance. The test uses the assignment-wide pattern and depends on the calibrated assistant and assumed genuine-student model. These sequences are illustrative, not measured student responses.
05 / Citation
Citation
Braun, T., Grebe, J., Rethfeld, L., & Rohrbach, M. (2026). Fighting Fire with Fire: On the Feasibility of Protecting Exercises Against AI Cheating. arXiv:2608.01112.
BibTeX+
@misc{braun2026fighting,
title = {Fighting Fire with Fire: On the Feasibility of Protecting Exercises Against AI Cheating},
author = {Tobias Braun and Jonas Grebe and Louis Rethfeld and Marcus Rohrbach},
year = {2026},
eprint = {2608.01112},
archivePrefix = {arXiv}
}