← Research

Reject, Remember, Retry: Why Multi-Turn Attackers Should Remember Their Mistakes

Jekaterina Novikova, Yuqing Huang, Jack Ji, Sagarika Suresh, Jithin Pradeep, Lovedeep Gondara

Existing multi-turn jailbreak attackers condition on the same conversation history as the target. We introduce $R^3$ (Reject, Remember, Retry), a black-box multi-LLM framework in which the attacker conditions on a compact state rather than the raw transcript. Turns that fail to raise the attack risk score are removed from the target's context but remembered by the attacker, together with the reason for failure. We call this asymmetric memory. Across two benchmarks and six target LLMs, $R^3$ outperforms state-of-the-art baselines by up to 53% in ASR@5 under matched query budgets, and a low-capacity attacker with $R^3$'s state is able to outperform high-capacity attackers without it by up to 34% in ASR@5. Ablations attribute most gains to remembering rejected turns, and the risk score acts as a coarse threshold rather than a progress measure.

PDF ↗

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace