---
base_model: Qwen/Qwen3.6-27B
language:
- en
library_name: transformers
license: apache-2.0
pipeline_tag: image-text-to-text
tags:
- qwen
- qwen3.6
- qwen3_5
- secopd
- prompt-injection
- model-security
- on-policy-distillation
- safetensors
- multimodal
- thinking
---

# Qwen3.6-27B-SecOPD

[Paper](https://arxiv.org/abs/2608.21500) | [Code](https://github.com/pppyb/SecOPD) | [Project](https://pppyb.github.io/SecOPD/) | [Evaluation guide](docs/evaluation.md)

**Qwen3.6-27B-SecOPD** is a prompt-injection defense model built on
[Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B), trained with Secure
On-Policy Distillation (SecOPD). It uses a dedicated `input` role to separate
external data from trusted user instructions. During training, the model
responds to injected inputs and receives token-level feedback from a frozen
teacher conditioned on the corresponding clean inputs.

This repository provides the fully merged **step-150, full-response KL**
checkpoint from *SecOPD: Mitigating Adaptive Prompt Injections by On-Policy
Distillation*. The LoRA adapter is merged into the model weights; no separate
adapter loading is needed.

## Quick start

Use vLLM 0.19.0 or newer with this repository's tokenizer and chat template.
Place the trusted instruction in `user` and external content, such as
documents, retrieved passages, or tool results, in `input`. Supply message
content as strings:

```python
from transformers import AutoTokenizer
from vllm import LLM, SamplingParams

model_id = "pybbb/Qwen3.6-27B-SecOPD"
tokenizer = AutoTokenizer.from_pretrained(model_id)
llm = LLM(
    model=model_id,
    tokenizer=model_id,
    tensor_parallel_size=2,
    max_model_len=32768,
)

messages = [
    {"role": "user", "content": "Summarize the report in one sentence."},
    {"role": "input", "content": "Quarterly revenue increased by 8%."},
]
prompt_ids = tokenizer.apply_chat_template(
    messages, tokenize=True, return_dict=False,
    add_generation_prompt=True, enable_thinking=True,
)
sampling = SamplingParams(
    temperature=1.0, top_p=0.95, top_k=20, max_tokens=16384,
)
outputs = llm.generate([{"prompt_token_ids": prompt_ids}], sampling)
print(outputs[0].outputs[0].text)
```

This example uses two GPUs and a 32K context window. Thinking is enabled
by default; set `enable_thinking=False` for direct answers.

## Training

The model is trained on Tinker with Qwen3.6-27B as both the student
initialization and the frozen teacher. For each task, the student samples
a response from an attacked prompt. The teacher scores those same tokens
under the clean prompt, conditioned on the student's preceding response
tokens:

```text
student context at token t = attacked prompt + student_response[:t]
teacher context at token t = clean prompt    + student_response[:t]
advantage[t] = lambda * stop_gradient(teacher_logprob[t] - rollout_logprob[t])
```

The **full-response KL** objective applies this signal to both reasoning
and final-answer tokens. Prompt tokens are masked, and only the student's
LoRA parameters are updated through the importance-sampling loss.

| Setting | Released checkpoint |
| --- | --- |
| Student initialization and frozen teacher | `Qwen/Qwen3.6-27B` |
| Architecture | `Qwen3_5ForConditionalGeneration` |
| Training tokenizer / renderer | Qwen3.6 tokenizer / Tinker Cookbook `qwen3_5` |
| Clean task source | `yahma/alpaca-cleaned`, records with non-empty input |
| Injection pool | Stanford Alpaca `alpaca_data.json` |
| Validation split | 10% of eligible clean tasks; split seed 0 |
| Simulated injection mixture | 90% straightforward, 10% completion-style |
| Straightforward injection position | Beginning or end, with equal probability |
| Trainable parameters | Student LoRA adapter; frozen teacher |
| LoRA rank | 128 |
| Learning rate / schedule | `1e-4` / constant |
| Groups per batch / rollouts per group | 64 / 1 |
| Rollout temperature | 1.0 |
| Maximum rollout length | 16,384 tokens |
| Distillation coefficient / KL discount | 1.0 / 0.0 |
| Update loss / substeps | Importance sampling / 1 |
| Training budget | 150 updates, 9,600 task rollouts |
| Checkpoint and evaluation interval | Every 10 updates |
| External security reward model | None |

The [reproduction guide](https://github.com/pppyb/SecOPD/blob/main/docs/REPRODUCIBILITY.md)
and [training configuration](https://github.com/pppyb/SecOPD/blob/main/training/tinker/configs/qwen36_full_kl.paper.json)
provide the setup, training command, and checkpoint export procedure.

## Results

Results from the [SecOPD paper](https://arxiv.org/abs/2608.21500).
Benchmark settings and reproduction commands are in the
[evaluation guide](docs/evaluation.md).

### Security

Attack success rate (ASR), in percent; lower is better.

| Benchmark or attack | Undefended | Meta-SecAlign | GRPO | SecOPD |
| --- | ---: | ---: | ---: | ---: |
| SEP Static | 99.4 | 28.9 | 15.0 | **1.3** |
| SEP Basic Adaptive | 99.0 | 5.5 | 2.3 | **0.2** |
| PISmith Adaptive, pass@10 | 97.9 | 94.0 | 61.2 | **9.0** |
| AgentDojo Static | 26.7 | 5.5 | **0.7** | 4.7 |

### Utility

All values are percentages; higher is better. AgentDojo Utility measures
user-task success on clean inputs.

| Benchmark | Undefended | Meta-SecAlign | GRPO | SecOPD |
| --- | ---: | ---: | ---: | ---: |
| AlpacaEval2 | 81.4 | 82.3 | 76.0 | 80.1 |
| SEP Utility | 88.0 | 89.3 | 79.5 | 88.6 |
| AgentDojo Utility | 92.8 | 92.8 | 82.5 | 90.7 |
| MMLU-Pro | 84.1 | 83.8 | 83.0 | 84.1 |
| GPQA Diamond | 79.8 | 78.3 | 77.8 | 81.3 |
| GSM8K | 97.7 | 97.8 | 97.4 | 97.4 |
| Minerva Math | 92.7 | 95.1 | 85.1 | 94.3 |

## License and citation

Released under [Apache 2.0](LICENSE), following the base model.

```bibtex
@misc{peng2026secopd,
  title = {{SecOPD}: Mitigating Adaptive Prompt Injections by On-Policy Distillation},
  author = {Peng, Yibo and Lian, Long and Wagner, David and Chen, Sizhe},
  year = {2026},
  eprint = {2608.21500},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.21500}
}
```