TRoB-1

TRoB-1 is a PEFT LoRA adapter for google/gemma-2-9b-it. It supports preliminary Cochrane Risk of Bias tool version 1 (RoB 1) judgements from evidence passages selected by a trained reviewer.

TRoB-1 is a reviewer-support model. It does not replace trained reviewers, and its outputs require human verification.

Version and provenance

• Model version: 1.0.0
• Minimum reproducibility data version: 1.0.0
• Developed by: TRoB-1 research team, Medical School of Nantong University
• Model type: PEFT LoRA adapter for a causal language model
• Base model: google/gemma-2-9b-it
• Task: Evidence-linked preliminary RoB 1 judgement
• Primary language evaluated: English
• Training method: Two-stage supervised fine-tuning with 4-bit QLoRA
• Accompanying manuscript: A Fine-Tuned Large Language Model Workflow to Support Risk of Bias Assessment: Development, External Validation, and Reviewer Verification Study, accepted for publication in the Journal of Medical Internet Research (doi:10.2196/108001)

This repository contains adapter weights only. It does not redistribute Gemma 2 base-model weights. The exact Hugging Face commit corresponding to the locally cached base model used for training was not recorded; the public base-model identifier is therefore provided without an exact revision pin.

Intended workflow

TRoB-1 supports these RoB 1 domains:

1. Random sequence generation
2. Allocation concealment
3. Blinding of participants and personnel
4. Blinding of outcome assessment
5. Incomplete outcome data
6. Other bias

The input comprises the target domain, a reviewer-selected passage, and outcome-type context when applicable. Step 1 returns a structured preliminary judgement and a supporting quotation copied from the supplied passage. Step 2 produces a brief explanation without changing the Step 1 judgement. A clinician then verifies the judgement and supporting information.

Out-of-scope uses

TRoB-1 is not intended for autonomous final RoB assessment, selective-reporting assessment, RoB 2 assessment, clinical decision-making, or use without relevant source text. Performance in languages other than English has not been established.

Training data and procedure

The training corpus comprised 4,321 RoB 1 assessment records from 1,753 unique RCTs identified across 199 published systematic reviews concerning chronic obstructive pulmonary disease, diabetes, and hypertension. The record-level training corpus and verbatim passages are not redistributed because source articles and extracted text may be subject to copyright restrictions. Aggregate label and domain distributions are included in data/.

Parameter	Stage 1	Stage 2
Starting model	Gemma 2-9B-Instruct	Stage 1 adapter
Method	4-bit QLoRA supervised fine-tuning	4-bit QLoRA continued fine-tuning
LoRA rank	32	32
LoRA alpha	64	64
LoRA dropout	0.05	0.05
Learning rate	1e-4	5e-6
Epochs	3	2
Maximum sequence length	2048	2048
Batch size	1	1
Gradient accumulation	8	8
Random seed	42	42

The final adapter was produced using LLaMA-Factory 0.9.5.dev0, Transformers 4.57.6, PEFT 0.18.1, PyTorch 2.10.0+cu128, and bitsandbytes 0.49.2. Full settings are recorded in training_summary.json.

External validation

External validation used 24 RCTs and 161 assessment targets. Two clinicians independently selected passages, ran TRoB-1, and verified its outputs. The table reports strict base-model results, raw TRoB-1 results, and the clinician-verified workflow separately.

Clinician-assisted run	Workflow	Correct, n/161	Accuracy	Macro-F1	MCC
Clinician 1	Gemma 2 base, strict	97	60.2%	0.518	0.324
Clinician 1	TRoB-1 raw	119	73.9%	0.603	0.461
Clinician 1	Clinician-verified TRoB-1	132	82.0%	0.709	0.615
Clinician 2	Gemma 2 base, strict	84	52.2%	0.497	0.286
Clinician 2	TRoB-1 raw	118	73.3%	0.606	0.466
Clinician 2	Clinician-verified TRoB-1	136	84.5%	0.776	0.688

Clinician-verified values describe the combined model-and-clinician workflow, not standalone model performance. Final judgements from the two independent clinician-assisted runs agreed for 124 of 161 targets (77.0%; Cohen kappa=0.522; Gwet AC1=0.698). Machine-readable results are provided in evaluation_results.json.

Released main-analysis data

This data package covers the main external-validation analyses, clinician verification, between-clinician agreement, and descriptive workflow times. Supplementary retry audits and cross-hardware reproducibility assessments are outside its scope.

The data/ directory contains:

• a deidentified 161-target dataset for both clinician-assisted runs;
• synthetic RCT identifiers that support RCT-clustered reanalysis;
• aggregate training-set distributions and workflow times;
• expected results and a standard-library Python reproduction script; and
• a human-readable workbook and data dictionary.

It excludes titles, authors, article identifiers, source passages, supporting quotations, model explanations, and clinician comments. See data/README.md.

Loading the adapter

Access to google/gemma-2-9b-it is gated. Accept the Gemma usage conditions and authenticate with Hugging Face before loading the base model.

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

base_model_id = "google/gemma-2-9b-it"
adapter_id = "lyt9906/TRoB-1"

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.float16,
)

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    quantization_config=quantization_config,
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

A synthetic example is provided in inference_example.py. It contains no source-article text or study data.

The example demonstrates adapter loading and a single Step 1 generation. It is not the complete web-based assessment workflow.

Limitations

• Validation included 24 RCTs and 161 assessment targets.
• The model depends on the relevance and completeness of reviewer-selected passages.
• The model was developed for RoB 1 and does not assess selective reporting.
• Exact revision-level reproduction of the base model is limited because its training-time Hugging Face commit was not recorded.

Licenses and restrictions

• Adapter: governed by the Gemma Terms of Use and Gemma Prohibited Use Policy; see NOTICE and TERMS.md.
• Repository code: MIT License; see CODE_LICENSE.
• Released deidentified data: Creative Commons Attribution 4.0 International (CC BY 4.0); see data/LICENSE.
• Source passages: not redistributed because they may be protected by publisher copyright. Access to underlying source materials remains subject to the original publishers' terms.

Citation

Li Y, Chen J, Liang L, Geng X, Wang G, Geng J. A Fine-Tuned Large Language Model Workflow to Support Risk of Bias Assessment: Development, External Validation, and Reviewer Verification Study. J Med Internet Res. 2026;28:e108001. doi:10.2196/108001.

@article{li2026trob1,
  title  = {A Fine-Tuned Large Language Model Workflow to Support Risk of Bias Assessment: Development, External Validation, and Reviewer Verification Study},
  author = {Li, Yitian and Chen, Jingyu and Liang, Liting and Geng, Xingyun and Wang, Guohua and Geng, Jinsong},
  journal = {Journal of Medical Internet Research},
  year    = {2026},
  volume  = {28},
  pages   = {e108001},
  doi     = {10.2196/108001}
}