Image Classification
Transformers
ONNX
Safetensors
timm
vit
detection
deepfake
forensics
deepfake_detection
community
opensight
Instructions to use buildborderless/CommunityForensics-DeepfakeDet-ViT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use buildborderless/CommunityForensics-DeepfakeDet-ViT with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-classification", model="buildborderless/CommunityForensics-DeepfakeDet-ViT") pipe("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/parrots.png")# Load model directly from transformers import AutoImageProcessor, AutoModelForImageClassification processor = AutoImageProcessor.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT") model = AutoModelForImageClassification.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT", device_map="auto") - timm
How to use buildborderless/CommunityForensics-DeepfakeDet-ViT with timm:
import timm model = timm.create_model("hf_hub:buildborderless/CommunityForensics-DeepfakeDet-ViT", pretrained=True) - Inference
- Notebooks
- Google Colab
- Kaggle
docs: overhaul README — fix notice, ONNX variant guide, test app, deprecate old ONNX repo
Browse files
LICENSE
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
MIT License
|
| 2 |
+
|
| 3 |
+
Copyright (c) 2024 Jeongsoo Park, Andrew Owens, University of Michigan
|
| 4 |
+
Copyright (c) 2025 Borderless (HuggingFace integration, ONNX exports, configuration)
|
| 5 |
+
|
| 6 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 7 |
+
of this software and associated documentation files (the "Software"), to deal
|
| 8 |
+
in the Software without restriction, including without limitation the rights
|
| 9 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 10 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 11 |
+
furnished to do so, subject to the following conditions:
|
| 12 |
+
|
| 13 |
+
The above copyright notice and this permission notice shall be included in all
|
| 14 |
+
copies or substantial portions of the Software.
|
| 15 |
+
|
| 16 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 17 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 18 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 19 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 20 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 21 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 22 |
+
SOFTWARE.
|
README.md
CHANGED
|
@@ -14,78 +14,154 @@ tags:
|
|
| 14 |
- deepfake_detection
|
| 15 |
- community
|
| 16 |
- opensight
|
|
|
|
| 17 |
---
|
| 18 |
|
| 19 |
-
#
|
| 20 |
|
| 21 |
-
|
| 22 |
|
| 23 |
-
##
|
| 24 |
|
| 25 |
-
|
| 26 |
|
| 27 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
|
| 29 |
-
|
| 30 |
|
| 31 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
-
##
|
| 34 |
-
|
| 35 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
|
| 37 |
-
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
|
| 43 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
|
| 45 |
### Links
|
| 46 |
-
- **Repository:** [JeongsooP/Community-Forensics](https://github.com/JeongsooP/Community-Forensics)
|
| 47 |
-
- **Paper:** [arXiv:2411.04125](https://arxiv.org/pdf/2411.04125)
|
| 48 |
-
- **Project Page:** https://jespark.net/projects/2024/community_forensics
|
| 49 |
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
-
|
| 53 |
-
-
|
| 54 |
|
| 55 |
-
|
| 56 |
-
- **Framework:** PyTorch 2.0
|
| 57 |
-
- **Precision:** bf16 mixed
|
| 58 |
-
- **Optimizer:** AdamW (lr=5e-5)
|
| 59 |
-
- **Epochs:** 10
|
| 60 |
-
- **Batch Size:** 32
|
| 61 |
|
| 62 |
-
##
|
| 63 |
-
### Unverified Testing Results
|
| 64 |
-
- Only unverified because we currently lack resources to evaluate a dataset over 1.4T large.
|
| 65 |
|
| 66 |
-
|
| 67 |
-
|---------------|-------|
|
| 68 |
-
| Accuracy | 97.2% |
|
| 69 |
-
| F1 Score | 0.968 |
|
| 70 |
-
| AUC-ROC | 0.992 |
|
| 71 |
-
| FP Rate | 2.1% |
|
| 72 |
|
| 73 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
|
| 75 |
-
|
| 76 |
|
| 77 |
-
-
|
| 78 |
|
| 79 |
## Citation
|
| 80 |
-
|
| 81 |
```bibtex
|
| 82 |
-
@
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
url={https://arxiv.org/abs/2411.04125},
|
| 90 |
}
|
| 91 |
-
```
|
|
|
|
| 14 |
- deepfake_detection
|
| 15 |
- community
|
| 16 |
- opensight
|
| 17 |
+
- onnx
|
| 18 |
---
|
| 19 |
|
| 20 |
+
# CommunityForensics DeepfakeDet-ViT
|
| 21 |
|
| 22 |
+
Vision Transformer (ViT-Small) trained on 2.7M samples across 4,803 generators for detecting AI-generated images. Presented in [Community Forensics: Using Thousands of Generators to Train Fake Image Detectors](https://huggingface.co/papers/2411.04125) (CVPR 2025).
|
| 23 |
|
| 24 |
+
## IMPORTANT — Configuration Fix (July 2026)
|
| 25 |
|
| 26 |
+
**If you downloaded this model before July 22, 2026, your local copy has incorrect config files.** The model weights were always correct, but the metadata was wrong. This silently degraded results or caused loading errors.
|
| 27 |
|
| 28 |
+
| Bug | Effect | Fixed Value |
|
| 29 |
+
|---|---|---|
|
| 30 |
+
| `num_classes: 1` | `from_pretrained()` **crashes** — classifier weight is `[2,384]`, not `[1,384]` | `2` |
|
| 31 |
+
| `num_attention_heads: 12` | **Silently wrong** — attention sliced 12×32d instead of correct 6×64d | `6` |
|
| 32 |
+
| Preprocessor `size: 384` | No center-crop — images processed differently than training | `{height:440, width:440}` + `do_center_crop` |
|
| 33 |
+
| Missing `id2label` / `label2id` | Raw `[0,1]` output unreadable | `{"0":"real","1":"fake"}` |
|
| 34 |
|
| 35 |
+
### How to verify you have the fix
|
| 36 |
|
| 37 |
+
```python
|
| 38 |
+
import json
|
| 39 |
+
with open("path/to/config.json") as f:
|
| 40 |
+
cfg = json.load(f)
|
| 41 |
+
assert cfg["num_classes"] == 2, "Still broken — re-download the model"
|
| 42 |
+
assert cfg["num_attention_heads"] == 6, "Still broken — re-download the model"
|
| 43 |
+
```
|
| 44 |
|
| 45 |
+
### One-liner migration
|
| 46 |
+
|
| 47 |
+
```bash
|
| 48 |
+
# Re-download the repo to get the fixed configs
|
| 49 |
+
git clone https://huggingface.co/buildborderless/CommunityForensics-DeepfakeDet-ViT
|
| 50 |
+
# Or if you use huggingface_hub:
|
| 51 |
+
# huggingface-cli download buildborderless/CommunityForensics-DeepfakeDet-ViT --local-dir ./
|
| 52 |
+
```
|
| 53 |
+
|
| 54 |
+
### If you were using the old custom wrapper (`modeling_vit_classifier.py`)
|
| 55 |
+
|
| 56 |
+
It has been moved to `scripts/` and marked deprecated. Switch to the standard HuggingFace path:
|
| 57 |
+
|
| 58 |
+
```python
|
| 59 |
+
from transformers import ViTForImageClassification, ViTImageProcessor
|
| 60 |
+
model = ViTForImageClassification.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
|
| 61 |
+
processor = ViTImageProcessor.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
### If you were using the ONNX repo
|
| 65 |
+
|
| 66 |
+
The separate [`buildborderless/CommunityForensics-DeepfakeDet-ViT-ONNX`](https://huggingface.co/buildborderless/CommunityForensics-DeepfakeDet-ViT-ONNX) repo is now deprecated. All ONNX models (including quantized variants) are now included here in the `onnx/` directory with corrected configs.
|
| 67 |
+
|
| 68 |
+
---
|
| 69 |
+
|
| 70 |
+
## Quick Start
|
| 71 |
+
|
| 72 |
+
```python
|
| 73 |
+
from transformers import ViTForImageClassification, ViTImageProcessor
|
| 74 |
+
from PIL import Image
|
| 75 |
+
|
| 76 |
+
model = ViTForImageClassification.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
|
| 77 |
+
processor = ViTImageProcessor.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
|
| 78 |
+
|
| 79 |
+
image = Image.open("suspicious_image.jpg")
|
| 80 |
+
inputs = processor(image, return_tensors="pt")
|
| 81 |
+
outputs = model(**inputs)
|
| 82 |
+
|
| 83 |
+
import torch
|
| 84 |
+
probs = torch.softmax(outputs.logits, dim=-1)[0]
|
| 85 |
+
print(f"real: {probs[0]:.4f}, fake: {probs[1]:.4f}")
|
| 86 |
+
print(f"verdict: {model.config.id2label[torch.argmax(probs).item()]}")
|
| 87 |
+
```
|
| 88 |
|
| 89 |
+
---
|
| 90 |
+
|
| 91 |
+
## ONNX Variants
|
| 92 |
+
|
| 93 |
+
Eight pre-exported ONNX models with different size/speed/accuracy trade-offs. All use the corrected 2-class config.
|
| 94 |
+
|
| 95 |
+
| Variant | Size | Speed (CPU) | Accuracy | Best For |
|
| 96 |
+
|---|---|---|---|---|
|
| 97 |
+
| `model.onnx` (full) | 138 MB | ★★★ | ★★★★★ | Server-side, maximum accuracy |
|
| 98 |
+
| `model_fp16.onnx` | 69 MB | ★★ | ★★★★★ | GPU inference, near-lossless |
|
| 99 |
+
| `model_int8.onnx` | 36 MB | ★★★★★ | ★★★★ | Fastest CPU, balanced |
|
| 100 |
+
| `model_uint8.onnx` | 36 MB | ★★★★★ | ★★★★ | Fast CPU, unsigned integer |
|
| 101 |
+
| `model_quantized.onnx` | 36 MB | ★★★★ | ★★★★ | General CPU deployment |
|
| 102 |
+
| `model_q4.onnx` | 24 MB | ★★★ | ★★★ | Low-memory, decent accuracy |
|
| 103 |
+
| `model_bnb4.onnx` | 22 MB | ★★ | ★★★ | Constrained edge devices |
|
| 104 |
+
| `model_q4f16.onnx` | 21 MB | ★★ | ★★★ | Smallest file, mobile/web |
|
| 105 |
+
|
| 106 |
+
**Key insight**: On CPU, INT8 variants are fastest (optimized kernels). Smaller 4-bit models are slower due to dequantization overhead — use them only when disk/RAM is the bottleneck.
|
| 107 |
+
|
| 108 |
+
```python
|
| 109 |
+
import onnxruntime as ort
|
| 110 |
+
import numpy as np
|
| 111 |
+
|
| 112 |
+
session = ort.InferenceSession("onnx/model_int8.onnx")
|
| 113 |
+
# Preprocess image: resize to 440, center-crop to 384, normalize
|
| 114 |
+
image = Image.open("image.jpg").resize((440, 440))
|
| 115 |
+
# ... center-crop + normalize ...
|
| 116 |
+
output = session.run(None, {"pixel_values": input_array})
|
| 117 |
+
logits = output[0][0] # shape [2]: [real_score, fake_score]
|
| 118 |
+
```
|
| 119 |
|
| 120 |
+
---
|
| 121 |
+
|
| 122 |
+
## Model Details
|
| 123 |
+
|
| 124 |
+
- **Developed by**: Jeongsoo Park and Andrew Owens, University of Michigan
|
| 125 |
+
- **HF integration + ONNX**: Borderless
|
| 126 |
+
- **Model type**: Vision Transformer (ViT-Small)
|
| 127 |
+
- **License**: MIT
|
| 128 |
+
- **Input**: RGB image, resized to 440×440, center-cropped to 384×384, CLIP-normalized
|
| 129 |
+
- **Output**: 2-class logits `[real, fake]`
|
| 130 |
+
- **Architecture**: hidden_size=384, 6 attention heads, 12 layers, patch_size=16
|
| 131 |
|
| 132 |
### Links
|
|
|
|
|
|
|
|
|
|
| 133 |
|
| 134 |
+
- **Original paper**: [arXiv:2411.04125](https://arxiv.org/pdf/2411.04125)
|
| 135 |
+
- **Original repository**: [JeongsooP/Community-Forensics](https://github.com/JeongsooP/Community-Forensics)
|
| 136 |
+
- **Project page**: https://jespark.net/projects/2024/community_forensics
|
| 137 |
+
- **Datasets**: [Full (1.1TB)](https://huggingface.co/datasets/OwensLab/CommunityForensics), [Small (278GB)](https://huggingface.co/datasets/OwensLab/CommunityForensics-Small), [Eval (206GB)](https://huggingface.co/datasets/OwensLab/CommunityForensics-Eval)
|
| 138 |
|
| 139 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 140 |
|
| 141 |
+
## Local Test App
|
|
|
|
|
|
|
| 142 |
|
| 143 |
+
A local Gradio app for testing PyTorch and ONNX models side by side with benchmarks:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 144 |
|
| 145 |
+
```bash
|
| 146 |
+
git clone https://huggingface.co/buildborderless/CommunityForensics-DeepfakeDet-ViT
|
| 147 |
+
cd CommunityForensics-DeepfakeDet-ViT
|
| 148 |
+
pip install -r test_app/requirements.txt
|
| 149 |
+
python test_app/app.py
|
| 150 |
+
```
|
| 151 |
|
| 152 |
+
Three tabs: PyTorch vs ONNX comparison, Benchmark (all 8 ONNX variants on test images), and help.
|
| 153 |
|
| 154 |
+
---
|
| 155 |
|
| 156 |
## Citation
|
| 157 |
+
|
| 158 |
```bibtex
|
| 159 |
+
@InProceedings{Park_2025_CVPR,
|
| 160 |
+
author = {Park, Jeongsoo and Owens, Andrew},
|
| 161 |
+
title = {Community Forensics: Using Thousands of Generators to Train Fake Image Detectors},
|
| 162 |
+
booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)},
|
| 163 |
+
month = {June},
|
| 164 |
+
year = {2025},
|
| 165 |
+
pages = {8245-8257}
|
|
|
|
| 166 |
}
|
| 167 |
+
```
|