ethix commited on
Commit
4c85d93
·
1 Parent(s): eebc18e

docs: overhaul README — fix notice, ONNX variant guide, test app, deprecate old ONNX repo

Browse files
Files changed (2) hide show
  1. LICENSE +22 -0
  2. README.md +127 -51
LICENSE ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2024 Jeongsoo Park, Andrew Owens, University of Michigan
4
+ Copyright (c) 2025 Borderless (HuggingFace integration, ONNX exports, configuration)
5
+
6
+ Permission is hereby granted, free of charge, to any person obtaining a copy
7
+ of this software and associated documentation files (the "Software"), to deal
8
+ in the Software without restriction, including without limitation the rights
9
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
10
+ copies of the Software, and to permit persons to whom the Software is
11
+ furnished to do so, subject to the following conditions:
12
+
13
+ The above copyright notice and this permission notice shall be included in all
14
+ copies or substantial portions of the Software.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
17
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
18
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
19
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
20
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
21
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
22
+ SOFTWARE.
README.md CHANGED
@@ -14,78 +14,154 @@ tags:
14
  - deepfake_detection
15
  - community
16
  - opensight
 
17
  ---
18
 
19
- # NOTICE: 7/22/2026
20
 
21
- - Ungated again. Friendly reminder to all to please respect the MIT license. Attribution should be made to the original project (see bottom) as well as this repository. Other than that, you're free to use the model as you wish.
22
 
23
- ## Trained on 2.7M samples across 4,803 generators (see Training Data)
24
 
25
- Model presented in [Community Forensics: Using Thousands of Generators to Train Fake Image Detectors](https://huggingface.co/papers/2411.04125).
26
 
27
- **Uploaded for community validation as part of OpenSight** - An upcoming open-source framework for adaptive deepfake detection.
 
 
 
 
 
28
 
29
- **Project OpenSight HF Spaces coming soon with an eval playground and eventually a leaderboard. Preview:**
30
 
31
- ![image/png](https://cdn-uploads.huggingface.co/production/uploads/639daf827270667011153fbc/AUmW697OefKN83BClM1ae.png)
 
 
 
 
 
 
32
 
33
- ## Model Details
34
- ### Model Description
35
- Vision Transformer (ViT) model trained on the largest dataset to-date for detecting AI-generated images in forensic applications.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36
 
37
- - **Developed by:** Jeongsoo Park and Andrew Owens, University of Michigan
38
- - **Model type:** Vision Transformer (ViT-Small)
39
- - **License:** MIT (compatible with CreativeML OpenRAIL-M referenced in [2411.04125v1.pdf])
40
- - **Finetuned from:** timm/vit_small_patch16_384.augreg_in21k_ft_in1k
41
- - **Adapted for HF** inference compatibility by Borderless.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
42
 
43
- **HF Space will be open sourced shortly showcasing various ways to run ultra-fast inference. Make sure to follow us for updates, as we will be releasing a slew of projects in the coming weeks.**
 
 
 
 
 
 
 
 
 
 
44
 
45
  ### Links
46
- - **Repository:** [JeongsooP/Community-Forensics](https://github.com/JeongsooP/Community-Forensics)
47
- - **Paper:** [arXiv:2411.04125](https://arxiv.org/pdf/2411.04125)
48
- - **Project Page:** https://jespark.net/projects/2024/community_forensics
49
 
50
- ## Training Details
51
- ### Training Data
52
- - 2.7mil images from 15+ generators, 4600+ models
53
- - Over 1.15TB worth of images
54
 
55
- ### Training Hyperparameters
56
- - **Framework:** PyTorch 2.0
57
- - **Precision:** bf16 mixed
58
- - **Optimizer:** AdamW (lr=5e-5)
59
- - **Epochs:** 10
60
- - **Batch Size:** 32
61
 
62
- ## Evaluation
63
- ### Unverified Testing Results
64
- - Only unverified because we currently lack resources to evaluate a dataset over 1.4T large.
65
 
66
- | Metric | Value |
67
- |---------------|-------|
68
- | Accuracy | 97.2% |
69
- | F1 Score | 0.968 |
70
- | AUC-ROC | 0.992 |
71
- | FP Rate | 2.1% |
72
 
73
- ![image/png](https://cdn-uploads.huggingface.co/production/uploads/639daf827270667011153fbc/g-dLzxLBw1RAuiplvFCxh.png)
 
 
 
 
 
74
 
75
- ## Re-sampled and refined dataset
76
 
77
- - **Coming soon™**
78
 
79
  ## Citation
80
- **BibTeX:**
81
  ```bibtex
82
- @misc{park2024communityforensics,
83
- title={Community Forensics: Using Thousands of Generators to Train Fake Image Detectors},
84
- author={Jeongsoo Park and Andrew Owens},
85
- year={2024},
86
- eprint={2411.04125},
87
- archivePrefix={arXiv},
88
- primaryClass={cs.CV},
89
- url={https://arxiv.org/abs/2411.04125},
90
  }
91
- ```
 
14
  - deepfake_detection
15
  - community
16
  - opensight
17
+ - onnx
18
  ---
19
 
20
+ # CommunityForensics DeepfakeDet-ViT
21
 
22
+ Vision Transformer (ViT-Small) trained on 2.7M samples across 4,803 generators for detecting AI-generated images. Presented in [Community Forensics: Using Thousands of Generators to Train Fake Image Detectors](https://huggingface.co/papers/2411.04125) (CVPR 2025).
23
 
24
+ ## IMPORTANT Configuration Fix (July 2026)
25
 
26
+ **If you downloaded this model before July 22, 2026, your local copy has incorrect config files.** The model weights were always correct, but the metadata was wrong. This silently degraded results or caused loading errors.
27
 
28
+ | Bug | Effect | Fixed Value |
29
+ |---|---|---|
30
+ | `num_classes: 1` | `from_pretrained()` **crashes** — classifier weight is `[2,384]`, not `[1,384]` | `2` |
31
+ | `num_attention_heads: 12` | **Silently wrong** — attention sliced 12×32d instead of correct 6×64d | `6` |
32
+ | Preprocessor `size: 384` | No center-crop — images processed differently than training | `{height:440, width:440}` + `do_center_crop` |
33
+ | Missing `id2label` / `label2id` | Raw `[0,1]` output unreadable | `{"0":"real","1":"fake"}` |
34
 
35
+ ### How to verify you have the fix
36
 
37
+ ```python
38
+ import json
39
+ with open("path/to/config.json") as f:
40
+ cfg = json.load(f)
41
+ assert cfg["num_classes"] == 2, "Still broken — re-download the model"
42
+ assert cfg["num_attention_heads"] == 6, "Still broken — re-download the model"
43
+ ```
44
 
45
+ ### One-liner migration
46
+
47
+ ```bash
48
+ # Re-download the repo to get the fixed configs
49
+ git clone https://huggingface.co/buildborderless/CommunityForensics-DeepfakeDet-ViT
50
+ # Or if you use huggingface_hub:
51
+ # huggingface-cli download buildborderless/CommunityForensics-DeepfakeDet-ViT --local-dir ./
52
+ ```
53
+
54
+ ### If you were using the old custom wrapper (`modeling_vit_classifier.py`)
55
+
56
+ It has been moved to `scripts/` and marked deprecated. Switch to the standard HuggingFace path:
57
+
58
+ ```python
59
+ from transformers import ViTForImageClassification, ViTImageProcessor
60
+ model = ViTForImageClassification.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
61
+ processor = ViTImageProcessor.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
62
+ ```
63
+
64
+ ### If you were using the ONNX repo
65
+
66
+ The separate [`buildborderless/CommunityForensics-DeepfakeDet-ViT-ONNX`](https://huggingface.co/buildborderless/CommunityForensics-DeepfakeDet-ViT-ONNX) repo is now deprecated. All ONNX models (including quantized variants) are now included here in the `onnx/` directory with corrected configs.
67
+
68
+ ---
69
+
70
+ ## Quick Start
71
+
72
+ ```python
73
+ from transformers import ViTForImageClassification, ViTImageProcessor
74
+ from PIL import Image
75
+
76
+ model = ViTForImageClassification.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
77
+ processor = ViTImageProcessor.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
78
+
79
+ image = Image.open("suspicious_image.jpg")
80
+ inputs = processor(image, return_tensors="pt")
81
+ outputs = model(**inputs)
82
+
83
+ import torch
84
+ probs = torch.softmax(outputs.logits, dim=-1)[0]
85
+ print(f"real: {probs[0]:.4f}, fake: {probs[1]:.4f}")
86
+ print(f"verdict: {model.config.id2label[torch.argmax(probs).item()]}")
87
+ ```
88
 
89
+ ---
90
+
91
+ ## ONNX Variants
92
+
93
+ Eight pre-exported ONNX models with different size/speed/accuracy trade-offs. All use the corrected 2-class config.
94
+
95
+ | Variant | Size | Speed (CPU) | Accuracy | Best For |
96
+ |---|---|---|---|---|
97
+ | `model.onnx` (full) | 138 MB | ★★★ | ★★★★★ | Server-side, maximum accuracy |
98
+ | `model_fp16.onnx` | 69 MB | ★★ | ★★★★★ | GPU inference, near-lossless |
99
+ | `model_int8.onnx` | 36 MB | ★★★★★ | ★★★★ | Fastest CPU, balanced |
100
+ | `model_uint8.onnx` | 36 MB | ★★★★★ | ★★★★ | Fast CPU, unsigned integer |
101
+ | `model_quantized.onnx` | 36 MB | ★★★★ | ★★★★ | General CPU deployment |
102
+ | `model_q4.onnx` | 24 MB | ★★★ | ★★★ | Low-memory, decent accuracy |
103
+ | `model_bnb4.onnx` | 22 MB | ★★ | ★★★ | Constrained edge devices |
104
+ | `model_q4f16.onnx` | 21 MB | ★★ | ★★★ | Smallest file, mobile/web |
105
+
106
+ **Key insight**: On CPU, INT8 variants are fastest (optimized kernels). Smaller 4-bit models are slower due to dequantization overhead — use them only when disk/RAM is the bottleneck.
107
+
108
+ ```python
109
+ import onnxruntime as ort
110
+ import numpy as np
111
+
112
+ session = ort.InferenceSession("onnx/model_int8.onnx")
113
+ # Preprocess image: resize to 440, center-crop to 384, normalize
114
+ image = Image.open("image.jpg").resize((440, 440))
115
+ # ... center-crop + normalize ...
116
+ output = session.run(None, {"pixel_values": input_array})
117
+ logits = output[0][0] # shape [2]: [real_score, fake_score]
118
+ ```
119
 
120
+ ---
121
+
122
+ ## Model Details
123
+
124
+ - **Developed by**: Jeongsoo Park and Andrew Owens, University of Michigan
125
+ - **HF integration + ONNX**: Borderless
126
+ - **Model type**: Vision Transformer (ViT-Small)
127
+ - **License**: MIT
128
+ - **Input**: RGB image, resized to 440×440, center-cropped to 384×384, CLIP-normalized
129
+ - **Output**: 2-class logits `[real, fake]`
130
+ - **Architecture**: hidden_size=384, 6 attention heads, 12 layers, patch_size=16
131
 
132
  ### Links
 
 
 
133
 
134
+ - **Original paper**: [arXiv:2411.04125](https://arxiv.org/pdf/2411.04125)
135
+ - **Original repository**: [JeongsooP/Community-Forensics](https://github.com/JeongsooP/Community-Forensics)
136
+ - **Project page**: https://jespark.net/projects/2024/community_forensics
137
+ - **Datasets**: [Full (1.1TB)](https://huggingface.co/datasets/OwensLab/CommunityForensics), [Small (278GB)](https://huggingface.co/datasets/OwensLab/CommunityForensics-Small), [Eval (206GB)](https://huggingface.co/datasets/OwensLab/CommunityForensics-Eval)
138
 
139
+ ---
 
 
 
 
 
140
 
141
+ ## Local Test App
 
 
142
 
143
+ A local Gradio app for testing PyTorch and ONNX models side by side with benchmarks:
 
 
 
 
 
144
 
145
+ ```bash
146
+ git clone https://huggingface.co/buildborderless/CommunityForensics-DeepfakeDet-ViT
147
+ cd CommunityForensics-DeepfakeDet-ViT
148
+ pip install -r test_app/requirements.txt
149
+ python test_app/app.py
150
+ ```
151
 
152
+ Three tabs: PyTorch vs ONNX comparison, Benchmark (all 8 ONNX variants on test images), and help.
153
 
154
+ ---
155
 
156
  ## Citation
157
+
158
  ```bibtex
159
+ @InProceedings{Park_2025_CVPR,
160
+ author = {Park, Jeongsoo and Owens, Andrew},
161
+ title = {Community Forensics: Using Thousands of Generators to Train Fake Image Detectors},
162
+ booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)},
163
+ month = {June},
164
+ year = {2025},
165
+ pages = {8245-8257}
 
166
  }
167
+ ```