Instructions to use brenindexdev/crivo-factual-fakebr with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Scikit-learn
How to use brenindexdev/crivo-factual-fakebr with Scikit-learn:
from huggingface_hub import hf_hub_download import joblib model = joblib.load( hf_hub_download("brenindexdev/crivo-factual-fakebr", "sklearn_model.joblib") ) # only load pickle files from sources you trust # read more about it here https://skops.readthedocs.io/en/stable/persistence.html - Notebooks
- Google Colab
- Kaggle
Crivo Factual β Veracity classifiers (v1 + v2 + unified)
Three classifiers that estimate the probability that a text in Brazilian Portuguese is fake:
- v1 β article specialist. Trained on the Fake.Br Corpus (Monteiro et al., 2018) β 7,200 news items, 3,600 fake + 3,600 true, in aligned pairs about the same fact. Only scores texts of 500 characters or more.
- v2 β generalist. Trained on FakenewsBR (Silva et al., 2026) β 36,594 canonical items: news, tweets, WhatsApp, COVID-19. Scores short text, but with a false-positive bias.
- unified β one model for any length. Trained on Fake.Br + FakenewsBR (35,035 items). Articles F1 0.9646, mixed F1 0.7924 β the best single model in the project.
Model: TF-IDF (unigrams + bigrams, sublinear, 50k features,
Portuguese stopwords) + TF-IDF of char n-grams (3β5, 30k) +
logistic regression (C=4.0 on the unified model; C=1.0 on v1/v2).
The artifact is the full pipeline (vectorizer + model), saved
with joblib. model_v1.joblib, model_v2.joblib and
model_unified.joblib are the three artifacts; model.joblib
(v1) is kept for compatibility.
Measured results
| Metric | v1 (articles, n=1440) | v2 (mixed, n=7319) | v2 on the v1 articles | unified (articles) | unified (mixed) | BERTimbau (articles) | BERTimbau (mixed) |
|---|---|---|---|---|---|---|---|
| Accuracy | 0.9708 | 0.8086 | 0.8854 | 0.9646 | 0.8386 | 0.9854 | 0.8561 |
| F1-macro | 0.9708 | 0.7676 | 0.8841 | 0.9646 | 0.7924 | 0.9854 | 0.8243 |
| ROC-AUC | 0.9960 | 0.8575 | 0.9888 | β | β | β | β |
| Brier | 0.0418 | 0.1444 | 0.0926 | β | β | β | β |
Real BERTimbau training on 17/11/2026, Kaggle T4 GPU (free
Colab had no GPU available). Same splits (seed 42) and same
tests as the linear models β the numbers are comparable. The
trained model weights are published in this repository, under
bertimbau/ (configuration, weights, tokenizer and metrics),
uploaded from the Kaggle session on 17/11/2026 (re-run of the
training; first run: articles 0.9868 / mixed 0.8184).
The v2 vs v1 comparison on articles is clean: the 1,440 items of the v1 test set were excluded from v2 training by fingerprint. The unified model uses the same splits (seed 42) and the same tests.
Usage (router)
import joblib
v1 = joblib.load("model_v1.joblib")
v2 = joblib.load("model_v2.joblib")
def suspicion(text):
if len(text.strip()) >= 500:
return float(v1.predict_proba([text])[0][1]), "v1"
return float(v2.predict_proba([text])[0][1]), "v2"
Usage (unified β one model, any length)
import joblib
uni = joblib.load("model_unified.joblib")
def suspicion(text):
return float(uni.predict_proba([text])[0][1])
Limits β read before using
- v2 does not replace v1. On full articles v1 is still better (F1 0.9708 against 0.8841). v2 is worth it for breadth (tweets, WhatsApp, COVID) and short text.
- BERTimbau is the best model on both tests, but mixed
stays below 0.95. Articles F1 0.9854 (goal beaten),
mixed F1 0.8243. Mixed clears the measured linear ceiling
without the worst sources (0.8183) by 0.6 points, on the full
set β the bottleneck is the data, not the model. See
crivo/ml/README.md. - False positives by source (v2). The score depends on the source: 5% on political news, 9% on Fake.Br articles, 22% on WhatsApp, 36% on COVID, 53% on MuMiN tweets, 58% on the true Kaggle news. Read it as a weak signal, not as a probability.
- Short text (v2). Short true text scores 0.46 and short fake 0.72 on average β it separates, but an ordinary true sentence can still score 0.80. The 500-character guard on v1 still applies to articles.
- Calibration not shipped. Isotonic calibration improves Brier, but shifts the operating point toward "fake" (false positives from 590 to 1,011 on the mixed test). The artifact is the model without calibration.
- It is a triage signal, not a verdict. In the Crivo Factual project, this score only prioritizes the checking queue. The verdict is still decided by deterministic rules over primary sources (CΓ’mara, Senado, DiΓ‘rio Oficial). No number from this model declares a news item fake.
- Corpus bias. Fake news has strong style markers
(sensationalism, clickbait); the model learns style as much
as content. Fake.Br is from 2016β2018; FakenewsBR runs to
Corpus citations
Monteiro R.A., Santos R.L.S., Pardo T.A.S., de Almeida T.A., Ruiz E.E.S., Vale O.A. (2018). Contributions to the Study of Fake News in Portuguese: New Corpus and Automatic Detection Results. PROPOR 2018, LNCS vol. 11122. Springer.
Silva et al. (2026). FakenewsBR: A Unified Corpus for Fake News Detection in Portuguese. DSW 2026. https://github.com/AKCIT-FN/fakenews-data
- Downloads last month
- -