kikori

Target-directed sentiment for rashomon: given a person and a Portuguese text, how does the text treat that person? Trained and documented in walteraandrade/kikori.

Contract

Read it from config.json["kikori"] rather than copying it:

• Input: the pair (person, text) encoded as [CLS] person [SEP] text [SEP], token_type_ids 0 for the first segment and 1 for the second, max_length 256. Cut only the text so the closing [SEP] stays; truncation: true on a pair in transformers.js drops it and moves the score.
• Output: 3 logits in the order neg, neu, pos. Score = (p_pos - p_neg) * 10, range -10..10; class cut neg <= -2.5, pos >= 2.5. A temperature of 3.5 is already folded into the classifier weights.
• Files: onnx/model.onnx (fp32, 436 MB) and onnx/model_quantized.onnx (per-channel dynamic int8, 110 MB) for @huggingface/transformers (dtype: "fp32" / "q8"); model.safetensors for Python.
• fixtures.json: 24 pairs with the expected fp32 and int8 scores. fp32 should match to 1e-3 on any runtime; int8 drifts between runtimes, up to 0.43 from Python to Node on these fixtures, so test it with a tolerance.

Numbers

Holdout of 469 (person, text) pairs labelled by an LLM teacher under the rules in the repo's LABELLING.md, teacher-human agreement ~0.73. pos recall is by the score cut, over the 45 pos pairs:

	acc	MAE	pos recall
fp32	0.770	1.687	0.47 (21/45)
int8	0.774	1.691	0.47 (21/45)
constant 0	0.59	2.16	0.00
previous release (2026-09-08), int8	0.757	1.767	0.18 (8/45)

Recipe: BERTimbau base, effective batch 32, lr 5e-5, 3 epochs with the last one kept, class-weighted cross-entropy, temperature calibration on a frozen validation split. Trained on 11182 teacher-labelled pairs; validation is 1353 of them, held out by document and frozen across the whole data loop (macro-F1 0.7666, pos recall 0.706).

pos is still the weak class, and the gain over the previous release is concentrated there. Overall holdout accuracy did not move in fp32 (0.772 -> 0.770): what changed is that the model now finds 21 of the 45 pos pairs instead of 15, and that int8 no longer destroys that recall — the previous release fell from 15/45 in fp32 to 8/45 in int8, this one does not move.

The gain came from choosing which pairs the teacher labels, with the training recipe frozen: two batches kept out of three judged, one targeting the people who almost never get a pos label, one targeting the highest p_pos pairs. On the frozen validation that is macro-F1 0.7305 -> 0.7630 and pos recall 0.6580 -> 0.6944, averaged over 5 training runs each.

Known bias

The person name acts as a prior learned from skewed training labels (Lula's pairs lean pos, Tarcísio's and the Bolsonaros' lean neg). It is still here. On the same short hostile sentence with the name swapped, this model scores Lula -0.3 and everyone else -4.8 to -4.9 (previous release: Lula +1.9, others -4.0 to -4.8). The gap narrowed on that sentence and did not narrow on two other hostile ones (spreads 2.4 and 4.0, against 2.8 and 3.9 before). Favourable and routine sentences show no name effect in either model (spread 0.6 and 0.2). Longer texts are read on their content.

Compare outlets on the same person, not people against each other. See the repository README, "Known bias".

This model is a ruler, not a judge. It is biased; the requirement is that it is biased the same way for every outlet, so comparisons between outlets stay valid.