reddgr
/

zero-shot-prompt-classifier-bart-ft

Text Classification

generated_from_keras_callback

Model card Files Files and versions

reddgr commited on Dec 16, 2024

Commit

2b45bcc

·

verified ·

1 Parent(s): 76acbe2

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -26,7 +26,7 @@ Below is a confusion matrix calculated on zero-shot inferences for the 10 most p
 The current version of the fine-tuned model outperforms the base model [facebook/bart-large-mnli](https://huggingface.co/facebook/bart-large-mnli) by 23 percentage points (57% accuracy vs 34% accuracy) in a test set with 10 candidate zero-shot classes (the most frequent categories in the test split of [reddgr/nli-chatbot-prompt-categorization](https://huggingface.co/datasets/reddgr/nli-chatbot-prompt-categorization)).
-The chart below compares the results for the 12 most popular candidate classes in the Test split:
 ![Zero-shot prompt classification confusion matrix for reddgr/zero-shot-prompt-classifier-bart-ft](https://talkingtochatbots.com/wp-content/uploads/2024/12/zero-shot-prompt-classification-comparison-12-classes-56-accuracy.png)

 The current version of the fine-tuned model outperforms the base model [facebook/bart-large-mnli](https://huggingface.co/facebook/bart-large-mnli) by 23 percentage points (57% accuracy vs 34% accuracy) in a test set with 10 candidate zero-shot classes (the most frequent categories in the test split of [reddgr/nli-chatbot-prompt-categorization](https://huggingface.co/datasets/reddgr/nli-chatbot-prompt-categorization)).
+The chart below compares the results for the 12 most popular candidate classes in the Test split, where the base model's zero-shot accuracy is outperformed by 25 percentage points:
 ![Zero-shot prompt classification confusion matrix for reddgr/zero-shot-prompt-classifier-bart-ft](https://talkingtochatbots.com/wp-content/uploads/2024/12/zero-shot-prompt-classification-comparison-12-classes-56-accuracy.png)