# Inference of bert-base with sst2 dataset

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-of-bert-base-with-sst2-dataset%2F46461%3E%5C
**Category:** Beginners\
**Created:** [July 11, 2023, 4:57pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-of-bert-base-with-sst2-dataset%2F46461 "2023-07-11T16:57:51Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![GhadeerAli](https://avatars.discourse-cdn.com/v4/letter/g/aeb1de/32.png) [@GhadeerAli](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FGhadeerAli)%5C
**Post date:** [July 11, 2023, 4:57pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-of-bert-base-with-sst2-dataset%2F46461%2F1 "2023-07-11T16:57:52Z")

</div>

run **bert-base-uncased** with **sst2** dataset, but all predictions are 1!!

from transformers import AutoModelForSequenceClassification, AutoTokenizer  
import datasets  
import torch  
from sklearn.metrics import accuracy\_score  
from transformers import pipeline  
from transformers.pipelines.pt\_utils import KeyDataset  
from datasets import load\_dataset  
import evaluate

# load pretrained model

model\_name=‘bert-base-uncased’  
model = AutoModelForSequenceClassification.from\_pretrained(model\_name)

# load the tokenixer for the pretrained model

tokenizer = AutoTokenizer.from\_pretrained(model\_name)

# create the pieline

generator = pipeline(task=‘sentiment-analysis’, model=model, tokenizer=tokenizer)

# load dataset

dataset = load\_dataset(“sst2”,split=“test”)

# pass the dataset to the pipeline

results = generator(KeyDataset(dataset, “sentence”))

# Extract true labels from the datasePrt

true\_labels = dataset[“label”]  
print(“True labels”, true\_labels)  
print(“TrueLabelSize=”,len(true\_labels))

# Extract the labels from the pipeline results

pred\_labels= [0 if result[“label”]==“LABEL\_0” else 1 for result in results]  
print(“PredLabels”,pred\_labels)  
print(“predLabelSize=”,len(pred\_labels))

# Load the accuracy metric

accuracy\_metric = evaluate.load(“accuracy”)

# Compute accuracy

accuracy = accuracy\_metric.compute(predictions=pred\_labels, references=true\_labels)

print(“Accuracy:”, accuracy)
