# How to fine-tune Bert on STS-B task?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-fine-tune-bert-on-sts-b-task%2F8950%3E%5C
**Category:** Beginners\
**Created:** [August 4, 2021, 7:26am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-fine-tune-bert-on-sts-b-task%2F8950 "2021-08-04T07:26:07Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![PososAgapo](https://avatars.discourse-cdn.com/v4/letter/p/74df32/32.png) [@PososAgapo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FPososAgapo)%5C
**Post date:** [August 4, 2021, 7:26am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-fine-tune-bert-on-sts-b-task%2F8950%2F1 "2021-08-04T07:26:07Z")

</div>

Hi, I am new to NLP and trying to reproduce fine-tune results of Bert. However, the STS-B task troubles me, from what I understand, the STS-B task is a regression task, but Bert treats it as a classification task. I do not quite know the transformation between scores and labels in detail, is anybody willing to give me a hint?

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [August 4, 2021, 7:54am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-fine-tune-bert-on-sts-b-task%2F8950%2F2 "2021-08-04T07:54:06Z")

</div>

This is all dealt with in the loss function: a model that is tasked with classification or regression is the same roughly, it just outputs a different number of labels. Inside the code of [`BertModelForSequenceClassification`](https://github.com/huggingface/transformers/blob/a28da4c4901c775be724ca1cec79ace32e6e80ee/src/transformers/models/bert/modeling_bert.py#L1543), you can see there is a test that picks a different loss function depending on the `problem_type`, and by default 1 label (like in STS-B) corresponds to a regression, so the mean-squared error is selected as a loss, instead of cross-entropy.

---

<div class="post-metadata">

**Author:** ![PososAgapo](https://avatars.discourse-cdn.com/v4/letter/p/74df32/32.png) [@PososAgapo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FPososAgapo)%5C
**Post date:** [August 4, 2021, 8:00am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-fine-tune-bert-on-sts-b-task%2F8950%2F4 "2021-08-04T08:00:41Z")

</div>

Thank you for your detailed reply, it saves me 😉
