# How can I load an LLM in 4-bits

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-can-i-load-an-llm-in-4-bits%2F49100%3E%5C
**Category:** 🤗Transformers\
**Created:** [August 2, 2023, 8:00am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-can-i-load-an-llm-in-4-bits%2F49100 "2023-08-02T08:00:08Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![shamanez](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shamanez/32/17994_2.png) [@shamanez](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fshamanez)%5C
**Post date:** [August 2, 2023, 8:00am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-can-i-load-an-llm-in-4-bits%2F49100%2F1 "2023-08-02T08:00:08Z")

</div>

I am working with [this](https://github.com/huggingface/peft/blob/main/examples/fp4_finetuning/finetune_fp4_opt_bnb_peft.py#L46) PEFT example of finetuning a model with 4 bits.

Although I use **load\_in\_4bit=True** , I get a log message as follows.

> Detected 8-bit loading: activating 8-bit loading for this model

Can someone please explain this to me?

My GPU is Tesla T4 16Gb
