# Deepspeed ZeRO2, PEFT, bitsnbytes training

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdeepspeed-zero2-peft-bitsnbytes-training%2F90331%3E%5C
**Category:** DeepSpeed\
**Created:** [June 4, 2024, 6:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdeepspeed-zero2-peft-bitsnbytes-training%2F90331 "2024-06-04T18:56:46Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![cdoern](https://avatars.discourse-cdn.com/v4/letter/c/f08c70/32.png) [@cdoern](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcdoern)%5C
**Post date:** [June 4, 2024, 6:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdeepspeed-zero2-peft-bitsnbytes-training%2F90331%2F1 "2024-06-04T18:56:46Z")

</div>

How can one train with a bitsnbytes quantized model, a PEFT config, and DeepSpeed? The docs don’t seem to cover this.

My training finishes successfully but then I have no pytorch\_model.bin. Because I am using PEFT, is there no `.bin`? Really just looking for a way to get the trained model, merge whatever adapters I need and produce a some final directory 🙏