# How does from\_pretrained work with ZeRO=3?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-does-from-pretrained-work-with-zero-3%2F50671%3E%5C
**Category:** DeepSpeed\
**Created:** [August 14, 2023, 5:28am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-does-from-pretrained-work-with-zero-3%2F50671 "2023-08-14T05:28:00Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![suzhu001](https://avatars.discourse-cdn.com/v4/letter/s/cdc98d/32.png) [@suzhu001](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsuzhu001)%5C
**Post date:** [August 14, 2023, 5:28am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-does-from-pretrained-work-with-zero-3%2F50671%2F1 "2023-08-14T05:28:00Z")

</div>

Following [nontrainer-deepspeed-integration](https://huggingface.co/docs/transformers/main_classes/deepspeed#nontrainer-deepspeed-integration),

> When not using [Trainer], to efficiently deploy DeepSpeed ZeRO-3, you must instantiate the [HfDeepSpeedConfig] object before instantiating the model and keep that object alive.

, I found that params are partitioned in different ranks. However, if I want to resize tokenizer, can I still use `resize_token_embeddings` after `from_pretrained`?