# AutoTrain Error DeepSpeed Zero-3

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautotrain-error-deepspeed-zero-3%2F103287%3E%5C
**Category:** DeepSpeed\
**Created:** [August 21, 2024, 9:26am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautotrain-error-deepspeed-zero-3%2F103287 "2024-08-21T09:26:32Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![ewwefqreger](https://avatars.discourse-cdn.com/v4/letter/e/258eb7/32.png) [@ewwefqreger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fewwefqreger)%5C
**Post date:** [August 21, 2024, 9:26am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautotrain-error-deepspeed-zero-3%2F103287%2F1 "2024-08-21T09:26:32Z")

</div>

I am trying to finetune Mixtral 8x7b using Hugging Face AutoTrain.

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/3X/4/7/47a176e1b21db974d9b012189e2b122f5596e00b.png)

I get this error message. Can you help me?

> Generating train split: 0 examples [00:00, ? examples/s]  
> Generating train split: 44 examples [00:00, 830.23 examples/s]  
> ERROR | 2024-08-21 09:19:18 | autotrain.trainers.common:wrapper:120 - train has failed due to an exception: Traceback (most recent call last):  
> File “/app/env/lib/python3.10/site-packages/autotrain/trainers/common.py”, line 117, in wrapper  
> return func(\*args, \*\*kwargs)  
> File “/app/env/lib/python3.10/site-packages/autotrain/trainers/clm/ **main**.py”, line 28, in train  
> train\_sft(config)  
> File “/app/env/lib/python3.10/site-packages/autotrain/trainers/clm/train\_clm\_sft.py”, line 46, in train  
> trainer = SFTTrainer(  
> File “/app/env/lib/python3.10/site-packages/huggingface\_hub/utils/\_deprecation.py”, line 101, in inner\_f  
> return f(\*args, \*\*kwargs)  
> File “/app/env/lib/python3.10/site-packages/trl/trainer/sft\_trainer.py”, line 413, in **init**  
> super(). **init** (  
> File “/app/env/lib/python3.10/site-packages/transformers/trainer.py”, line 444, in **init**  
> raise ValueError(  
> ValueError: Model was not initialized with `Zero-3` despite being configured for DeepSpeed Zero-3. Please re-initialize your model via `Model.from_pretrained(...)` or `Model.from_config(...)` after creating your `TrainingArguments`!
> 
> ERROR | 2024-08-21 09:19:18 | autotrain.trainers.common:wrapper:121 - Model was not initialized with `Zero-3` despite being configured for DeepSpeed Zero-3. Please re-initialize your model via `Model.from_pretrained(...)` or `Model.from_config(...)` after creating your `TrainingArguments`!  
> INFO | 2024-08-21 09:19:18 | autotrain.trainers.common:pause\_space:77 - Pausing space…

---

<div class="post-metadata">

**Author:** ![mahmutc](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mahmutc/32/64626_2.png) [@mahmutc](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmahmutc)%5C
**Post date:** [August 21, 2024, 9:51am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautotrain-error-deepspeed-zero-3%2F103287%2F2 "2024-08-21T09:51:26Z")

</div>

it seems similar to [Running on Multiple GPU with DeepSpeed. Error: Model was not initialized with Zero-3 despite being configured for Deepspeed Zero-3. Please re-initialize your model via Model.from\_pretrained or Model.from\_config after creating your TrainingArguments! · Issue #32901 · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/issues/32901).