# Accelerator OOM

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-oom%2F41838%3E%5C
**Category:** 🤗Accelerate\
**Created:** [June 2, 2023, 12:53am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-oom%2F41838 "2023-06-02T00:53:43Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![adamits](https://avatars.discourse-cdn.com/v4/letter/a/9de053/32.png) [@adamits](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fadamits)%5C
**Post date:** [June 2, 2023, 12:53am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-oom%2F41838%2F1 "2023-06-02T00:53:43Z")

</div>

Hi,

I am using `Accelerate` to run distributed inference (in which I use scores from a pretrained model to do other things in a program). Currently I am getting an OOM error pretty far into my eval dataset. Since I have a fixed batch size and pad all batches to the same max sequence length, it seems like the model/a particular batch being too big are not the problem.

I was hoping someone could help me understand more clearly what happens under the hood so I can debug.

When I run:

```auto
accelerator = Accelerator()
dataloader, NLI_model = accelerator.prepare(
     dataloader, NLI_model
)

```

1. Does the entire dataloader get put on GPU by the prepare method? So if I store a bunch of samples on my dataset, do these sit in GPU RAM? Or do they get put on GPU in the collator?
2. Since I am running on several datasets, I call `accelerator.prepare` many times. Could this be somehow accumulating memory?
  - Should I put everything in one dataset first?
  - Should I be using the same accelerator, and if so, do I need to do something to clear the old dataloader before adding a new one?

Thanks!

---

<div class="post-metadata">

**Author:** ![smangrul](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/smangrul/32/3791_2.png) [@smangrul](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsmangrul)%5C
**Post date:** [June 23, 2023, 9:48am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-oom%2F41838%2F2 "2023-06-23T09:48:47Z")

</div>

Please prepare model only once, let us know if that helps

---

<div class="post-metadata">

**Author:** ![adamits](https://avatars.discourse-cdn.com/v4/letter/a/9de053/32.png) [@adamits](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fadamits)%5C
**Post date:** [July 5, 2023, 9:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-oom%2F41838%2F3 "2023-07-05T21:56:32Z")

</div>

It does not help. I suspect this does not impact memory usage at all, though perhaps it gives me a slight benefit to runtime?