# Questions about steps with gradient accumulation

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestions-about-steps-with-gradient-accumulation%2F18597%3E%5C
**Category:** Beginners\
**Created:** [June 1, 2022, 7:47am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestions-about-steps-with-gradient-accumulation%2F18597 "2022-06-01T07:47:13Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![shelf](https://avatars.discourse-cdn.com/v4/letter/s/bc8723/32.png) [@shelf](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fshelf)%5C
**Post date:** [June 1, 2022, 7:47am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestions-about-steps-with-gradient-accumulation%2F18597%2F1 "2022-06-01T07:47:13Z")

</div>

For example, I use batch size=64, max steps=1000, log, evaluate and save steps=100.

Question 1: I want to confirm that if I use gradient accumulation (batch size=32, gradient accumulation steps=2), I don’t need to change step args (1000 → 2000, 100 → 200). Will transformers take care of gradient\_accumulation\_steps \* xxx\_step totally? I found the [gradient\_accumulation\_steps document](https://huggingface.co/docs/transformers/main_classes/trainer#transformers.TrainingArguments.gradient_accumulation_steps) and [this discussion](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fexplicitly-set-number-of-training-steps-using-trainer%2F1127%2F2). But I didn’t find any exact example.

Question 2: How about warm-up? Do I just keep the original args config?

Question 3: Will the data be fed into model in the same order? (I don’t set seed with `--seed`, so the default seed 42 is used)

---

<div class="post-metadata">

**Author:** ![logvinata](https://avatars.discourse-cdn.com/v4/letter/l/22d042/32.png) [@logvinata](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flogvinata)%5C
**Post date:** [July 19, 2023, 11:47am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestions-about-steps-with-gradient-accumulation%2F18597%2F2 "2023-07-19T11:47:00Z")

</div>

Hi, @shelf! Did you find the answers?