# Using DistributedSampler with accelerate

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-distributedsampler-with-accelerate%2F148474%3E%5C
**Category:** 🤗Transformers\
**Created:** [April 2, 2025, 2:12am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-distributedsampler-with-accelerate%2F148474 "2025-04-02T02:12:22Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [April 2, 2025, 7:53am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-distributedsampler-with-accelerate%2F148474%2F2 "2025-04-02T07:53:12Z")

</div>

There may be no advantage to explicitly using DistributedSampler…

> [@Can accelerator handle the distributed sampler?](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcan-accelerator-handle-the-distributed-sampler%2F12943)%3A
>
> As far as I know, for Pytorch, RandomSampler can not be directly used in the distributed data parallel training since DistributedSampler is desired (this [link](https://discuss.pytorch.org/t/how-to-use-my-own-sampler-when-i-already-use-distributedsampler/62143) discusses the problem). I am wondering whether accelerator.prepare(dataloader) handles the data split for multiple GPUs if I use the RandomSampler, so that the sub-dataset on each device are exclusive.

> You don’t have to worry about using a distributed sampler with Accelerate. Whatever your sampler is, Accelerate will automatically shard it for all processes.

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-distributedsampler-with-accelerate%2F148474)._