# Accelerate - WeightedRandomSampler Dataloader

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerate-weightedrandomsampler-dataloader%2F92721%3E%5C
**Category:** Intermediate\
**Created:** [June 18, 2024, 8:02am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerate-weightedrandomsampler-dataloader%2F92721 "2024-06-18T08:02:02Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![FrsECM](https://avatars.discourse-cdn.com/v4/letter/f/7993a0/32.png) [@FrsECM](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FFrsECM)%5C
**Post date:** [June 18, 2024, 8:02am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerate-weightedrandomsampler-dataloader%2F92721%2F1 "2024-06-18T08:02:02Z")

</div>

Hi !  
Like you mention in this post :  
[Accelerator .prepare() replaces custom DataLoader Sampler - 🤗Accelerate - Hugging Face Forums](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-prepare-replaces-custom-dataloader-sampler%2F43392)

When we use a custom sampler, it is used on downstream processes.

But today it works like that :

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/3X/9/5/957b4fcf161a0809226f1fa3a18393dd74573295.png)  
Sampler will generate batches, and batches will be assigned to different processes depending on their indexes.

When we have a weighted sampler, It’s probable to get the same datarows in multiple batches. It means we can have the same data on multiple processes.

I would like to make sure, each process uses independant data…

**Is it currently possible ?**  
**If yes how ?**  
**If no, how should i implement it ?**

I created a script in order to reproduce the issue, it is here :  
[Dataloader WeightedRandomSampler + Distributed Training · Issue #2865 · huggingface/accelerate (github.com)](https://github.com/huggingface/accelerate/issues/2865)

Thanks for your help and feedback

---

<div class="post-metadata">

**Author:** ![FrsECM](https://avatars.discourse-cdn.com/v4/letter/f/7993a0/32.png) [@FrsECM](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FFrsECM)%5C
**Post date:** [June 18, 2024, 8:03am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerate-weightedrandomsampler-dataloader%2F92721%2F2 "2024-06-18T08:03:15Z")

</div>

In the end, what i would like is more a behavior like that :  
 ![](https://us1.discourse-cdn.com/hellohellohello/original/3X/e/1/e1759389e0edfd0be9d7104f10cad07e81e4eb59.png)