# Extend load\_from\_disk and save\_to\_disk to remote storage

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fextend-load-from-disk-and-save-to-disk-to-remote-storage%2F1449%3E%5C
**Category:** 🤗Datasets\
**Created:** [October 9, 2020, 12:52pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fextend-load-from-disk-and-save-to-disk-to-remote-storage%2F1449 "2020-10-09T12:52:43Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![vblagoje](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/vblagoje/32/20818_2.png) [@vblagoje](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvblagoje)%5C
**Post date:** [October 9, 2020, 12:52pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fextend-load-from-disk-and-save-to-disk-to-remote-storage%2F1449%2F1 "2020-10-09T12:52:43Z")

</div>

Hello everyone,

As datasets are sliced, diced, and transformed it makes sense to load/save them from disk for future use. But why just local disk when this load/save can be extended to any remote storage (S3, GS etc)?

It would be great to have remote storage load/save added in the next release 🙂

Many thanks

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/64813_2.png) [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [October 12, 2020, 8:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fextend-load-from-disk-and-save-to-disk-to-remote-storage%2F1449%2F2 "2020-10-12T08:37:20Z")

</div>

You’re right that would be cool 🙂  
Thanks for the suggestion

We think it could also be useful to have some sort of versioning for hosted datasets. Do you feel like it could be something you’d be interested to have ?

---

<div class="post-metadata">

**Author:** ![vblagoje](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/vblagoje/32/20818_2.png) [@vblagoje](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvblagoje)%5C
**Post date:** [October 12, 2020, 8:53am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fextend-load-from-disk-and-save-to-disk-to-remote-storage%2F1449%2F3 "2020-10-12T08:53:33Z")

</div>

That would be nice to have. I guess you can have the mechanism for versioning currently available. I am more “concerned” about hooking up and changing different storage providers seamlessly and easily. Also, how to expose API to users for service authentication etc.

---

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvalhalla)%5C
**Post date:** [October 12, 2020, 3:40pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fextend-load-from-disk-and-save-to-disk-to-remote-storage%2F1449%2F4 "2020-10-12T15:40:25Z")

</div>

+1 for versioning.
