Request Details on GPU ad memory requirements
#16
by
DragoZatch
- opened
I would i check on if anyone have tried to run the model on the GPU's and find out how much GPU memory is required for this model. Also wanted to know the max memory requirement for full scale and full context length support .
You can check the minimum deployment requirements on our GitHub.
This comment has been hidden (marked as Off-Topic)
I'm testing a distributed cluster to run this full-weights on consumer cards (pooling 4090s) to bypass the VRAM limit. let me know if you want to run a test job.