Deploy Salesforce/SFR-Embedding-Mistral on HF Inference Endpoint?
#6
by rlsChapters - opened
Hi all. I am trying to deploy the model on a HuggingFace Inference Endpoint and keep getting this error
Server message:Endpoint failed to start. raise NotImplementedError(\"sharded is not supported for AutoModel\")
Has anyone encountered this and how should I go about fixing it? Thanks!
Hi @rlsChapters ,
We didn't meet this issue on our end. Could you try the following package version:
torch==2.0.0
transformers>=4.31.0
Thanks for the suggestion @yliu279
I added the requirements.txt file with
torch==2.0.0
transformers>=4.31.0