# Multi-lora serving with adapters on S3

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmulti-lora-serving-with-adapters-on-s3%2F117629%3E%5C
**Category:** Amazon SageMaker\
**Created:** [November 12, 2024, 8:55pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmulti-lora-serving-with-adapters-on-s3%2F117629 "2024-11-12T20:55:42Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![lpvitrai](https://avatars.discourse-cdn.com/v4/letter/l/ac91a4/32.png) [@lpvitrai](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flpvitrai)%5C
**Post date:** [November 12, 2024, 8:55pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmulti-lora-serving-with-adapters-on-s3%2F117629%2F1 "2024-11-12T20:55:42Z")

</div>

I’ve read [this great article](https://huggingface.co/blog/multi-lora-serving) and want to deploy a base model (Qwen2.5 7B) and use it with different adapters. My model is fine-tuned via SageMaker and saved in S3.

I created the model like this:

```auto
config = {
  'HF_MODEL_ID': "/opt/ml/model", 
  'LORA_ADAPTERS': 'adapter_id=adapters/my_adapter_1'
}

llm_model = HuggingFaceModel(
  role=role,
  model_data={'S3DataSource':{'S3Uri': "s3://.../qwen-25-7-all-linear-.../output/model/",'S3DataType': 'S3Prefix','CompressionType': 'None'}},
  image_uri=llm_image,
  env=config
)

```

The structure of my S3 bucket is like this:

```auto
model/
--- all model files
--- adapters/
------ my_adapter_1/
--------- adapter_config.json
---------adapter_model.safetensors

```

When I try to deploy the endpoint, I get this error:

```auto
UnexpectedStatusException: Error hosting endpoint huggingface-pytorch-tgi-inference-2024-11-12-20-51-47-645: Failed. Reason: error: Key of model data S3 object 's3://sagemaker-ca-central-1-.../qwen-25-7-all-linear-.../output/model/adapters/' maps to invalid local file path..

```

Any guidance on how to properly do this and organize the files properly?

thanks!