Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 4096,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

CONTRIBUTING.md

@@ -0,0 +1,65 @@
+# Contributing
+
+## Developer Certificate of Origin
+
+The Developer Certificate of Origin (DCO) is a way to certify that you wrote or otherwise have the right to submit the code you are contributing to the project.
+
+By signing off on your contribution, you certify the following DCO (Version 1.1):
+* Full text of the DCO (https://developercertificate.org/):
+  ```
+    Developer Certificate of Origin
+    Version 1.1
+    Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
+    Everyone is permitted to copy and distribute verbatim copies of this
+    license document, but changing it is not allowed.
+    Developer's Certificate of Origin 1.1
+    By making a contribution to this project, I certify that:
+    (a) The contribution was created in whole or in part by me and I
+        have the right to submit it under the open source license
+        indicated in the file; or
+    (b) The contribution is based upon previous work that, to the best
+        of my knowledge, is covered under an appropriate open source
+        license and I have the right under that license to submit that
+        work with modifications, whether created in whole or in part
+        by me, under the same open source license (unless I am
+        permitted to submit under a different license), as indicated
+        in the file; or
+    (c) The contribution was provided directly to me by some other
+        person who certified (a), (b) or (c) and I have not modified
+        it.
+    (d) I understand and agree that this project and the contribution
+        are public and that a record of the contribution (including all
+        personal information I submit with it, including my sign-off) is
+        maintained indefinitely and may be redistributed consistent with
+        this project or the open source license(s) involved.
+  ```
+
+
+## Sign your work
+
+To certify the DCO, you must add a "Signed-off-by" line to your commit messages.
+
+**Important:** You must use your real name (no pseudonyms or anonymous contributions).
+
+**Example:**
+
+```text
+Fix typo in tokenizer config
+
+Signed-off-by: Your Name <your@email.com>
+```
+
+### Git Command
+
+If you are using the command line, you can sign off automatically by adding the `-s` flag:
+
+```bash
+git commit -s -m "Your commit message"
+```
+
+## Submitting changes
+
+1. **Fork the repository** on Hugging Face.
+2. **Create a branch** for your fix.
+3. **Commit your changes** with the `-s` flag to sign them.
+4. **Open a Pull Request** (PR) on the Community tab of the model page.

LICENSE

@@ -0,0 +1,55 @@
+All binary model files and source code files are licensed under the OpenMDW-1.1 License.
+
+------------
+Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+
+OpenMDW License Agreement, version 1.1 (OpenMDW-1.1)
+
+By exercising rights granted to you under this agreement, you accept and agree
+to its terms.
+
+As used in this agreement, "Model Materials" means the materials provided to
+you under this agreement, consisting of: (1) one or more machine learning
+models (including architecture and parameters); and (2) all related artifacts
+(including associated data, documentation and software) that are provided to
+you hereunder.
+
+Subject to your compliance with this agreement, permission is hereby granted,
+free of charge, to deal in the Model Materials without restriction, including
+under all copyright, patent, database, and trade secret rights included or
+embodied therein.
+
+If you distribute any portion of the Model Materials, you shall retain in your
+distribution (1) a copy of this agreement, and (2) all copyright notices and
+other notices of origin included in the Model Materials that are applicable to
+your distribution.
+
+If you file, maintain, or voluntarily participate in a lawsuit against any
+person or entity asserting that the Model Materials directly or indirectly
+infringe any patent or copyright, then all rights and grants made to you
+hereunder are terminated, unless that lawsuit was in response to a
+corresponding lawsuit first brought against you.
+
+This agreement does not impose any restrictions or obligations with respect to
+any use, modification, or sharing of any outputs generated by using the Model
+Materials.
+
+THE MODEL MATERIALS ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE, TITLE, NONINFRINGEMENT, ACCURACY, OR THE
+ABSENCE OF LATENT OR OTHER DEFECTS OR ERRORS, WHETHER OR NOT DISCOVERABLE, ALL
+TO THE GREATEST EXTENT PERMISSIBLE UNDER APPLICABLE LAW.
+
+YOU ARE SOLELY RESPONSIBLE FOR (1) CLEARING RIGHTS OF OTHER PERSONS THAT MAY
+APPLY TO THE MODEL MATERIALS OR ANY USE THEREOF, INCLUDING WITHOUT LIMITATION
+ANY PERSON'S COPYRIGHTS OR OTHER RIGHTS INCLUDED OR EMBODIED IN THE MODEL
+MATERIALS; (2) OBTAINING ANY NECESSARY CONSENTS, PERMISSIONS OR OTHER RIGHTS
+REQUIRED FOR ANY USE OF THE MODEL MATERIALS; OR (3) PERFORMING ANY DUE
+DILIGENCE OR UNDERTAKING ANY OTHER INVESTIGATIONS INTO THE MODEL MATERIALS OR
+ANYTHING INCORPORATED OR EMBODIED THEREIN.
+
+IN NO EVENT SHALL THE PROVIDERS OF THE MODEL MATERIALS BE LIABLE FOR ANY CLAIM,
+DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE MODEL MATERIALS, THE
+USE THEREOF OR OTHER DEALINGS THEREIN.

NOTICE

@@ -0,0 +1,5 @@
+mistralai/Ministral-3-8B-Instruct-2512 is licensed under the Apache-2.0 License.
+
+- You must cause any modified files to carry prominent notices stating that You changed the files.
+- You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works.
+- If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License.

README.md

@@ -0,0 +1,682 @@
+---
+license: other
+license_name: openmdw-1.1
+license_link: >-
+  https://openmdw.ai/license/1-1/
+tags:
+  - text
+  - text-embeddings
+  - retrieval
+  - semantic-search
+  - transformers
+  - rag
+language:
+  - multilingual
+library_name: transformers
+---
+
+# Model Overview
+
+### Description:
+**Nemotron-3-Embed-8B-BF16** is a versatile text embedding model trained by NVIDIA and optimized for retrieval and semantic similarity tasks. It provides strong multilingual and cross-lingual retrieval capabilities and is designed to serve as a foundational component in text-based Retrieval-Augmented Generation (RAG) systems. This model was evaluated across 34 languages: English, Arabic, Assamese, Bengali, Bulgarian, Chinese, Danish, Dutch, Finnish, French, German, Hindi, Hinglish, Indonesian, Italian, Japanese, Korean, Malay, Marathi, Nepalese, Norwegian, Persian, Portuguese, Romanian, Russian, Spanish, Swahili, Swedish, Tamil, Telugu, Thai, Ukrainian, Urdu, Vietnamese.
+
+The model generates dense vector embeddings from multilingual text inputs, enabling retrieval, semantic search, and (agentic) RAG workflows. As a core component of text retrieval systems, an embedding model transforms text, such as questions or passages, into dense vector representations. These models are typically transformer encoders that process input tokens and produce embeddings suitable for efficient similarity matching.
+
+**Nemotron-3-Embed-8B-BF16** achieves state-of-the-art performance on the multilingual [RTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard?benchmark_name=RTEB%28beta%29) as of July XX, 2026.
+
+This model is ready for commercial use.
+
+### License/Terms of Use:
+
+This model and its associated configuration files are licensed under the [OpenMDW License Agreement, version 1.1 (OpenMDW-1.1)](https://openmdw.ai/license/1-1/). Additional Information: Built with [Ministral-3-8B-Instruct-2512](https://huggingface.co/mistralai/Ministral-3-8B-Instruct-2512) which is released under Apache 2.0.
+
+This project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use.
+
+### Deployment Geography:
+Global
+
+### Use Case: <br>
+
+The **Nemotron-3-Embed-8B-BF16** is most suitable for users who want to build a multilingual question-and-answer application over a large text corpus, leveraging the latest dense retrieval technologies.
+
+### Release Date: <br>
+
+07/15/2026 via https://huggingface.co/nvidia/Nemotron-3-Embed-8B-BF16
+
+## Model Architecture:
+**Architecture Type:** Transformer <br>
+
+**Network Architecture:** [Ministral-3-8B-Instruct-2512](https://huggingface.co/mistralai/Ministral-3-8B-Instruct-2512) based encoder model. <br>
+
+**Number of model parameters:** The model has approximately 8B parameters. <br>
+
+**Hidden Size:** 4096 <br>
+
+The **Nemotron-3-Embed-8B-BF16** model is a transformer-based text embedding model trained with bidirectional attention masking, where the final embedding vector is obtained by applying average pooling to the transformer’s token-level representations. It encodes each input text into a dense embedding vector of dimension 4096.
+
+
+## Input(s):
+**Input Type(s):** Text <br>
+
+**Input Format(s):** <br>
+- Text: List of strings <br>
+
+**Input Parameters:** <br>
+- Text: One-Dimensional (1D) <br>
+
+**Other Properties Related to Input:** Text inputs should be tokenized by the model tokenizer. The model’s max sequence length is 32768. Longer inputs should be chunked or truncated. <br>
+
+## Output(s):
+
+**Output Type(s):** Floats <br>
+
+**Output Format(s):** <br>
+- List of float arrays <br>
+
+**Output Parameters:** One-Dimensional (1D) embedding vector per input text string <br>
+
+**Other Properties Related to Output:** The model outputs a 4096-dimensional embedding vector for each input text string. <br>
+
+Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions. <br>
+
+
+## Usage
+
+For local Python examples, use the Hugging Face model ID by default. If you are working from a local checkout, replace `MODEL_ID` with that path. For vLLM online serving from a local checkpoint, use the local checkpoint example.
+
+```python
+MODEL_ID = "nvidia/Nemotron-3-Embed-8B-BF16"
+```
+
+Use this model for retrieval-style embeddings. Add the `query: ` prefix for queries and the `passage: ` prefix for documents. Embeddings are L2-normalized, so dot product and cosine similarity are equivalent. The output tables use `q[i]` for queries and `d[i]` for documents. Scores are rounded to four decimals. Exact values can vary by runtime and package version.
+
+### Local Python Dependencies
+
+The BF16 checkpoints support Transformers `5.2.0` and above. The examples also require a CUDA-enabled PyTorch installation that matches your driver and CUDA environment.
+
+If you are not using an NVIDIA PyTorch container, install PyTorch first. Use the [PyTorch local installation selector](https://pytorch.org/get-started/locally/) to choose the command that matches your operating system, package manager, and CUDA environment. If the default PyPI wheel matches your CUDA environment, run:
+
+```bash
+pip install --upgrade torch
+```
+
+For non-default CUDA environments, the selector can include an additional `--index-url` argument. Use that CUDA-specific index URL when the default PyPI wheel does not match your CUDA environment.
+
+Then install Transformers and Sentence Transformers:
+
+```bash
+pip install --upgrade "transformers>=5.2.0" "sentence-transformers>=5.4.1"
+```
+
+NVIDIA tested the examples in `nvcr.io/nvidia/pytorch:26.06-py3` with the container-provided Torch and CUDA stack. Inside NVIDIA PyTorch containers, do not upgrade Torch. Install only the missing packages:
+
+```bash
+pip install --upgrade "transformers>=5.2.0" "sentence-transformers>=5.4.1"
+```
+
+The tested `nvcr.io/nvidia/pytorch:26.06-py3` container includes `flash-attn`, so the snippets use FlashAttention-2 by default. If your environment does not have FlashAttention-2, set `attn_implementation` or `ATTN_IMPLEMENTATION` to `sdpa`.
+
+
+### Sentence Transformers
+
+Use Sentence Transformers for the simplest local Python interface. It reads the saved query and document prompts and normalization metadata.
+
+```python
+import torch
+from sentence_transformers import SentenceTransformer
+
+MODEL_ID = "nvidia/Nemotron-3-Embed-8B-BF16"
+
+model = SentenceTransformer(
+    MODEL_ID,
+    device="cuda",
+    model_kwargs={
+        "dtype": torch.bfloat16,
+        "attn_implementation": "flash_attention_2",
+    },
+    processor_kwargs={"padding_side": "left"},
+)
+model.max_seq_length = 32768
+
+QUERIES = [
+    "Write a Python function that counts the frequency of each element in a list of lists.",
+    "Write a function that orders a dictionary with tuple keys by the product of each key's tuple values.",
+    "What symptoms and common triggers help distinguish eczema from other inflammatory skin conditions?",
+    "How can someone reduce exposure to pollen during allergy season?",
+]
+
+DOCUMENTS = [
+    "def frequency_lists(list1):\n    flattened = [item for sublist in list1 for item in sublist]\n    counts = {}\n    for item in flattened:\n        if item in counts:\n            counts[item] += 1\n        else:\n            counts[item] = 1\n    return counts",
+    "def sort_dict_item(test_dict):\n    return {key: test_dict[key] for key in sorted(test_dict.keys(), key=lambda ele: ele[0] * ele[1])}",
+    "Eczema commonly causes itchy, dry, inflamed patches of skin. The affected areas may look red, scaly, cracked, or darker than the surrounding skin depending on skin tone. Symptoms can flare after exposure to irritants, allergens, stress, or changes in weather.",
+    "People with pollen allergy can reduce exposure by staying indoors on dry, windy days, avoiding early-morning outdoor activity, and going outside after rain when pollen levels are lower. They should check pollen forecasts, close windows and doors when counts are high, and consider starting allergy medication before symptoms begin if high pollen is expected. After being outside, showering, changing clothes, avoiding outdoor laundry drying, and wearing a face mask for yard work can help limit pollen contact.",
+]
+query_embeddings = model.encode_query(QUERIES, batch_size=1, convert_to_tensor=True)
+document_embeddings = model.encode_document(DOCUMENTS, batch_size=1, convert_to_tensor=True)
+
+scores = model.similarity(query_embeddings, document_embeddings)
+print("Similarity scores:")
+print(f"{'':>8}" + "".join(f"d[{i}]   " for i in range(scores.shape[1])))
+for query_index, row in enumerate(scores):
+    print(f"q[{query_index}]   " + "  ".join(f"{score.item():>7.4f}" for score in row))
+```
+
+<details>
+<summary>Sentence Transformers Expected Output</summary>
+
+```text
+Similarity scores:
+          d[0]     d[1]     d[2]     d[3]   
+q[0]    0.7852   0.0496   0.0115  -0.0299
+q[1]    0.0457   0.6562  -0.0645   0.0452
+q[2]   -0.0016  -0.0530   0.6602   0.1123
+q[3]   -0.0170  -0.0036   0.1147   0.8008
+```
+
+</details>
+
+### Transformers
+
+Use Transformers when you want manual control over tokenization, pooling, or batching.
+
+```python
+import torch
+import torch.nn.functional as F
+from transformers import AutoModel, AutoTokenizer
+
+MODEL_ID = "nvidia/Nemotron-3-Embed-8B-BF16"
+MAX_LENGTH = 32768
+BATCH_SIZE = 1
+DTYPE = torch.bfloat16
+ATTN_IMPLEMENTATION = "flash_attention_2"
+
+QUERIES = [
+    "Write a Python function that counts the frequency of each element in a list of lists.",
+    "Write a function that orders a dictionary with tuple keys by the product of each key's tuple values.",
+    "What symptoms and common triggers help distinguish eczema from other inflammatory skin conditions?",
+    "How can someone reduce exposure to pollen during allergy season?",
+]
+
+DOCUMENTS = [
+    "def frequency_lists(list1):\n    flattened = [item for sublist in list1 for item in sublist]\n    counts = {}\n    for item in flattened:\n        if item in counts:\n            counts[item] += 1\n        else:\n            counts[item] = 1\n    return counts",
+    "def sort_dict_item(test_dict):\n    return {key: test_dict[key] for key in sorted(test_dict.keys(), key=lambda ele: ele[0] * ele[1])}",
+    "Eczema commonly causes itchy, dry, inflamed patches of skin. The affected areas may look red, scaly, cracked, or darker than the surrounding skin depending on skin tone. Symptoms can flare after exposure to irritants, allergens, stress, or changes in weather.",
+    "People with pollen allergy can reduce exposure by staying indoors on dry, windy days, avoiding early-morning outdoor activity, and going outside after rain when pollen levels are lower. They should check pollen forecasts, close windows and doors when counts are high, and consider starting allergy medication before symptoms begin if high pollen is expected. After being outside, showering, changing clothes, avoiding outdoor laundry drying, and wearing a face mask for yard work can help limit pollen contact.",
+]
+
+def average_pool(last_hidden_state: torch.Tensor, attention_mask: torch.Tensor) -> torch.Tensor:
+    last_hidden = last_hidden_state.masked_fill(~attention_mask[..., None].bool(), 0.0)
+    return last_hidden.sum(dim=1) / attention_mask.sum(dim=1)[..., None]
+
+if not torch.cuda.is_available():
+    raise RuntimeError("CUDA is required for practical BF16 inference.")
+
+device = torch.device("cuda")
+
+tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, padding_side="left")
+if tokenizer.pad_token is None:
+    tokenizer.pad_token = tokenizer.eos_token
+
+model = AutoModel.from_pretrained(
+    MODEL_ID,
+    dtype=DTYPE,
+    attn_implementation=ATTN_IMPLEMENTATION,
+).to(device)
+model.eval()
+
+def encode_texts(texts: list[str]) -> torch.Tensor:
+    embedding_batches = []
+
+    for start in range(0, len(texts), BATCH_SIZE):
+        encoded = tokenizer(
+            texts[start : start + BATCH_SIZE],
+            max_length=MAX_LENGTH,
+            truncation=True,
+            padding=True,
+            return_tensors="pt",
+        )
+        encoded = {name: tensor.to(device) for name, tensor in encoded.items()}
+
+        with torch.inference_mode():
+            output = model(**encoded)
+            pooled = average_pool(output.last_hidden_state, encoded["attention_mask"])
+            embeddings = F.normalize(pooled, p=2, dim=-1)
+
+        embedding_batches.append(embeddings.detach().cpu().to(torch.float32))
+
+    return torch.cat(embedding_batches, dim=0)
+
+embeddings = encode_texts(
+    ["query: " + query for query in QUERIES]
+    + ["passage: " + doc for doc in DOCUMENTS]
+)
+query_embeddings = embeddings[: len(QUERIES)]
+document_embeddings = embeddings[len(QUERIES) :]
+
+scores = query_embeddings @ document_embeddings.T
+print("Similarity scores:")
+print(f"{'':>8}" + "".join(f"d[{i}]   " for i in range(scores.shape[1])))
+for query_index, row in enumerate(scores):
+    print(f"q[{query_index}]   " + "  ".join(f"{score.item():>7.4f}" for score in row))
+```
+
+<details>
+<summary>Transformers Expected Output</summary>
+
+```text
+Similarity scores:
+          d[0]     d[1]     d[2]     d[3]   
+q[0]    0.7856   0.0496   0.0113  -0.0298
+q[1]    0.0458   0.6505  -0.0642   0.0451
+q[2]   -0.0018  -0.0529   0.6609   0.1124
+q[3]   -0.0171  -0.0034   0.1151   0.8004
+```
+
+</details>
+
+### vLLM Dependencies
+
+For BF16, use `vllm==0.25.0` for `/v2/embed` serving. NVIDIA also validated `vllm serve "$MODEL_ID"` with `vllm/vllm-openai:v0.21.0`, `nvcr.io/nvidia/vllm:26.06-py3`, and `vllm/vllm-openai:v0.24.0` for this checkpoint.
+
+```bash
+pip install --upgrade "vllm==0.25.0" openai requests numpy
+```
+
+### vLLM Offline Python
+
+Use the offline Python API when you want local vLLM inference without running an HTTP server. `LLM.embed` accepts formatted strings, so add the `query: ` and `passage: ` prefixes manually.
+
+<details>
+<summary>vLLM Offline Python Example</summary>
+
+```python
+import numpy as np
+from vllm import LLM
+
+MODEL_ID = "nvidia/Nemotron-3-Embed-8B-BF16"
+
+QUERIES = [
+    "Write a Python function that counts the frequency of each element in a list of lists.",
+    "Write a function that orders a dictionary with tuple keys by the product of each key's tuple values.",
+    "What symptoms and common triggers help distinguish eczema from other inflammatory skin conditions?",
+    "How can someone reduce exposure to pollen during allergy season?",
+]
+
+DOCUMENTS = [
+    "def frequency_lists(list1):\n    flattened = [item for sublist in list1 for item in sublist]\n    counts = {}\n    for item in flattened:\n        if item in counts:\n            counts[item] += 1\n        else:\n            counts[item] = 1\n    return counts",
+    "def sort_dict_item(test_dict):\n    return {key: test_dict[key] for key in sorted(test_dict.keys(), key=lambda ele: ele[0] * ele[1])}",
+    "Eczema commonly causes itchy, dry, inflamed patches of skin. The affected areas may look red, scaly, cracked, or darker than the surrounding skin depending on skin tone. Symptoms can flare after exposure to irritants, allergens, stress, or changes in weather.",
+    "People with pollen allergy can reduce exposure by staying indoors on dry, windy days, avoiding early-morning outdoor activity, and going outside after rain when pollen levels are lower. They should check pollen forecasts, close windows and doors when counts are high, and consider starting allergy medication before symptoms begin if high pollen is expected. After being outside, showering, changing clothes, avoiding outdoor laundry drying, and wearing a face mask for yard work can help limit pollen contact.",
+]
+
+def main():
+    llm = LLM(model=MODEL_ID)
+    texts = ["query: " + query for query in QUERIES] + [
+        "passage: " + doc for doc in DOCUMENTS
+    ]
+    outputs = llm.embed(texts, use_tqdm=False)
+    embeddings = np.array(
+        [output.outputs.embedding for output in outputs],
+        dtype=np.float32,
+    )
+
+    query_embeddings = embeddings[: len(QUERIES)]
+    document_embeddings = embeddings[len(QUERIES) :]
+
+    scores = query_embeddings @ document_embeddings.T
+    print("Similarity scores:")
+    print(f"{'':>8}" + "".join(f"d[{i}]   " for i in range(scores.shape[1])))
+    for query_index, row in enumerate(scores):
+        print(f"q[{query_index}]   " + "  ".join(f"{score:>7.4f}" for score in row))
+
+
+if __name__ == "__main__":
+    main()
+```
+
+</details>
+
+<details>
+<summary>vLLM Offline Python Expected Output</summary>
+
+```text
+Similarity scores:
+          d[0]     d[1]     d[2]     d[3]   
+q[0]    0.7846   0.0489   0.0118  -0.0301
+q[1]    0.0454   0.6505  -0.0645   0.0455
+q[2]   -0.0018  -0.0531   0.6611   0.1120
+q[3]   -0.0166  -0.0040   0.1148   0.7983
+```
+
+</details>
+
+### vLLM Online Serving
+
+```bash
+MODEL_ID=nvidia/Nemotron-3-Embed-8B-BF16
+
+vllm serve "$MODEL_ID"
+```
+
+vLLM defaults to port `8000`. Add host and port when you need an explicit bind address or a non-default port:
+
+```bash
+vllm serve "$MODEL_ID" --host 0.0.0.0 --port 8000
+```
+
+If you serve the Hugging Face model ID, the served model name defaults to `nvidia/Nemotron-3-Embed-8B-BF16`. If you serve a local checkpoint path, vLLM still reads the model config and weights from that path; `--served-model-name` only sets the model name accepted by API requests.
+
+```bash
+MODEL_PATH=/path/to/local/Nemotron-3-Embed-8B-BF16
+vllm serve "$MODEL_PATH" --host 0.0.0.0 --port 8000 --served-model-name nvidia/Nemotron-3-Embed-8B-BF16
+```
+
+With `--served-model-name`, client requests continue to use `MODEL = "nvidia/Nemotron-3-Embed-8B-BF16"`. If you omit it, use the served name reported by `/v1/models` in client requests.
+
+#### Recommended Retrieval Endpoint
+
+After the server is running, use `/v2/embed` for retrieval. Send raw query and document strings. `input_type` applies the saved query and document prompts:
+
+```python
+import numpy as np
+import requests
+
+MODEL = "nvidia/Nemotron-3-Embed-8B-BF16"
+URL = "http://localhost:8000/v2/embed"
+
+QUERIES = [
+    "Write a Python function that counts the frequency of each element in a list of lists.",
+    "Write a function that orders a dictionary with tuple keys by the product of each key's tuple values.",
+    "What symptoms and common triggers help distinguish eczema from other inflammatory skin conditions?",
+    "How can someone reduce exposure to pollen during allergy season?",
+]
+
+DOCUMENTS = [
+    "def frequency_lists(list1):\n    flattened = [item for sublist in list1 for item in sublist]\n    counts = {}\n    for item in flattened:\n        if item in counts:\n            counts[item] += 1\n        else:\n            counts[item] = 1\n    return counts",
+    "def sort_dict_item(test_dict):\n    return {key: test_dict[key] for key in sorted(test_dict.keys(), key=lambda ele: ele[0] * ele[1])}",
+    "Eczema commonly causes itchy, dry, inflamed patches of skin. The affected areas may look red, scaly, cracked, or darker than the surrounding skin depending on skin tone. Symptoms can flare after exposure to irritants, allergens, stress, or changes in weather.",
+    "People with pollen allergy can reduce exposure by staying indoors on dry, windy days, avoiding early-morning outdoor activity, and going outside after rain when pollen levels are lower. They should check pollen forecasts, close windows and doors when counts are high, and consider starting allergy medication before symptoms begin if high pollen is expected. After being outside, showering, changing clothes, avoiding outdoor laundry drying, and wearing a face mask for yard work can help limit pollen contact.",
+]
+
+def embed(input_type: str, texts: list[str]) -> np.ndarray:
+    response = requests.post(
+        URL,
+        json={
+            "model": MODEL,
+            "input_type": input_type,
+            "texts": texts,
+            "embedding_types": ["float"],
+            "truncate": "END",
+        },
+        timeout=120,
+    )
+    response.raise_for_status()
+    return np.array(response.json()["embeddings"]["float"], dtype=np.float32)
+
+query_embeddings = embed("query", QUERIES)
+document_embeddings = embed("document", DOCUMENTS)
+
+scores = query_embeddings @ document_embeddings.T
+print("Similarity scores:")
+print(f"{'':>8}" + "".join(f"d[{i}]   " for i in range(scores.shape[1])))
+for query_index, row in enumerate(scores):
+    print(f"q[{query_index}]   " + "  ".join(f"{score:>7.4f}" for score in row))
+```
+
+<details>
+<summary>Recommended Retrieval Endpoint Expected Output</summary>
+
+```text
+Similarity scores:
+          d[0]     d[1]     d[2]     d[3]   
+q[0]    0.7848   0.0489   0.0117  -0.0298
+q[1]    0.0454   0.6512  -0.0644   0.0455
+q[2]   -0.0021  -0.0529   0.6614   0.1121
+q[3]   -0.0169  -0.0033   0.1151   0.7989
+```
+
+</details>
+
+You can also use the OpenAI-compatible `/v1/embeddings` endpoint. For those requests, pass strings in `input` and manually prefix them with `query: ` or `passage: `.
+
+
+## Software Integration:
+**Runtime Engine(s):**  PyTorch, vLLM<br>
+
+**Supported Hardware Microarchitecture Compatibility:** <br>
+- NVIDIA Ampere
+- NVIDIA Blackwell
+- NVIDIA Hopper
+
+**Supported Operating System(s):** <br>
+* Linux <br>
+
+The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment. <br>
+
+## Model Version(s):
+**Nemotron-3-Embed-8B-BF16** <br>
+
+## Training, Testing, and Evaluation Datasets:
+
+### Dataset Overview:
+**Total Size:** 50M+ data samples<br>
+**Dataset Partition:** Training [100%], Testing [N/A — evaluation benchmarks used separately], Validation [N/A — evaluation benchmarks used separately].
+
+Model training was conducted using publicly available, commercially permissible datasets and synthetically generated datasets. Synthetic datasets were created either by generating queries from seed documents or by generating complete question–answer pairs through LLM-based prompting using the LLMs listed below.<br>
+
+## Public Datasets:
+
+| Dataset name | Reference |
+| --- | --- |
+| MIRACL | [https://huggingface.co/datasets/miracl/miracl](https://huggingface.co/datasets/miracl/miracl) |
+| MLDR | [https://huggingface.co/datasets/Shitao/MLDR](https://huggingface.co/datasets/Shitao/MLDR) |
+| HotpotQA | [https://hotpotqa.github.io/](https://hotpotqa.github.io/) |
+| NQ | [https://huggingface.co/datasets/sentence-transformers/embedding-training-data](https://huggingface.co/datasets/sentence-transformers/embedding-training-data) |
+| SQuAD | [https://rajpurkar.github.io/SQuAD-explorer/](https://rajpurkar.github.io/SQuAD-explorer/) |
+| Stack Exchange | [https://archive.org/details/stackexchange](https://archive.org/details/stackexchange) |
+| HoVer | [https://hover-nlp.github.io/](https://hover-nlp.github.io/) |
+| TAT-QA | [https://huggingface.co/datasets/next-tat/TAT-QA](https://huggingface.co/datasets/next-tat/TAT-QA) |
+| FinQA | [https://github.com/czyssrs/FinQA/tree/main](https://github.com/czyssrs/FinQA/tree/main) |
+| PubMedQA | [https://huggingface.co/datasets/qiaojin/PubMedQA/viewer/pqa_labeled](https://huggingface.co/datasets/qiaojin/PubMedQA/viewer/pqa_labeled) |
+| MedQuAD | [https://github.com/abachaa/MedQuAD](https://github.com/abachaa/MedQuAD) |
+| JaQuAD | [https://huggingface.co/datasets/SkelterLabsInc/JaQuAD](https://huggingface.co/datasets/SkelterLabsInc/JaQuAD) |
+| coir_apps | [https://huggingface.co/datasets/CoIR-Retrieval/apps](https://huggingface.co/datasets/CoIR-Retrieval/apps) |
+| coir_cosqa | [https://huggingface.co/datasets/CoIR-Retrieval/cosqa](https://huggingface.co/datasets/CoIR-Retrieval/cosqa) |
+| coir_stackoverflow_qa | [https://huggingface.co/datasets/CoIR-Retrieval/stackoverflow-qa](https://huggingface.co/datasets/CoIR-Retrieval/stackoverflow-qa) |
+| coir_codetrans_dl | [https://huggingface.co/datasets/CoIR-Retrieval/codetrans-dl](https://huggingface.co/datasets/CoIR-Retrieval/codetrans-dl) |
+| coir_codetrans_contest | [https://huggingface.co/datasets/CoIR-Retrieval/codetrans-contest](https://huggingface.co/datasets/CoIR-Retrieval/codetrans-contest) |
+| synthetic_text2sql | [https://huggingface.co/datasets/CoIR-Retrieval/synthetic-text2sql](https://huggingface.co/datasets/CoIR-Retrieval/synthetic-text2sql) |
+| SWE-bench | [https://huggingface.co/datasets/princeton-nlp/SWE-bench/viewer/default/train](https://huggingface.co/datasets/princeton-nlp/SWE-bench/viewer/default/train) |
+| MLQA | [https://github.com/facebookresearch/MLQA](https://github.com/facebookresearch/MLQA) |
+| SpartQA | [HLR/SpartQA_generation](https://github.com/HLR/SpartQA_generation) |
+| Winogrande | [allenai/winogrande](https://github.com/allenai/winogrande) |
+| TempReason | [https://huggingface.co/datasets/tonytan48/TempReason](https://huggingface.co/datasets/tonytan48/TempReason) |
+| PAQ | [https://huggingface.co/datasets/sentence-transformers/embedding-training-data](https://huggingface.co/datasets/sentence-transformers/embedding-training-data) |
+| Wikipedia | [https://huggingface.co/datasets/wikimedia/wikipedia](https://huggingface.co/datasets/wikimedia/wikipedia) |
+| CCNews | [https://commoncrawl.org/2016/10/news-dataset-available/](https://commoncrawl.org/2016/10/news-dataset-available/) |
+| S2ORC | [https://huggingface.co/datasets/sentence-transformers/embedding-training-data](https://huggingface.co/datasets/sentence-transformers/embedding-training-data) |
+| Reddit | [https://huggingface.co/datasets/sentence-transformers/reddit-title-body](https://huggingface.co/datasets/sentence-transformers/reddit-title-body) |
+
+## Synthetic Datasets:
+
+Synthetic query-document pairs were generated either from scratch or by using seed datasets to generate queries with the models listed below.
+
+<table>
+  <tr>
+    <th>LLMs used to generate synthetic datasets:</th>
+  </tr>
+  <tr>
+    <td>Qwen/Qwen3-Next-80B-A3B-Instruct<br>Qwen/Qwen3-235B-A22B<br>Qwen/Qwen3.5-397B-A17B<br>Qwen/Qwen3.6-27B<br>Qwen/Qwen3.6-35B-A3B</td>
+  </tr>
+  <tr>
+    <td>google/gemma-4-31B-it</td>
+  </tr>
+  <tr>
+    <td>openai/gpt-oss-120b<br>openai/gpt-oss-20b</td>
+  </tr>
+  <tr>
+    <td>nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16<br>nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4</td>
+  </tr>
+</table>
+
+<table>
+  <tr>
+    <th colspan="2">Seed Datasets</th>
+  </tr>
+  <tr>
+    <th>Dataset</th>
+    <th>Reference</th>
+  </tr>
+  <tr>
+    <td>FinePdfs</td>
+    <td><a href="https://huggingface.co/datasets/HuggingFaceFW/finepdfs">https://huggingface.co/datasets/HuggingFaceFW/finepdfs</a></td>
+  </tr>
+  <tr>
+    <td>CentralActs</td>
+    <td><a href="https://zenodo.org/records/5088102">https://zenodo.org/records/5088102</a></td>
+  </tr>
+  <tr>
+    <td>BRIGHT</td>
+    <td><a href="https://huggingface.co/datasets/xlangai/BRIGHT">https://huggingface.co/datasets/xlangai/BRIGHT</a></td>
+  </tr>
+  <tr>
+    <td>MultiHiertt</td>
+    <td><a href="https://github.com/psunlpgroup/MultiHiertt">psunlpgroup/MultiHiertt</a></td>
+  </tr>
+</table>
+
+
+## Training Dataset:
+
+### Data Modality:
+* Text <br>
+
+### Training Data Size:
+
+**Text Training Data Size:** 50M+ data samples <br>
+
+**Data Collection Method by dataset:** Hybrid: Human, Automated, Synthetic <br>
+
+**Labeling Method by dataset:** Hybrid: Human, Automated, Synthetic <br>
+
+**Properties:** Model training was conducted on text datasets using question–passage pairs from publicly available, commercially permissible datasets and synthetically generated datasets. <br>
+
+
+### Testing Dataset:
+
+**Data Collection Method by dataset:** Not Applicable <br>
+
+**Labeling Method by dataset:** Not Applicable <br>
+
+**Properties:** Not Applicable. Model quality was assessed using the evaluation benchmark datasets described in the Evaluation Dataset subsection. <br>
+
+
+### Evaluation Dataset:
+
+**Data Collection Method by dataset:** Hybrid: Human, Automated, Synthetic <br>
+
+**Labeling Method by dataset:** Hybrid: Human, Automated, Synthetic <br>
+
+**Properties:** This model is evaluated on 16 public tasks on [Retrieval Embedding Benchmark (RTEB)](https://huggingface.co/blog/rteb), a new benchmark designed to reliably evaluate the retrieval accuracy of embedding models for real-world applications. More details on RTEB can be found on their [leaderboard](https://huggingface.co/spaces/mteb/leaderboard?benchmark_name=RTEB%28beta%29).
+
+We also evaluated the model on the [MMTEB (Retrieval) benchmark datasets](https://huggingface.co/spaces/mteb/leaderboard) ([paper](https://arxiv.org/pdf/2502.13595)), and on the eight text datasets (extracted via OCR) from [ViDoRe-V3 benchmark](https://mteb-leaderboard.hf.space/benchmark/ViDoRe(v3)).
+
+We set the model sequence length to 4096 for the evaluation results below.
+
+<table>
+  <tr>
+    <th colspan="4">Text Retrieval benchmarks (chunk retrieval) - Avg. NDCG@10</th>
+  </tr>
+  <tr>
+    <th>Model Name</th>
+    <th>RTEB 16</th>
+    <th>ViDoRE-V3 text</th>
+    <th>MMTEB (Retrieval)</th>
+  </tr>
+  <tr>
+    <td>llama-nemotron-embed-1b-v2</td>
+    <td>60.47</td>
+    <td>52.10</td>
+    <td>59.58</td>
+  </tr>
+  <tr>
+    <td>llama-nemotron-embed-vl-1b-v2</td>
+    <td>61.98</td>
+    <td>52.54</td>
+    <td>59.71</td>
+  </tr>
+  <tr>
+    <td>Nemotron-3-Embed-1B-BF16</td>
+    <td>72.38</td>
+    <td>57.74</td>
+    <td>71.04</td>
+  </tr>
+  <tr>
+    <td>Nemotron-3-Embed-8B-BF16</td>
+    <td>78.46</td>
+    <td>60.60</td>
+    <td>75.45</td>
+  </tr>
+</table>
+
+
+## Inference:
+**Acceleration Engine:** PyTorch, vLLM <br>
+**Test Hardware:**
+- NVIDIA Ampere - A100 80GB
+- NVIDIA Hopper - H100 80GB
+
+
+## Ethical Considerations:
+NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. <br>
+
+For more detailed information on ethical considerations for this model, please see the Model Card++ Bias, Explainability, Safety & Security, and Privacy Subcards. <br>
+
+Please report model quality, risk, security vulnerabilities or NVIDIA AI concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/). <br>
+
+## Bias
+
+| Field | Response |
+| ----- | ----- |
+| Participation considerations from adversely impacted groups [protected classes](https://www.senate.ca.gov/content/protected-classes) in model design and testing: | None |
+| Measures taken to mitigate against unwanted bias: | None |
+| Bias Metric (If Measured): | None |
+
+## Explainability
+
+| Field | Response |
+| ----- | ----- |
+| Intended Task/Domain: | Passage and query embedding for question and answer retrieval |
+| Model Type: | Transformer encoder |
+| Intended Users: | Generative AI creators working with conversational AI models - users who want to build a multilingual question and answer application over a large text corpus, leveraging the latest dense retrieval technologies. |
+| Output: | Array of float numbers (Dense Vector Representation for the input text) |
+| Describe how the model works: | Model transforms the tokenized input text into a dense vector representation. |
+| Name the adversely impacted groups this has been tested to deliver comparable outcomes regardless of: | Not Applicable |
+| Technical Limitations & Mitigation: | The model’s max sequence length is 32768. Therefore, the longer text inputs should be truncated. |
+| Verified to have met prescribed NVIDIA quality standards: | Yes |
+| Performance Metrics: | Accuracy, Throughput, and Latency |
+| Potential Known Risks: | This model does not always guarantee to retrieve the correct passage(s) for a given query. |
+| Licensing: | This model and its associated configuration files are licensed under the [OpenMDW License Agreement, version 1.1 (OpenMDW-1.1)](https://openmdw.ai/license/1-1/). Additional Information: Built with [Ministral-3-8B-Instruct-2512](https://huggingface.co/mistralai/Ministral-3-8B-Instruct-2512) which is released under Apache 2.0. |
+
+## Privacy
+
+| Field | Response |
+| ----- | ----- |
+| Generatable or reverse engineerable personal data? | None |
+| Was consent obtained for any personal data used? | Not Applicable |
+| Personal data used to create this model? | None Known |
+| How often is the dataset reviewed? | Before Every Release |
+| Is there provenance for all datasets used in training? | Yes |
+| Does data labeling (annotation, metadata) comply with privacy laws? | Yes |
+| Was data from user interactions with the AI model (e.g. user input and prompts) used to train the model? | Yes |
+| Is data compliant with data subject requests for data correction or removal, if such a request was made? | No, not possible with externally-sourced data. |
+| Applicable Privacy Policy | https://www.nvidia.com/en-us/about-nvidia/privacy-policy/ |
+
+## Safety
+
+| Field | Response |
+| ----- | ----- |
+| Model Application(s): | Text Embedding for Retrieval |
+| Describe the physical safety impact (if present). | Not Applicable |
+| Use Case Restrictions: | This model and its associated configuration files are licensed under the [OpenMDW License Agreement, version 1.1 (OpenMDW-1.1)](https://openmdw.ai/license/1-1/). Additional Information: Built with [Ministral-3-8B-Instruct-2512](https://huggingface.co/mistralai/Ministral-3-8B-Instruct-2512) which is released under Apache 2.0. |
+| Model and dataset restrictions: | The Principle of least privilege (PoLP) is applied limiting access for dataset generation and model development. Restrictions enforce dataset access during training, and dataset license constraints adhered to. |

THIRD_PARTY_NOTICES.md

@@ -0,0 +1,471 @@
+Copyright "Hugging Face, Inc." - Apache License 2.0
+
+License Text([https://github.com/huggingface/sentence-transformers/blob/main/LICENSE](https://github.com/huggingface/sentence-transformers/blob/main/LICENSE))
+
+This notice applies to **sentence-transformers**.
+
+Copyright 2019-2025
+
+Ubiquitous Knowledge Processing (UKP) Lab
+
+Technische Universität Darmstadt
+
+Copyright 2025-present
+
+Hugging Face, Inc.
+
+                                Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+-----
+
+
+Copyright "Hugging Face, Inc." - Apache License 2.0
+License Text([https://github.com/huggingface/transformers/blob/main/LICENSE](https://github.com/huggingface/transformers/blob/main/LICENSE))
+
+This notice applies to **transformers**.
+
+Copyright 2018- The Hugging Face team. All rights reserved.
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+-----
+
+Copyright "PyTorch Contributors" - BSD-style License
+License Text([https://github.com/pytorch/pytorch/blob/main/LICENSE](https://www.google.com/search?q=https://github.com/pytorch/pytorch/blob/main/LICENSE))
+
+This notice applies to **torch**.
+
+From PyTorch:
+
+Copyright (c) 2016-     Facebook, Inc            (Adam Paszke)
+Copyright (c) 2014-     Facebook, Inc            (Soumith Chintala)
+Copyright (c) 2011-2014 Idiap Research Institute (Ronan Collobert)
+Copyright (c) 2012-2014 Deepmind Technologies    (Koray Kavukcuoglu)
+Copyright (c) 2011-2012 NEC Laboratories America (Koray Kavukcuoglu)
+Copyright (c) 2011-2013 NYU                      (Clement Farabet)
+Copyright (c) 2006-2010 NEC Laboratories America (Ronan Collobert, Leon Bottou, Iain Melvin, Jason Weston)
+Copyright (c) 2006      Idiap Research Institute (Samy Bengio)
+Copyright (c) 2001-2004 Idiap Research Institute (Ronan Collobert, Samy Bengio, Johnny Mariethoz)
+
+From Caffe2:
+
+Copyright (c) 2016-present, Facebook Inc. All rights reserved.
+
+All contributions by Facebook:
+Copyright (c) 2016 Facebook Inc.
+
+All contributions by Google:
+Copyright (c) 2015 Google Inc.
+All rights reserved.
+
+All contributions by Yangqing Jia:
+Copyright (c) 2015 Yangqing Jia
+All rights reserved.
+
+All contributions by Kakao Brain:
+Copyright 2019-2020 Kakao Brain
+
+All contributions by Cruise LLC:
+Copyright (c) 2022 Cruise LLC.
+All rights reserved.
+
+All contributions by Tri Dao:
+Copyright (c) 2024 Tri Dao.
+All rights reserved.
+
+All contributions by Arm:
+Copyright (c) 2021, 2023-2025 Arm Limited and/or its affiliates
+
+All contributions from Caffe:
+Copyright(c) 2013, 2014, 2015, the respective contributors
+All rights reserved.
+
+All other contributions:
+Copyright(c) 2015, 2016 the respective contributors
+All rights reserved.
+
+Caffe2 uses a copyright model similar to Caffe: each contributor holds
+copyright over their contributions to Caffe2. The project versioning records
+all such contribution and copyright details. If a contributor wants to further
+mark their specific copyright on a particular contribution, they should
+indicate their copyright solely in the commit message of the change when it is
+committed.
+
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright
+   notice, this list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright
+   notice, this list of conditions and the following disclaimer in the
+   documentation and/or other materials provided with the distribution.
+
+3. Neither the names of Facebook, Deepmind Technologies, NYU, NEC Laboratories America
+   and IDIAP Research Institute nor the names of its contributors may be
+   used to endorse or promote products derived from this software without
+   specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
+LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGE.

config.json

@@ -0,0 +1,47 @@
+{
+  "architectures": [
+    "Ministral3Model"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "is_causal": false,
+  "llama_4_scaling": {
+    "beta": 0.1,
+    "original_max_position_embeddings": 16384
+  },
+  "max_position_embeddings": 262144,
+  "model_type": "ministral3",
+  "nemo_version": "0.3.0rc0",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 34,
+  "num_key_value_heads": 8,
+  "pad_token_id": 11,
+  "pooling": "avg",
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "apply_yarn_scaling": false,
+    "beta_fast": 32.0,
+    "beta_slow": 1.0,
+    "factor": 16.0,
+    "llama_4_scaling_beta": 0.1,
+    "mscale": 1.0,
+    "mscale_all_dim": 1.0,
+    "original_max_position_embeddings": 16384,
+    "rope_theta": 1000000.0,
+    "rope_type": "yarn",
+    "type": "yarn"
+  },
+  "rope_theta": 1000000.0,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.5.0",
+  "use_cache": true,
+  "vocab_size": 131072
+}

config_sentence_transformers.json

@@ -0,0 +1,13 @@
+{
+  "__version__": {
+    "sentence_transformers": "5.4.1",
+    "transformers": "5.5.0",
+    "pytorch": "2.11.0+cu130"
+  },
+  "prompts": {
+    "query": "query: ",
+    "document": "passage: "
+  },
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model-00001-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f76c1359913d8bd47a00b84fa00e078369beed6eee4eec8bcd9662b59f79257
+size 4966645872

model-00002-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b6a31ecd38827c15c86645477f3384d2389940adac6c73d5ffbd1fd924bf4bc
+size 4932506104

model-00003-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:46321d661b89b96ec19f62aa526561c9b39b4f2847168ef617737429da4901c2
+size 4932506112

model-00004-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb9b7f0e415cc4fb6b744cdcf45f03a623d7e9a1da625a7cab2b32323658dccc
+size 1073741960

model.safetensors.index.json

@@ -0,0 +1,315 @@
+{
+  "metadata": {
+    "total_size": 15905366016
+  },
+  "weight_map": {
+    "embed_tokens.weight": "model-00004-of-00004.safetensors",
+    "layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.0.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.0.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.0.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.0.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.1.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.1.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.1.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.1.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.10.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.10.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.10.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.10.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.10.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.10.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.10.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.10.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.10.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.11.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.11.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.11.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.11.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.11.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.11.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.11.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.11.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.11.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.12.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.12.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.12.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.12.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.12.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.12.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.12.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.12.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.12.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.13.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.13.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.13.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.13.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.13.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.13.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.13.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.13.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.14.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.14.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.14.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.14.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.14.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.14.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.14.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.14.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.14.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.15.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.15.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.15.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.15.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.15.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.15.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.15.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.15.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.15.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.16.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.16.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.16.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.16.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.16.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.16.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.16.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.16.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.16.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.17.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.17.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.17.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.17.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.17.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.17.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.17.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.17.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.17.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.18.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.18.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.18.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.18.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.18.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.18.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.18.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.18.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.18.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.19.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.19.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.19.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.19.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.19.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.19.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.19.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.19.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.19.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.2.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.2.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.2.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.2.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.2.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.2.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.2.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.2.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.2.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.20.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.20.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.20.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.20.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.20.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.20.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.20.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.20.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.20.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.21.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.21.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.21.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.21.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.21.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.21.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.21.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.21.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.21.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.22.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.22.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.22.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.22.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.22.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.22.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.22.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.22.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.22.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.23.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.23.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.23.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.23.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.23.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.23.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.23.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.23.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.23.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.24.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.24.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.24.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.24.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.24.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.24.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.24.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.24.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.24.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.25.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.25.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.25.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.25.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.25.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.25.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.25.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.25.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.25.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.26.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.26.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.26.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.26.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.26.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.26.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.26.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.26.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.26.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.27.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.27.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.27.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.27.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.27.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.27.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.27.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.27.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.27.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.28.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.28.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.28.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.28.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.28.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.28.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.28.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.28.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.28.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.29.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.29.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.29.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.29.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.29.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.29.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.29.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.29.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.29.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.3.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.3.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.3.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.3.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.3.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.3.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.3.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.3.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.3.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.30.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.30.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.30.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.30.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.30.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.30.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.30.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.30.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.30.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.31.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.31.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.31.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.31.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.31.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.31.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.31.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.31.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.31.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.32.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.32.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.32.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.32.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.32.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.32.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.32.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.32.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.32.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.33.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.33.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.33.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.33.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "layers.33.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.33.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.33.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.33.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.33.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.4.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.4.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.4.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.4.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.4.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.4.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.4.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.4.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.4.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.5.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.5.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.5.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.5.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.5.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.5.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.5.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.5.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.5.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.6.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.6.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.6.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.6.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.6.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.6.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.6.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.6.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.6.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.7.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.7.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.7.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.7.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.7.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.7.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.7.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.7.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.7.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.8.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.8.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.8.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.8.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.8.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.8.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.8.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.8.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.8.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.9.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.9.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.9.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.9.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "layers.9.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "layers.9.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.9.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.9.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "layers.9.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "norm.weight": "model-00001-of-00004.safetensors"
+  }
+}

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 32768,
+  "do_lower_case": false
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:797410dfb649a5b9ba92bc4fef7dbf4022d00e73de6867c4ac199a8846439421
+size 17077678

tokenizer_config.json

@@ -0,0 +1,1014 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "eos_token": "</s>",
+  "extra_special_tokens": [
+    "<unk>",
+    "<s>",
+    "</s>",
+    "[INST]",
+    "[/INST]",
+    "[AVAILABLE_TOOLS]",
+    "[/AVAILABLE_TOOLS]",
+    "[TOOL_RESULTS]",
+    "[/TOOL_RESULTS]",
+    "[TOOL_CALLS]",
+    "[IMG]",
+    "<pad>",
+    "[IMG_BREAK]",
+    "[IMG_END]",
+    "[PREFIX]",
+    "[MIDDLE]",
+    "[SUFFIX]",
+    "[SYSTEM_PROMPT]",
+    "[/SYSTEM_PROMPT]",
+    "[TOOL_CONTENT]",
+    "<SPECIAL_20>",
+    "<SPECIAL_21>",
+    "<SPECIAL_22>",
+    "<SPECIAL_23>",
+    "[AUDIO]",
+    "[BEGIN_AUDIO]",
+    "<SPECIAL_26>",
+    "<SPECIAL_27>",
+    "<SPECIAL_28>",
+    "<SPECIAL_29>",
+    "<SPECIAL_30>",
+    "<SPECIAL_31>",
+    "[ARGS]",
+    "[CALL_ID]",
+    "[THINK]",
+    "[/THINK]",
+    "<SPECIAL_36>",
+    "<SPECIAL_37>",
+    "<SPECIAL_38>",
+    "<SPECIAL_39>",
+    "<SPECIAL_40>",
+    "<SPECIAL_41>",
+    "<SPECIAL_42>",
+    "<SPECIAL_43>",
+    "<SPECIAL_44>",
+    "<SPECIAL_45>",
+    "<SPECIAL_46>",
+    "<SPECIAL_47>",
+    "<SPECIAL_48>",
+    "<SPECIAL_49>",
+    "<SPECIAL_50>",
+    "<SPECIAL_51>",
+    "<SPECIAL_52>",
+    "<SPECIAL_53>",
+    "<SPECIAL_54>",
+    "<SPECIAL_55>",
+    "<SPECIAL_56>",
+    "<SPECIAL_57>",
+    "<SPECIAL_58>",
+    "<SPECIAL_59>",
+    "<SPECIAL_60>",
+    "<SPECIAL_61>",
+    "<SPECIAL_62>",
+    "<SPECIAL_63>",
+    "<SPECIAL_64>",
+    "<SPECIAL_65>",
+    "<SPECIAL_66>",
+    "<SPECIAL_67>",
+    "<SPECIAL_68>",
+    "<SPECIAL_69>",
+    "<SPECIAL_70>",
+    "<SPECIAL_71>",
+    "<SPECIAL_72>",
+    "<SPECIAL_73>",
+    "<SPECIAL_74>",
+    "<SPECIAL_75>",
+    "<SPECIAL_76>",
+    "<SPECIAL_77>",
+    "<SPECIAL_78>",
+    "<SPECIAL_79>",
+    "<SPECIAL_80>",
+    "<SPECIAL_81>",
+    "<SPECIAL_82>",
+    "<SPECIAL_83>",
+    "<SPECIAL_84>",
+    "<SPECIAL_85>",
+    "<SPECIAL_86>",
+    "<SPECIAL_87>",
+    "<SPECIAL_88>",
+    "<SPECIAL_89>",
+    "<SPECIAL_90>",
+    "<SPECIAL_91>",
+    "<SPECIAL_92>",
+    "<SPECIAL_93>",
+    "<SPECIAL_94>",
+    "<SPECIAL_95>",
+    "<SPECIAL_96>",
+    "<SPECIAL_97>",
+    "<SPECIAL_98>",
+    "<SPECIAL_99>",
+    "<SPECIAL_100>",
+    "<SPECIAL_101>",
+    "<SPECIAL_102>",
+    "<SPECIAL_103>",
+    "<SPECIAL_104>",
+    "<SPECIAL_105>",
+    "<SPECIAL_106>",
+    "<SPECIAL_107>",
+    "<SPECIAL_108>",
+    "<SPECIAL_109>",
+    "<SPECIAL_110>",
+    "<SPECIAL_111>",
+    "<SPECIAL_112>",
+    "<SPECIAL_113>",
+    "<SPECIAL_114>",
+    "<SPECIAL_115>",
+    "<SPECIAL_116>",
+    "<SPECIAL_117>",
+    "<SPECIAL_118>",
+    "<SPECIAL_119>",
+    "<SPECIAL_120>",
+    "<SPECIAL_121>",
+    "<SPECIAL_122>",
+    "<SPECIAL_123>",
+    "<SPECIAL_124>",
+    "<SPECIAL_125>",
+    "<SPECIAL_126>",
+    "<SPECIAL_127>",
+    "<SPECIAL_128>",
+    "<SPECIAL_129>",
+    "<SPECIAL_130>",
+    "<SPECIAL_131>",
+    "<SPECIAL_132>",
+    "<SPECIAL_133>",
+    "<SPECIAL_134>",
+    "<SPECIAL_135>",
+    "<SPECIAL_136>",
+    "<SPECIAL_137>",
+    "<SPECIAL_138>",
+    "<SPECIAL_139>",
+    "<SPECIAL_140>",
+    "<SPECIAL_141>",
+    "<SPECIAL_142>",
+    "<SPECIAL_143>",
+    "<SPECIAL_144>",
+    "<SPECIAL_145>",
+    "<SPECIAL_146>",
+    "<SPECIAL_147>",
+    "<SPECIAL_148>",
+    "<SPECIAL_149>",
+    "<SPECIAL_150>",
+    "<SPECIAL_151>",
+    "<SPECIAL_152>",
+    "<SPECIAL_153>",
+    "<SPECIAL_154>",
+    "<SPECIAL_155>",
+    "<SPECIAL_156>",
+    "<SPECIAL_157>",
+    "<SPECIAL_158>",
+    "<SPECIAL_159>",
+    "<SPECIAL_160>",
+    "<SPECIAL_161>",
+    "<SPECIAL_162>",
+    "<SPECIAL_163>",
+    "<SPECIAL_164>",
+    "<SPECIAL_165>",
+    "<SPECIAL_166>",
+    "<SPECIAL_167>",
+    "<SPECIAL_168>",
+    "<SPECIAL_169>",
+    "<SPECIAL_170>",
+    "<SPECIAL_171>",
+    "<SPECIAL_172>",
+    "<SPECIAL_173>",
+    "<SPECIAL_174>",
+    "<SPECIAL_175>",
+    "<SPECIAL_176>",
+    "<SPECIAL_177>",
+    "<SPECIAL_178>",
+    "<SPECIAL_179>",
+    "<SPECIAL_180>",
+    "<SPECIAL_181>",
+    "<SPECIAL_182>",
+    "<SPECIAL_183>",
+    "<SPECIAL_184>",
+    "<SPECIAL_185>",
+    "<SPECIAL_186>",
+    "<SPECIAL_187>",
+    "<SPECIAL_188>",
+    "<SPECIAL_189>",
+    "<SPECIAL_190>",
+    "<SPECIAL_191>",
+    "<SPECIAL_192>",
+    "<SPECIAL_193>",
+    "<SPECIAL_194>",
+    "<SPECIAL_195>",
+    "<SPECIAL_196>",
+    "<SPECIAL_197>",
+    "<SPECIAL_198>",
+    "<SPECIAL_199>",
+    "<SPECIAL_200>",
+    "<SPECIAL_201>",
+    "<SPECIAL_202>",
+    "<SPECIAL_203>",
+    "<SPECIAL_204>",
+    "<SPECIAL_205>",
+    "<SPECIAL_206>",
+    "<SPECIAL_207>",
+    "<SPECIAL_208>",
+    "<SPECIAL_209>",
+    "<SPECIAL_210>",
+    "<SPECIAL_211>",
+    "<SPECIAL_212>",
+    "<SPECIAL_213>",
+    "<SPECIAL_214>",
+    "<SPECIAL_215>",
+    "<SPECIAL_216>",
+    "<SPECIAL_217>",
+    "<SPECIAL_218>",
+    "<SPECIAL_219>",
+    "<SPECIAL_220>",
+    "<SPECIAL_221>",
+    "<SPECIAL_222>",
+    "<SPECIAL_223>",
+    "<SPECIAL_224>",
+    "<SPECIAL_225>",
+    "<SPECIAL_226>",
+    "<SPECIAL_227>",
+    "<SPECIAL_228>",
+    "<SPECIAL_229>",
+    "<SPECIAL_230>",
+    "<SPECIAL_231>",
+    "<SPECIAL_232>",
+    "<SPECIAL_233>",
+    "<SPECIAL_234>",
+    "<SPECIAL_235>",
+    "<SPECIAL_236>",
+    "<SPECIAL_237>",
+    "<SPECIAL_238>",
+    "<SPECIAL_239>",
+    "<SPECIAL_240>",
+    "<SPECIAL_241>",
+    "<SPECIAL_242>",
+    "<SPECIAL_243>",
+    "<SPECIAL_244>",
+    "<SPECIAL_245>",
+    "<SPECIAL_246>",
+    "<SPECIAL_247>",
+    "<SPECIAL_248>",
+    "<SPECIAL_249>",
+    "<SPECIAL_250>",
+    "<SPECIAL_251>",
+    "<SPECIAL_252>",
+    "<SPECIAL_253>",
+    "<SPECIAL_254>",
+    "<SPECIAL_255>",
+    "<SPECIAL_256>",
+    "<SPECIAL_257>",
+    "<SPECIAL_258>",
+    "<SPECIAL_259>",
+    "<SPECIAL_260>",
+    "<SPECIAL_261>",
+    "<SPECIAL_262>",
+    "<SPECIAL_263>",
+    "<SPECIAL_264>",
+    "<SPECIAL_265>",
+    "<SPECIAL_266>",
+    "<SPECIAL_267>",
+    "<SPECIAL_268>",
+    "<SPECIAL_269>",
+    "<SPECIAL_270>",
+    "<SPECIAL_271>",
+    "<SPECIAL_272>",
+    "<SPECIAL_273>",
+    "<SPECIAL_274>",
+    "<SPECIAL_275>",
+    "<SPECIAL_276>",
+    "<SPECIAL_277>",
+    "<SPECIAL_278>",
+    "<SPECIAL_279>",
+    "<SPECIAL_280>",
+    "<SPECIAL_281>",
+    "<SPECIAL_282>",
+    "<SPECIAL_283>",
+    "<SPECIAL_284>",
+    "<SPECIAL_285>",
+    "<SPECIAL_286>",
+    "<SPECIAL_287>",
+    "<SPECIAL_288>",
+    "<SPECIAL_289>",
+    "<SPECIAL_290>",
+    "<SPECIAL_291>",
+    "<SPECIAL_292>",
+    "<SPECIAL_293>",
+    "<SPECIAL_294>",
+    "<SPECIAL_295>",
+    "<SPECIAL_296>",
+    "<SPECIAL_297>",
+    "<SPECIAL_298>",
+    "<SPECIAL_299>",
+    "<SPECIAL_300>",
+    "<SPECIAL_301>",
+    "<SPECIAL_302>",
+    "<SPECIAL_303>",
+    "<SPECIAL_304>",
+    "<SPECIAL_305>",
+    "<SPECIAL_306>",
+    "<SPECIAL_307>",
+    "<SPECIAL_308>",
+    "<SPECIAL_309>",
+    "<SPECIAL_310>",
+    "<SPECIAL_311>",
+    "<SPECIAL_312>",
+    "<SPECIAL_313>",
+    "<SPECIAL_314>",
+    "<SPECIAL_315>",
+    "<SPECIAL_316>",
+    "<SPECIAL_317>",
+    "<SPECIAL_318>",
+    "<SPECIAL_319>",
+    "<SPECIAL_320>",
+    "<SPECIAL_321>",
+    "<SPECIAL_322>",
+    "<SPECIAL_323>",
+    "<SPECIAL_324>",
+    "<SPECIAL_325>",
+    "<SPECIAL_326>",
+    "<SPECIAL_327>",
+    "<SPECIAL_328>",
+    "<SPECIAL_329>",
+    "<SPECIAL_330>",
+    "<SPECIAL_331>",
+    "<SPECIAL_332>",
+    "<SPECIAL_333>",
+    "<SPECIAL_334>",
+    "<SPECIAL_335>",
+    "<SPECIAL_336>",
+    "<SPECIAL_337>",
+    "<SPECIAL_338>",
+    "<SPECIAL_339>",
+    "<SPECIAL_340>",
+    "<SPECIAL_341>",
+    "<SPECIAL_342>",
+    "<SPECIAL_343>",
+    "<SPECIAL_344>",
+    "<SPECIAL_345>",
+    "<SPECIAL_346>",
+    "<SPECIAL_347>",
+    "<SPECIAL_348>",
+    "<SPECIAL_349>",
+    "<SPECIAL_350>",
+    "<SPECIAL_351>",
+    "<SPECIAL_352>",
+    "<SPECIAL_353>",
+    "<SPECIAL_354>",
+    "<SPECIAL_355>",
+    "<SPECIAL_356>",
+    "<SPECIAL_357>",
+    "<SPECIAL_358>",
+    "<SPECIAL_359>",
+    "<SPECIAL_360>",
+    "<SPECIAL_361>",
+    "<SPECIAL_362>",
+    "<SPECIAL_363>",
+    "<SPECIAL_364>",
+    "<SPECIAL_365>",
+    "<SPECIAL_366>",
+    "<SPECIAL_367>",
+    "<SPECIAL_368>",
+    "<SPECIAL_369>",
+    "<SPECIAL_370>",
+    "<SPECIAL_371>",
+    "<SPECIAL_372>",
+    "<SPECIAL_373>",
+    "<SPECIAL_374>",
+    "<SPECIAL_375>",
+    "<SPECIAL_376>",
+    "<SPECIAL_377>",
+    "<SPECIAL_378>",
+    "<SPECIAL_379>",
+    "<SPECIAL_380>",
+    "<SPECIAL_381>",
+    "<SPECIAL_382>",
+    "<SPECIAL_383>",
+    "<SPECIAL_384>",
+    "<SPECIAL_385>",
+    "<SPECIAL_386>",
+    "<SPECIAL_387>",
+    "<SPECIAL_388>",
+    "<SPECIAL_389>",
+    "<SPECIAL_390>",
+    "<SPECIAL_391>",
+    "<SPECIAL_392>",
+    "<SPECIAL_393>",
+    "<SPECIAL_394>",
+    "<SPECIAL_395>",
+    "<SPECIAL_396>",
+    "<SPECIAL_397>",
+    "<SPECIAL_398>",
+    "<SPECIAL_399>",
+    "<SPECIAL_400>",
+    "<SPECIAL_401>",
+    "<SPECIAL_402>",
+    "<SPECIAL_403>",
+    "<SPECIAL_404>",
+    "<SPECIAL_405>",
+    "<SPECIAL_406>",
+    "<SPECIAL_407>",
+    "<SPECIAL_408>",
+    "<SPECIAL_409>",
+    "<SPECIAL_410>",
+    "<SPECIAL_411>",
+    "<SPECIAL_412>",
+    "<SPECIAL_413>",
+    "<SPECIAL_414>",
+    "<SPECIAL_415>",
+    "<SPECIAL_416>",
+    "<SPECIAL_417>",
+    "<SPECIAL_418>",
+    "<SPECIAL_419>",
+    "<SPECIAL_420>",
+    "<SPECIAL_421>",
+    "<SPECIAL_422>",
+    "<SPECIAL_423>",
+    "<SPECIAL_424>",
+    "<SPECIAL_425>",
+    "<SPECIAL_426>",
+    "<SPECIAL_427>",
+    "<SPECIAL_428>",
+    "<SPECIAL_429>",
+    "<SPECIAL_430>",
+    "<SPECIAL_431>",
+    "<SPECIAL_432>",
+    "<SPECIAL_433>",
+    "<SPECIAL_434>",
+    "<SPECIAL_435>",
+    "<SPECIAL_436>",
+    "<SPECIAL_437>",
+    "<SPECIAL_438>",
+    "<SPECIAL_439>",
+    "<SPECIAL_440>",
+    "<SPECIAL_441>",
+    "<SPECIAL_442>",
+    "<SPECIAL_443>",
+    "<SPECIAL_444>",
+    "<SPECIAL_445>",
+    "<SPECIAL_446>",
+    "<SPECIAL_447>",
+    "<SPECIAL_448>",
+    "<SPECIAL_449>",
+    "<SPECIAL_450>",
+    "<SPECIAL_451>",
+    "<SPECIAL_452>",
+    "<SPECIAL_453>",
+    "<SPECIAL_454>",
+    "<SPECIAL_455>",
+    "<SPECIAL_456>",
+    "<SPECIAL_457>",
+    "<SPECIAL_458>",
+    "<SPECIAL_459>",
+    "<SPECIAL_460>",
+    "<SPECIAL_461>",
+    "<SPECIAL_462>",
+    "<SPECIAL_463>",
+    "<SPECIAL_464>",
+    "<SPECIAL_465>",
+    "<SPECIAL_466>",
+    "<SPECIAL_467>",
+    "<SPECIAL_468>",
+    "<SPECIAL_469>",
+    "<SPECIAL_470>",
+    "<SPECIAL_471>",
+    "<SPECIAL_472>",
+    "<SPECIAL_473>",
+    "<SPECIAL_474>",
+    "<SPECIAL_475>",
+    "<SPECIAL_476>",
+    "<SPECIAL_477>",
+    "<SPECIAL_478>",
+    "<SPECIAL_479>",
+    "<SPECIAL_480>",
+    "<SPECIAL_481>",
+    "<SPECIAL_482>",
+    "<SPECIAL_483>",
+    "<SPECIAL_484>",
+    "<SPECIAL_485>",
+    "<SPECIAL_486>",
+    "<SPECIAL_487>",
+    "<SPECIAL_488>",
+    "<SPECIAL_489>",
+    "<SPECIAL_490>",
+    "<SPECIAL_491>",
+    "<SPECIAL_492>",
+    "<SPECIAL_493>",
+    "<SPECIAL_494>",
+    "<SPECIAL_495>",
+    "<SPECIAL_496>",
+    "<SPECIAL_497>",
+    "<SPECIAL_498>",
+    "<SPECIAL_499>",
+    "<SPECIAL_500>",
+    "<SPECIAL_501>",
+    "<SPECIAL_502>",
+    "<SPECIAL_503>",
+    "<SPECIAL_504>",
+    "<SPECIAL_505>",
+    "<SPECIAL_506>",
+    "<SPECIAL_507>",
+    "<SPECIAL_508>",
+    "<SPECIAL_509>",
+    "<SPECIAL_510>",
+    "<SPECIAL_511>",
+    "<SPECIAL_512>",
+    "<SPECIAL_513>",
+    "<SPECIAL_514>",
+    "<SPECIAL_515>",
+    "<SPECIAL_516>",
+    "<SPECIAL_517>",
+    "<SPECIAL_518>",
+    "<SPECIAL_519>",
+    "<SPECIAL_520>",
+    "<SPECIAL_521>",
+    "<SPECIAL_522>",
+    "<SPECIAL_523>",
+    "<SPECIAL_524>",
+    "<SPECIAL_525>",
+    "<SPECIAL_526>",
+    "<SPECIAL_527>",
+    "<SPECIAL_528>",
+    "<SPECIAL_529>",
+    "<SPECIAL_530>",
+    "<SPECIAL_531>",
+    "<SPECIAL_532>",
+    "<SPECIAL_533>",
+    "<SPECIAL_534>",
+    "<SPECIAL_535>",
+    "<SPECIAL_536>",
+    "<SPECIAL_537>",
+    "<SPECIAL_538>",
+    "<SPECIAL_539>",
+    "<SPECIAL_540>",
+    "<SPECIAL_541>",
+    "<SPECIAL_542>",
+    "<SPECIAL_543>",
+    "<SPECIAL_544>",
+    "<SPECIAL_545>",
+    "<SPECIAL_546>",
+    "<SPECIAL_547>",
+    "<SPECIAL_548>",
+    "<SPECIAL_549>",
+    "<SPECIAL_550>",
+    "<SPECIAL_551>",
+    "<SPECIAL_552>",
+    "<SPECIAL_553>",
+    "<SPECIAL_554>",
+    "<SPECIAL_555>",
+    "<SPECIAL_556>",
+    "<SPECIAL_557>",
+    "<SPECIAL_558>",
+    "<SPECIAL_559>",
+    "<SPECIAL_560>",
+    "<SPECIAL_561>",
+    "<SPECIAL_562>",
+    "<SPECIAL_563>",
+    "<SPECIAL_564>",
+    "<SPECIAL_565>",
+    "<SPECIAL_566>",
+    "<SPECIAL_567>",
+    "<SPECIAL_568>",
+    "<SPECIAL_569>",
+    "<SPECIAL_570>",
+    "<SPECIAL_571>",
+    "<SPECIAL_572>",
+    "<SPECIAL_573>",
+    "<SPECIAL_574>",
+    "<SPECIAL_575>",
+    "<SPECIAL_576>",
+    "<SPECIAL_577>",
+    "<SPECIAL_578>",
+    "<SPECIAL_579>",
+    "<SPECIAL_580>",
+    "<SPECIAL_581>",
+    "<SPECIAL_582>",
+    "<SPECIAL_583>",
+    "<SPECIAL_584>",
+    "<SPECIAL_585>",
+    "<SPECIAL_586>",
+    "<SPECIAL_587>",
+    "<SPECIAL_588>",
+    "<SPECIAL_589>",
+    "<SPECIAL_590>",
+    "<SPECIAL_591>",
+    "<SPECIAL_592>",
+    "<SPECIAL_593>",
+    "<SPECIAL_594>",
+    "<SPECIAL_595>",
+    "<SPECIAL_596>",
+    "<SPECIAL_597>",
+    "<SPECIAL_598>",
+    "<SPECIAL_599>",
+    "<SPECIAL_600>",
+    "<SPECIAL_601>",
+    "<SPECIAL_602>",
+    "<SPECIAL_603>",
+    "<SPECIAL_604>",
+    "<SPECIAL_605>",
+    "<SPECIAL_606>",
+    "<SPECIAL_607>",
+    "<SPECIAL_608>",
+    "<SPECIAL_609>",
+    "<SPECIAL_610>",
+    "<SPECIAL_611>",
+    "<SPECIAL_612>",
+    "<SPECIAL_613>",
+    "<SPECIAL_614>",
+    "<SPECIAL_615>",
+    "<SPECIAL_616>",
+    "<SPECIAL_617>",
+    "<SPECIAL_618>",
+    "<SPECIAL_619>",
+    "<SPECIAL_620>",
+    "<SPECIAL_621>",
+    "<SPECIAL_622>",
+    "<SPECIAL_623>",
+    "<SPECIAL_624>",
+    "<SPECIAL_625>",
+    "<SPECIAL_626>",
+    "<SPECIAL_627>",
+    "<SPECIAL_628>",
+    "<SPECIAL_629>",
+    "<SPECIAL_630>",
+    "<SPECIAL_631>",
+    "<SPECIAL_632>",
+    "<SPECIAL_633>",
+    "<SPECIAL_634>",
+    "<SPECIAL_635>",
+    "<SPECIAL_636>",
+    "<SPECIAL_637>",
+    "<SPECIAL_638>",
+    "<SPECIAL_639>",
+    "<SPECIAL_640>",
+    "<SPECIAL_641>",
+    "<SPECIAL_642>",
+    "<SPECIAL_643>",
+    "<SPECIAL_644>",
+    "<SPECIAL_645>",
+    "<SPECIAL_646>",
+    "<SPECIAL_647>",
+    "<SPECIAL_648>",
+    "<SPECIAL_649>",
+    "<SPECIAL_650>",
+    "<SPECIAL_651>",
+    "<SPECIAL_652>",
+    "<SPECIAL_653>",
+    "<SPECIAL_654>",
+    "<SPECIAL_655>",
+    "<SPECIAL_656>",
+    "<SPECIAL_657>",
+    "<SPECIAL_658>",
+    "<SPECIAL_659>",
+    "<SPECIAL_660>",
+    "<SPECIAL_661>",
+    "<SPECIAL_662>",
+    "<SPECIAL_663>",
+    "<SPECIAL_664>",
+    "<SPECIAL_665>",
+    "<SPECIAL_666>",
+    "<SPECIAL_667>",
+    "<SPECIAL_668>",
+    "<SPECIAL_669>",
+    "<SPECIAL_670>",
+    "<SPECIAL_671>",
+    "<SPECIAL_672>",
+    "<SPECIAL_673>",
+    "<SPECIAL_674>",
+    "<SPECIAL_675>",
+    "<SPECIAL_676>",
+    "<SPECIAL_677>",
+    "<SPECIAL_678>",
+    "<SPECIAL_679>",
+    "<SPECIAL_680>",
+    "<SPECIAL_681>",
+    "<SPECIAL_682>",
+    "<SPECIAL_683>",
+    "<SPECIAL_684>",
+    "<SPECIAL_685>",
+    "<SPECIAL_686>",
+    "<SPECIAL_687>",
+    "<SPECIAL_688>",
+    "<SPECIAL_689>",
+    "<SPECIAL_690>",
+    "<SPECIAL_691>",
+    "<SPECIAL_692>",
+    "<SPECIAL_693>",
+    "<SPECIAL_694>",
+    "<SPECIAL_695>",
+    "<SPECIAL_696>",
+    "<SPECIAL_697>",
+    "<SPECIAL_698>",
+    "<SPECIAL_699>",
+    "<SPECIAL_700>",
+    "<SPECIAL_701>",
+    "<SPECIAL_702>",
+    "<SPECIAL_703>",
+    "<SPECIAL_704>",
+    "<SPECIAL_705>",
+    "<SPECIAL_706>",
+    "<SPECIAL_707>",
+    "<SPECIAL_708>",
+    "<SPECIAL_709>",
+    "<SPECIAL_710>",
+    "<SPECIAL_711>",
+    "<SPECIAL_712>",
+    "<SPECIAL_713>",
+    "<SPECIAL_714>",
+    "<SPECIAL_715>",
+    "<SPECIAL_716>",
+    "<SPECIAL_717>",
+    "<SPECIAL_718>",
+    "<SPECIAL_719>",
+    "<SPECIAL_720>",
+    "<SPECIAL_721>",
+    "<SPECIAL_722>",
+    "<SPECIAL_723>",
+    "<SPECIAL_724>",
+    "<SPECIAL_725>",
+    "<SPECIAL_726>",
+    "<SPECIAL_727>",
+    "<SPECIAL_728>",
+    "<SPECIAL_729>",
+    "<SPECIAL_730>",
+    "<SPECIAL_731>",
+    "<SPECIAL_732>",
+    "<SPECIAL_733>",
+    "<SPECIAL_734>",
+    "<SPECIAL_735>",
+    "<SPECIAL_736>",
+    "<SPECIAL_737>",
+    "<SPECIAL_738>",
+    "<SPECIAL_739>",
+    "<SPECIAL_740>",
+    "<SPECIAL_741>",
+    "<SPECIAL_742>",
+    "<SPECIAL_743>",
+    "<SPECIAL_744>",
+    "<SPECIAL_745>",
+    "<SPECIAL_746>",
+    "<SPECIAL_747>",
+    "<SPECIAL_748>",
+    "<SPECIAL_749>",
+    "<SPECIAL_750>",
+    "<SPECIAL_751>",
+    "<SPECIAL_752>",
+    "<SPECIAL_753>",
+    "<SPECIAL_754>",
+    "<SPECIAL_755>",
+    "<SPECIAL_756>",
+    "<SPECIAL_757>",
+    "<SPECIAL_758>",
+    "<SPECIAL_759>",
+    "<SPECIAL_760>",
+    "<SPECIAL_761>",
+    "<SPECIAL_762>",
+    "<SPECIAL_763>",
+    "<SPECIAL_764>",
+    "<SPECIAL_765>",
+    "<SPECIAL_766>",
+    "<SPECIAL_767>",
+    "<SPECIAL_768>",
+    "<SPECIAL_769>",
+    "<SPECIAL_770>",
+    "<SPECIAL_771>",
+    "<SPECIAL_772>",
+    "<SPECIAL_773>",
+    "<SPECIAL_774>",
+    "<SPECIAL_775>",
+    "<SPECIAL_776>",
+    "<SPECIAL_777>",
+    "<SPECIAL_778>",
+    "<SPECIAL_779>",
+    "<SPECIAL_780>",
+    "<SPECIAL_781>",
+    "<SPECIAL_782>",
+    "<SPECIAL_783>",
+    "<SPECIAL_784>",
+    "<SPECIAL_785>",
+    "<SPECIAL_786>",
+    "<SPECIAL_787>",
+    "<SPECIAL_788>",
+    "<SPECIAL_789>",
+    "<SPECIAL_790>",
+    "<SPECIAL_791>",
+    "<SPECIAL_792>",
+    "<SPECIAL_793>",
+    "<SPECIAL_794>",
+    "<SPECIAL_795>",
+    "<SPECIAL_796>",
+    "<SPECIAL_797>",
+    "<SPECIAL_798>",
+    "<SPECIAL_799>",
+    "<SPECIAL_800>",
+    "<SPECIAL_801>",
+    "<SPECIAL_802>",
+    "<SPECIAL_803>",
+    "<SPECIAL_804>",
+    "<SPECIAL_805>",
+    "<SPECIAL_806>",
+    "<SPECIAL_807>",
+    "<SPECIAL_808>",
+    "<SPECIAL_809>",
+    "<SPECIAL_810>",
+    "<SPECIAL_811>",
+    "<SPECIAL_812>",
+    "<SPECIAL_813>",
+    "<SPECIAL_814>",
+    "<SPECIAL_815>",
+    "<SPECIAL_816>",
+    "<SPECIAL_817>",
+    "<SPECIAL_818>",
+    "<SPECIAL_819>",
+    "<SPECIAL_820>",
+    "<SPECIAL_821>",
+    "<SPECIAL_822>",
+    "<SPECIAL_823>",
+    "<SPECIAL_824>",
+    "<SPECIAL_825>",
+    "<SPECIAL_826>",
+    "<SPECIAL_827>",
+    "<SPECIAL_828>",
+    "<SPECIAL_829>",
+    "<SPECIAL_830>",
+    "<SPECIAL_831>",
+    "<SPECIAL_832>",
+    "<SPECIAL_833>",
+    "<SPECIAL_834>",
+    "<SPECIAL_835>",
+    "<SPECIAL_836>",
+    "<SPECIAL_837>",
+    "<SPECIAL_838>",
+    "<SPECIAL_839>",
+    "<SPECIAL_840>",
+    "<SPECIAL_841>",
+    "<SPECIAL_842>",
+    "<SPECIAL_843>",
+    "<SPECIAL_844>",
+    "<SPECIAL_845>",
+    "<SPECIAL_846>",
+    "<SPECIAL_847>",
+    "<SPECIAL_848>",
+    "<SPECIAL_849>",
+    "<SPECIAL_850>",
+    "<SPECIAL_851>",
+    "<SPECIAL_852>",
+    "<SPECIAL_853>",
+    "<SPECIAL_854>",
+    "<SPECIAL_855>",
+    "<SPECIAL_856>",
+    "<SPECIAL_857>",
+    "<SPECIAL_858>",
+    "<SPECIAL_859>",
+    "<SPECIAL_860>",
+    "<SPECIAL_861>",
+    "<SPECIAL_862>",
+    "<SPECIAL_863>",
+    "<SPECIAL_864>",
+    "<SPECIAL_865>",
+    "<SPECIAL_866>",
+    "<SPECIAL_867>",
+    "<SPECIAL_868>",
+    "<SPECIAL_869>",
+    "<SPECIAL_870>",
+    "<SPECIAL_871>",
+    "<SPECIAL_872>",
+    "<SPECIAL_873>",
+    "<SPECIAL_874>",
+    "<SPECIAL_875>",
+    "<SPECIAL_876>",
+    "<SPECIAL_877>",
+    "<SPECIAL_878>",
+    "<SPECIAL_879>",
+    "<SPECIAL_880>",
+    "<SPECIAL_881>",
+    "<SPECIAL_882>",
+    "<SPECIAL_883>",
+    "<SPECIAL_884>",
+    "<SPECIAL_885>",
+    "<SPECIAL_886>",
+    "<SPECIAL_887>",
+    "<SPECIAL_888>",
+    "<SPECIAL_889>",
+    "<SPECIAL_890>",
+    "<SPECIAL_891>",
+    "<SPECIAL_892>",
+    "<SPECIAL_893>",
+    "<SPECIAL_894>",
+    "<SPECIAL_895>",
+    "<SPECIAL_896>",
+    "<SPECIAL_897>",
+    "<SPECIAL_898>",
+    "<SPECIAL_899>",
+    "<SPECIAL_900>",
+    "<SPECIAL_901>",
+    "<SPECIAL_902>",
+    "<SPECIAL_903>",
+    "<SPECIAL_904>",
+    "<SPECIAL_905>",
+    "<SPECIAL_906>",
+    "<SPECIAL_907>",
+    "<SPECIAL_908>",
+    "<SPECIAL_909>",
+    "<SPECIAL_910>",
+    "<SPECIAL_911>",
+    "<SPECIAL_912>",
+    "<SPECIAL_913>",
+    "<SPECIAL_914>",
+    "<SPECIAL_915>",
+    "<SPECIAL_916>",
+    "<SPECIAL_917>",
+    "<SPECIAL_918>",
+    "<SPECIAL_919>",
+    "<SPECIAL_920>",
+    "<SPECIAL_921>",
+    "<SPECIAL_922>",
+    "<SPECIAL_923>",
+    "<SPECIAL_924>",
+    "<SPECIAL_925>",
+    "<SPECIAL_926>",
+    "<SPECIAL_927>",
+    "<SPECIAL_928>",
+    "<SPECIAL_929>",
+    "<SPECIAL_930>",
+    "<SPECIAL_931>",
+    "<SPECIAL_932>",
+    "<SPECIAL_933>",
+    "<SPECIAL_934>",
+    "<SPECIAL_935>",
+    "<SPECIAL_936>",
+    "<SPECIAL_937>",
+    "<SPECIAL_938>",
+    "<SPECIAL_939>",
+    "<SPECIAL_940>",
+    "<SPECIAL_941>",
+    "<SPECIAL_942>",
+    "<SPECIAL_943>",
+    "<SPECIAL_944>",
+    "<SPECIAL_945>",
+    "<SPECIAL_946>",
+    "<SPECIAL_947>",
+    "<SPECIAL_948>",
+    "<SPECIAL_949>",
+    "<SPECIAL_950>",
+    "<SPECIAL_951>",
+    "<SPECIAL_952>",
+    "<SPECIAL_953>",
+    "<SPECIAL_954>",
+    "<SPECIAL_955>",
+    "<SPECIAL_956>",
+    "<SPECIAL_957>",
+    "<SPECIAL_958>",
+    "<SPECIAL_959>",
+    "<SPECIAL_960>",
+    "<SPECIAL_961>",
+    "<SPECIAL_962>",
+    "<SPECIAL_963>",
+    "<SPECIAL_964>",
+    "<SPECIAL_965>",
+    "<SPECIAL_966>",
+    "<SPECIAL_967>",
+    "<SPECIAL_968>",
+    "<SPECIAL_969>",
+    "<SPECIAL_970>",
+    "<SPECIAL_971>",
+    "<SPECIAL_972>",
+    "<SPECIAL_973>",
+    "<SPECIAL_974>",
+    "<SPECIAL_975>",
+    "<SPECIAL_976>",
+    "<SPECIAL_977>",
+    "<SPECIAL_978>",
+    "<SPECIAL_979>",
+    "<SPECIAL_980>",
+    "<SPECIAL_981>",
+    "<SPECIAL_982>",
+    "<SPECIAL_983>",
+    "<SPECIAL_984>",
+    "<SPECIAL_985>",
+    "<SPECIAL_986>",
+    "<SPECIAL_987>",
+    "<SPECIAL_988>",
+    "<SPECIAL_989>",
+    "<SPECIAL_990>",
+    "<SPECIAL_991>",
+    "<SPECIAL_992>",
+    "<SPECIAL_993>",
+    "<SPECIAL_994>",
+    "<SPECIAL_995>",
+    "<SPECIAL_996>",
+    "<SPECIAL_997>",
+    "<SPECIAL_998>",
+    "<SPECIAL_999>"
+  ],
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 32768,
+  "pad_token": "<pad>",
+  "padding_side": "left",
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<unk>"
+}