Update README.md
Browse files
README.md
CHANGED
|
@@ -13,6 +13,12 @@ tags:
|
|
| 13 |
This repository contains an NVFP4 quantized version of the
|
| 14 |
[fdtn-ai/Foundation-Sec-8B-Instruct](https://huggingface.co/fdtn-ai/Foundation-Sec-8B-Instruct) model, optimized for NVIDIA Spark using TensorRT Model Optimizer.
|
| 15 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
## Quantization Details
|
| 17 |
- Quantization Method: NVFP4
|
| 18 |
- Base Model: [fdtn-ai/Foundation-Sec-8B-Instruct](https://huggingface.co/fdtn-ai/Foundation-Sec-8B-Instruct)
|
|
@@ -23,4 +29,7 @@ This repository contains an NVFP4 quantized version of the
|
|
| 23 |
Refer to TensorRT-LLM or your deployment stack for loading NVFP4 artifacts.
|
| 24 |
|
| 25 |
## License
|
| 26 |
-
(Inherit from base model
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
This repository contains an NVFP4 quantized version of the
|
| 14 |
[fdtn-ai/Foundation-Sec-8B-Instruct](https://huggingface.co/fdtn-ai/Foundation-Sec-8B-Instruct) model, optimized for NVIDIA Spark using TensorRT Model Optimizer.
|
| 15 |
|
| 16 |
+
🚀 Quantizing the Foundation-Sec-8B model to NVFP4 can significantly reduce its memory footprint by up to 3.5 times, allowing it to run on hardware with less VRAM.
|
| 17 |
+
This process also increases inference speed by reducing the memory bandwidth bottleneck and leveraging optimizations specific to NVIDIA's Blackwell architecture.
|
| 18 |
+
Read more about NVFP4 at NVIDIA (https://developer.nvidia.com/blog/introducing-nvfp4-for-efficient-and-accurate-low-precision-inference/)
|
| 19 |
+
|
| 20 |
+
❇️ NVIDIA Pretraining Large Language Models with NVFP4 Paper(https://arxiv.org/abs/2509.25149)
|
| 21 |
+
|
| 22 |
## Quantization Details
|
| 23 |
- Quantization Method: NVFP4
|
| 24 |
- Base Model: [fdtn-ai/Foundation-Sec-8B-Instruct](https://huggingface.co/fdtn-ai/Foundation-Sec-8B-Instruct)
|
|
|
|
| 29 |
Refer to TensorRT-LLM or your deployment stack for loading NVFP4 artifacts.
|
| 30 |
|
| 31 |
## License
|
| 32 |
+
(Inherit from base model)
|
| 33 |
+
|
| 34 |
+
## Contacts
|
| 35 |
+
@guerilla7 | Ron F. Del Rosario | LinkedIn:(https://www.linkedin.com/in/ronaldfloresdelrosario/)
|