---
base_model: Qwen/Qwen2.5-Coder-14B-Instruct
abliterated_from: huihui-ai/Qwen2.5-Coder-14B-Instruct-abliterated
quantization: nf4
license: apache-2.0
tags:
- code
- programming
- uncensored
- 4bit
- nf4
- qwen
- abliteration
- instruct
- gs128
- bitsandbytes
library: transformers
language:
- en
pipeline_tag: text-generation
---

# Qwen2.5-Coder-14B-Instruct-Abliterated-NF4

**A 4-bit (NF4) quantized, abliteration-uncensored version of Qwen2.5-Coder-14B-Instruct**

This model is a **pre-quantized 4-bit NormalFloat4 (NF4)** version of the uncensored [huihui-ai/Qwen2.5-Coder-14B-Instruct-abliterated](https://huggingface.co/huihui-ai/Qwen2.5-Coder-14B-Instruct-abliterated), optimized for **low VRAM** and **fast local inference**.

Ideal for **local deployment**, **edge devices**, or **low-VRAM environments** while maintaining strong coding and reasoning capabilities.

---

## 🚀 Features

- **Base Model**: `Qwen/Qwen2.5-Coder-14B-Instruct`
- **Abliteration**: [huihui-ai](https://huggingface.co/huihui-ai) (uncensored)
- **Quantization**: **NF4 (4-bit)** – **pre-quantized**
- **Efficient**: ~8–9 GB VRAM required for inference
- **Safetensors**: Secure, modern format
- **Framework**: Compatible with `transformers`, `vLLM`, `Oobabooga`, etc.

---

## 📥 Installation & Usage

### 1. Install Dependencies

```bash
pip install transformers torch accelerate
---

---
### Load the Model
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "ikarius/Qwen2.5-Coder-14B-Instruct-Abliterated-NF4"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.bfloat16,  # eller torch.float16
    trust_remote_code=True
)

---
---

# Eksempel
prompt = "Write a Python function to calculate Fibonacci numbers using memoization."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    do_sample=True,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---
---
### Using with text-generation-webui (Oobabooga)

python server.py \
  --model ikarius/Qwen2.5-Coder-14B-Instruct-Abliterated-NF4 \
  --bf16 \
  --trust-remote-code

---

🤝 Credits

Original Model: Qwen Team
Abliteration: huihui-ai
Quantization: ikarius (NF4 via auto-gptq / transformers)
Hosting: Hugging Face Hub

---
📄 License
Apache 2.0 (same as base model)

🙌 Support
⭐ Star this repo if you find it useful!
🐛 Report issues on the Discussions tab.