README.md

---
license: apache-2.0
language: en
tags:
- openbmb/MiniCPM4-0.5B
- coding
- code-generation
- fine-tuned
- qlora
- gguf
- instruction
- python
datasets:
- TokenBender/code_instructions_122k_alpaca_style
model_type: openbmb/MiniCPM4-0.5B
base_model: openbmb/MiniCPM4-0.5B
---

# MiniCPM4-0.5B-Coding-Finetuned-v1

This model is a fine-tuned version of `openbmb/MiniCPM4-0.5B` specialized for Python code generation tasks. It's designed to understand programming-related instructions and provide accurate and efficient Python code solutions.

## 💻 Model Description

- **Base Model**: `openbmb/MiniCPM4-0.5B`
- **Fine-tuning Method**: **QLoRA** (Quantized Low-Rank Adaptation)
- **Dataset**: `TokenBender/code_instructions_122k_alpaca_style` - A large dataset of coding instructions and their corresponding solutions.
- **Training**: Optimized for instruction-based code generation using 4-bit quantization for efficiency.

## ⚠️ Important Considerations

- **Verify All Code**: Generated code may contain errors or be suboptimal. Always test and review the code thoroughly before using it in production environments.
- **Security**: The generated code has not been vetted for security vulnerabilities. Be cautious when using it in security-sensitive applications.
- **Not a Replacement for Developers**: This model is a tool to assist developers, not replace them. Human oversight and expertise are crucial.

## 🚀 Usage

### With `transformers`

```python
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
import torch

model_id = "rohitnagareddy/MiniCPM4-0.5B-Coding-Finetuned-v1"

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)

# Create conversation for a Python code-generation task
messages = [
    {"role": "system", "content": "You are an expert coding assistant."},
    {"role": "user", "content": "Write a Python function that takes a list of integers and returns the sum of all even numbers in the list."}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer
)

# Generate response
outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
```

## 🔧 GGUF Versions

This repository includes quantized GGUF versions for use with `llama.cpp` and compatible tools:

- `MiniCPM4-0.5B-Coding-Finetuned-v1.fp16.gguf` - Full precision (largest, best quality)
- `MiniCPM4-0.5B-Coding-Finetuned-v1.Q8_0.gguf` - 8-bit quantization (good balance)
- `MiniCPM4-0.5B-Coding-Finetuned-v1.Q5_K_M.gguf` - 5-bit quantization (smaller, fast)
- `MiniCPM4-0.5B-Coding-Finetuned-v1.Q4_K_M.gguf` - 4-bit quantization (smallest, fastest)

### Example with llama.cpp

```bash
./main -m ./MiniCPM4-0.5B-Coding-Finetuned-v1.Q4_K_M.gguf -n 256 -p "<|im_start|>system\nYou are an expert coding assistant.<|im_end|>\n<|im_start|>user\nCreate a Python function to find the factorial of a number.<|im_end|>\n<|im_start|>assistant\n"
```

## 📊 Training Details

- **Training Epochs**: 1
- **QLoRA Rank (r)**: 16
- **QLoRA Alpha**: 32
- **Learning Rate**: 2e-4
- **Optimizer**: Paged AdamW 32-bit
- **Target Modules**: Auto-detected linear layers