README.md

---
license: apache-2.0
language:
  - en
  - lt
pipeline_tag: text-generation
tags:
  - gguf
  - llama.cpp
  - openwebui
  - lithuanian
  - bilingual
  - local-ai
---

# 🧠 ZygAI — Neutral Bilingual AI Engine (LT / EN)

**ZygAI** is a neutral, locally hosted AI engine designed for **Lithuanian 🇱🇹 and English 🇬🇧** language tasks.  
It is built for **local inference**, **server-based usage**, and **runtime personas**.

> ZygAI is a **base engine**, not a chatbot persona.  
> Behavior and specialization are applied at runtime (OpenWebUI / API).

# ✨ Key Features
- 🇱🇹 / 🇬🇧 **True bilingual support**
- ⚡ Optimized **GGUF** models for `llama.cpp`
- 🧩 Supports **runtime personas** (MiniGPTs, system prompts)
- 🧠 Clean identity — **no vendor branding**
- 🖥️ Designed for **systemd + server deployments**
- 🔀 Supports **GGUF shards** (no merge required)

# 🧠 Architecture Overview
ZygAI (base engine)  
├── Q4 → fast / high throughput  
├── Q5 → balanced / general usage  
└── Q8 → high quality / reasoning

- **ZygAI** = neutral engine
- No hardcoded system prompt in the model

# 📦 Available Quantizations
| Quantization | Purpose | Notes |
|-------------|--------|------|
| **Q4_K_M** | Fast | Best speed, low memory |
| **Q5_K_M** | Balanced | Default general use |
| **Q8_0** | High quality | Best reasoning, higher RAM |

> Models may be provided as **GGUF shards** (`-00001-of-00002.gguf`).  
> `llama.cpp` loads shards automatically — **no merge required**.

# 🚀 Running ZygAI (llama.cpp server)

# Example: Q4 (shard-based)
```
./llama-server \
  -m ZygAI-q4_k_m-00001-of-00002.gguf \
  --host 0.0.0.0 \
  --port 8081 \
  --ctx-size 4096 \
  --threads 4 \
  --batch-size 2048 \
  --jinja
```
  
# Multiple models (recommended)
| Model | Port |
| --- | --- |
| Q4 | 8081 |
| Q5 | 8082 |
| Q8 | 8083 |



# 🧰 Using with OpenWebUI
*   Provider: **OpenAI (local)**
    
*   Base URL: `http://127.0.0.1:PORT/v1`
    
*   Auth: none
    
# Important
ZygAI is designed for **llama.cpp backend**.  
System prompts and personas work **correctly only with llama.cpp**, not Ollama.



# 🎭 Personas (Recommended)
ZygAI is intentionally **neutral**.  
Specialization is applied via runtime personas:



# 🌍 Language Behavior
*   Responds **in the same language as the user**
    
*   No automatic language switching
    
*   No mixed-language replies unless requested
```
Examples:
User (EN):
> What is Lithuania?
Assistant:
> Lithuania is a country located in the Baltic region of Eastern Europe.
User (LT):
> Kada Lietuva įstojo į Europos Sąjungą?.
Assistant:
> Lietuva įstojo į Europos Sąjungą 2004 m. gegužės 1 d.
```

# 📜 License
Apache 2.0  
This repository provides **inference-only model files**.  
Base model weights originate from publicly available sources and are redistributed according to their respective licenses.


# 🔒 Notes
*   ZygAI is **not** ChatGPT
    
*   ZygAI is **not** a vendor-branded assistant
    
*   ZygAI is designed for **local-first, privacy-respecting AI**

# 📖 Citation

If you use **ZygAI** in research, development, or documentation, please cite it as follows:
```
@software{zygai-7b,
  title        = {ZygAI: Neutral Bilingual AI Engine for Lithuanian and English},
  author       = {Mažeika, Žygimantas},
  year         = {2025},
  publisher    = {Hugging Face},
  url          = {https://huggingface.co/ZygAI},
  license      = {Apache-2.0},
  note         = {Local-first GGUF models optimized for llama.cpp with runtime personas}
}
```