Post
566
i love reinforcement learning
🤏 quecto mode
appvoid
appvoid
AI & ML interests
Working on small sota models
Recent Activity
liked a model 36 minutes ago
bench-labs/PixelModel-v5 updated a model about 22 hours ago
CEAMFA/palmer-007-preview liked a dataset about 24 hours ago
unstonio/pixelgpt-24x24-20kOrganizations
Posts 30
Post
1394
A Small Model is All You Need. Meet
After 3 years of experiments, we are finally releasing our flagship tiny model: **palmer-006**.
If you are building for edge hardware, SBCs (Raspberry Pi, etc.), or low-power devices, this is for you. Inspired by Andrej Karpathy's idea of a self-contained "cognitive core," we wanted to see how much power we could pack into a sub-100M parameter footprint.
🧠 **How we "Palmerized" it:**
We believe in starting our experiments with the absolute strongest baseline possible.
1. Light fine-tuning on highly curated data
2. Model merging
3. Another light fine-tuning round
4. Adjusted Mamba for maximum token speed ⚡️
⚠️ *Note: This is a foundational language model. It has not been instruction-tuned yet!*
Also, since this needs instruction tuning next to become a chat assistant—**what dataset would you recommend we use for the instruct tune?**
---
🔗 **Quick Links & Info:**
* **License:** Open for research, education, hobby, and modification! (For commercial use/hosted APIs, shoot an email to nosoyhackercodigo@gmail.com. *PS: Donators can claim a free commercial license!*)
* **Attribution:** Built using AI tech from the Technology Innovation Institute (TII).
Can't wait to see what you build at the edge. Let me know your prompt completions below! 👇
appvoid/palmer-006
palmer-006 (90M)After 3 years of experiments, we are finally releasing our flagship tiny model: **palmer-006**.
If you are building for edge hardware, SBCs (Raspberry Pi, etc.), or low-power devices, this is for you. Inspired by Andrej Karpathy's idea of a self-contained "cognitive core," we wanted to see how much power we could pack into a sub-100M parameter footprint.
🧠 **How we "Palmerized" it:**
We believe in starting our experiments with the absolute strongest baseline possible.
1. Light fine-tuning on highly curated data
2. Model merging
3. Another light fine-tuning round
4. Adjusted Mamba for maximum token speed ⚡️
⚠️ *Note: This is a foundational language model. It has not been instruction-tuned yet!*
Also, since this needs instruction tuning next to become a chat assistant—**what dataset would you recommend we use for the instruct tune?**
---
🔗 **Quick Links & Info:**
* **License:** Open for research, education, hobby, and modification! (For commercial use/hosted APIs, shoot an email to nosoyhackercodigo@gmail.com. *PS: Donators can claim a free commercial license!*)
* **Attribution:** Built using AI tech from the Technology Innovation Institute (TII).
Can't wait to see what you build at the edge. Let me know your prompt completions below! 👇
appvoid/palmer-006
models 359
appvoid/palmer-006
Text Generation • 91.1M • Updated • 102 • 8
appvoid/a-cool-model
Text Generation • 0.4B • Updated • 218 • 2
appvoid/cloud-11
Text Generation • 0.4B • Updated • 401
appvoid/cloud-10
Text Generation • 0.4B • Updated • 396
appvoid/cloud-09
Text Generation • 0.4B • Updated • 396
appvoid/cloud-08
Text Generation • 0.4B • Updated • 409
appvoid/cloud-06
Text Generation • 0.4B • Updated • 407
appvoid/cloud-05
Text Generation • 0.4B • Updated • 416
appvoid/appvoid-cloud-04-q8_0-GGUF
0.4B • Updated • 109
appvoid/cloud-04
Text Generation • 0.4B • Updated • 408
datasets 21
appvoid/rewrite-dpo
Viewer • Updated • 5k • 38
appvoid/mark
Viewer • Updated • 595 • 14
appvoid/raw-corpus
Viewer • Updated • 1.6M • 46 • 1
appvoid/tweety
Viewer • Updated • 3.91M • 14
appvoid/no-prompt-oasst-mini
Viewer • Updated • 100 • 25
appvoid/text-niro-v1
Viewer • Updated • 1.06k • 14
appvoid/niro-v1
Viewer • Updated • 1.06k • 34
appvoid/reasoning
Viewer • Updated • 19.9k • 43
appvoid/reflection-v1
Viewer • Updated • 389 • 13
appvoid/no-prompt
Viewer • Updated • 1M • 78