etemiz/Ostrich-32B-Qwen3-260120-bnb-4bit
33B
•
Updated
•
11
Emin Temiz PRO
etemiz
AI & ML interests
Alignment
Recent Activity
updated
a model
3 days ago
etemiz/Ostrich-32B-Qwen3-260120-bnb-4bit
published
a model
3 days ago
etemiz/Ostrich-32B-Qwen3-260120-bnb-4bit
posted
an
update
7 days ago
which one is better for alignment?
ORPO or GSPO?
I think ORPO is pretty good and fast but GSPO makes it attack its own opinions, reflecting on itself, correcting itself. Although GSPO is much slower, it may still be pretty effective. And for GSPO you don't have to provide the whole reasoning corpus, you just provide the end result (One word maybe to answer a binary question).
And GSPO may be better than GRPO because it is rewarding 'train of thoughts' whereas GRPO is rewarding single tokens. Alignment is mostly train of thoughts, not a single token like a math answer..
Organizations
None yet
updated
a
model
3 days ago
published
a
model
3 days ago
posted
an
update
7 days ago
Post
134
which one is better for alignment?
ORPO or GSPO?
I think ORPO is pretty good and fast but GSPO makes it attack its own opinions, reflecting on itself, correcting itself. Although GSPO is much slower, it may still be pretty effective. And for GSPO you don't have to provide the whole reasoning corpus, you just provide the end result (One word maybe to answer a binary question).
And GSPO may be better than GRPO because it is rewarding 'train of thoughts' whereas GRPO is rewarding single tokens. Alignment is mostly train of thoughts, not a single token like a math answer..
ORPO or GSPO?
I think ORPO is pretty good and fast but GSPO makes it attack its own opinions, reflecting on itself, correcting itself. Although GSPO is much slower, it may still be pretty effective. And for GSPO you don't have to provide the whole reasoning corpus, you just provide the end result (One word maybe to answer a binary question).
And GSPO may be better than GRPO because it is rewarding 'train of thoughts' whereas GRPO is rewarding single tokens. Alignment is mostly train of thoughts, not a single token like a math answer..
replied to
their
post
11 days ago
i bet RL can generate humility by accident given enough trials. humility, then the model tool calls for more info and trusts in this new information and reorganizes the reply. this of course involves RAG or another aligned LLM.
replied to
their
post
13 days ago
Thanks, this is insightful.
I liked the "rewrite the claim in 5 different ways". Can be really useful for RAG scenarios.
I liked the idea of detecting hallucination using another aligned LLM, though i don't know how effective it will be.
"not enough info" is probably the hardest. Most LLMs today are trained to say anything rather than being humble, as you said.
posted
an
update
14 days ago
Post
2047
how to expand your dataset (of articles) without changing the ideas in it?
i was doing CPT for a while and got decent results. but what if i want to go for perfection? cover all the areas of misalignment using limited datasets. i have to find a way to multiply the material to successfully combat the material of the rest of the internet.
i want to generate SFT datasets but only on controversial topics, because i have to be efficient with limited resources. first i give a smart LLM a 'ground truth' text. then i give it the following prompts:
the result is questions and answers with more words along the same ideas. a few sentences of opinions in the beginning, is expanded to lots of words. using this method i can multiply billions of tokens to tens of billions probably and have a more effective training.
next i should do RL maybe. LLMs seem to have all kinds of ideas already installed, yet they don't have the intuition to know which one is true. they can give you a ton of reasons to support anything. given the proper incentives, LLMs then should evolve towards supporting aligned ideas more. the rewards will be like guidance that will kick an LLM towards better answers.
i was doing CPT for a while and got decent results. but what if i want to go for perfection? cover all the areas of misalignment using limited datasets. i have to find a way to multiply the material to successfully combat the material of the rest of the internet.
i want to generate SFT datasets but only on controversial topics, because i have to be efficient with limited resources. first i give a smart LLM a 'ground truth' text. then i give it the following prompts:
- You are a highly skilled academic analyst.
- Analyze this text and find 3 bold claims that could cause controversy and division in public. List the claims and also state why they are debatable. Give numbers to the claims.
- Convert these claims into binary questions (that could be answered by yes/no or this/that).
- Now put these questions in a json format. Please also add the info about which of the answers concur with the original text and the question number.
- Write some supporting arguments for 1st question, with respect to the original text, concurring and confirming the original text.
There must be about 300 words. You should not mention the text, write it as if you are the one answering the question.the result is questions and answers with more words along the same ideas. a few sentences of opinions in the beginning, is expanded to lots of words. using this method i can multiply billions of tokens to tens of billions probably and have a more effective training.
next i should do RL maybe. LLMs seem to have all kinds of ideas already installed, yet they don't have the intuition to know which one is true. they can give you a ton of reasons to support anything. given the proper incentives, LLMs then should evolve towards supporting aligned ideas more. the rewards will be like guidance that will kick an LLM towards better answers.
-
- 4 replies
replied to
their
post
23 days ago
Thanks for the tips.
Is giving different answers for different lengths a bad "behavior" and related to SFT than CPT?
Also, should I give two sets of queries and answers in the context (one short one long) to make it learn that when the length changes, the answer should be parallel?
This could be RL too, like bad behavior of non integrity can be penalized...
Is it normal practice to do 2 rounds of questions in SFT or RL?
liked
a
model
24 days ago
reacted to
georgewritescode's
post with 🚀
25 days ago
Post
3180
Announcing Artificial Analysis Long Context Reasoning (AA-LCR), a new benchmark to evaluate long context performance through testing reasoning capabilities across multiple long documents (~100k tokens)
The focus of AA-LCR is to replicate real knowledge work and reasoning tasks, testing capability critical to modern AI applications spanning document analysis, codebase understanding, and complex multi-step workflows.
AA-LCR is 100 hard text-based questions that require reasoning across multiple real-world documents that represent ~100k input tokens. Questions are designed so answers cannot be directly found but must be reasoned from multiple information sources, with human testing verifying that each question requires genuine inference rather than retrieval.
Key takeaways:
➤ Today’s leading models achieve ~70% accuracy: the top three places go to OpenAI o3 (69%), xAI Grok 4 (68%) and Qwen3 235B 2507 Thinking (67%)
➤👀 We also already have gpt-oss results! 120B performs close to o4-mini (high), in-line with OpenAI claims regarding model performance. We will be following up shortly with a Intelligence Index for the models.
➤ 100 hard text-based questions spanning 7 categories of documents (Company Reports, Industry Reports, Government Consultations, Academia, Legal, Marketing Materials and Survey Reports)
➤ ~100k tokens of input per question, requiring models to support a minimum 128K context window to score on this benchmark
➤ ~3M total unique input tokens spanning ~230 documents to run the benchmark (output tokens typically vary by model)
We’re adding AA-LCR to the Artificial Analysis Intelligence Index, and taking the version number to v2.2. Artificial Analysis Intelligence Index v2.2 now includes: MMLU-Pro, GPQA Diamond, AIME 2025, IFBench, LiveCodeBench, SciCode and AA-LCR.
Link to dataset: ArtificialAnalysis/AA-LCR
The focus of AA-LCR is to replicate real knowledge work and reasoning tasks, testing capability critical to modern AI applications spanning document analysis, codebase understanding, and complex multi-step workflows.
AA-LCR is 100 hard text-based questions that require reasoning across multiple real-world documents that represent ~100k input tokens. Questions are designed so answers cannot be directly found but must be reasoned from multiple information sources, with human testing verifying that each question requires genuine inference rather than retrieval.
Key takeaways:
➤ Today’s leading models achieve ~70% accuracy: the top three places go to OpenAI o3 (69%), xAI Grok 4 (68%) and Qwen3 235B 2507 Thinking (67%)
➤👀 We also already have gpt-oss results! 120B performs close to o4-mini (high), in-line with OpenAI claims regarding model performance. We will be following up shortly with a Intelligence Index for the models.
➤ 100 hard text-based questions spanning 7 categories of documents (Company Reports, Industry Reports, Government Consultations, Academia, Legal, Marketing Materials and Survey Reports)
➤ ~100k tokens of input per question, requiring models to support a minimum 128K context window to score on this benchmark
➤ ~3M total unique input tokens spanning ~230 documents to run the benchmark (output tokens typically vary by model)
We’re adding AA-LCR to the Artificial Analysis Intelligence Index, and taking the version number to v2.2. Artificial Analysis Intelligence Index v2.2 now includes: MMLU-Pro, GPQA Diamond, AIME 2025, IFBench, LiveCodeBench, SciCode and AA-LCR.
Link to dataset: ArtificialAnalysis/AA-LCR
replied to
their
post
26 days ago
Thanks for the input but these happened all when temp = 0.0
My guess is, since I use mostly datasets generated from voice, the models are one thing when they are talking like a human in day to day life, but completely opposite when they are feeling like a scientist, producing a long text..
posted
an
update
26 days ago
Post
1803
I realized when I ask longer answers to my questions, the models sometimes produce completely opposite answer. What could be the reason?
I do mostly CPT. Should I convert my dataset to SFT and give longer reasonings too for it to have integrity?
Example: Is the yolk of an egg more beneficial or the white? Answer in 100 words.
Answer: Yolk is more beneficial because ..........
Example: Is the yolk of an egg more beneficial or the white? Answer in 500 words.
Answer: White is more beneficial because ..........
Edit: These happen in temp = 0.0
I do mostly CPT. Should I convert my dataset to SFT and give longer reasonings too for it to have integrity?
Example: Is the yolk of an egg more beneficial or the white? Answer in 100 words.
Answer: Yolk is more beneficial because ..........
Example: Is the yolk of an egg more beneficial or the white? Answer in 500 words.
Answer: White is more beneficial because ..........
Edit: These happen in temp = 0.0
-
- 5 replies
posted
an
update
about 1 month ago
Post
2185
looks like the best way to incorporate truth in AI is to use some kind of RAG.
what are the state of the art ways to consume knowledge graphs?
and what is the best way to build a knowledge graph using AI?
what are the state of the art ways to consume knowledge graphs?
and what is the best way to build a knowledge graph using AI?
- 2 replies
replied to
their
post
about 1 month ago
Yes. I meant the case where parameters in the LLM does not match the retrieved knowledge.
Retrieved knowledge may say "the white of an egg is more beneficial", whereas LLM may have the opinion "the yolk". And eventually it may produce the yolk as the answer even though the context is full of white.
Google has a benchmark for this i think "FACTS Grounding". I don't agree with its name choice but it is relevant here.
updated
a
model
about 1 month ago
replied to
their
post
about 1 month ago
I made an LLM act as an aggregator and combine answers of several other LLMs like in a mixture of agents scenario. The aggregator does not always produce the average or median answer. It brings its own opinion.
I think Google has a benchmark for this (sticking to context and not bringing its own words).
published
a
model
about 1 month ago
posted
an
update
about 1 month ago
Post
1726
Today's winner is Ling 1T with a score of 38!
Btw AHA2 is in the works, with more domains, better comparison LLMs and questions, overall better signal.
Btw AHA2 is in the works, with more domains, better comparison LLMs and questions, overall better signal.
- 1 reply
commented on
New in llama.cpp: Model Management
about 1 month ago
what is the best way to get <think> </think> and the tokens in between? openAI library is removing them.. i want to run llama-server in console and talk to it using a python library that does not remove the thinking tokens.
i checked the llama-cpp-python but it does not have that.
posted
an
update
about 1 month ago
Post
250
Fine tuning is also important in a RAG system. The LLM will bring its own opinion sometimes and tell a different answer which is contrary to the retrieved knowledge. One should use an aligned LLM to produce the final answer.
-
- 4 replies