Welcome to the first version of my newsletter, where I talk about everything that I found interesting and/or should be mentioned that happened in the past week or what i found in the past week. I am still figuring out which structure i like the most but this is my first try. I will start with a short list of Releases, Paper Insights and Other News and then will go into a longer text, the topic of the week. I hope you learn something and subscribe if you like to receive this newsletter (hopefully weekly)
Releases
Codex-CLI
OpenAI released an Agentic Coding Tool that is like a hybrid of Devin and Claude Code, runs in your Terminal and can access your filesystem, does version control and is free during the Preview.
Intellect-2 32B [1]
Prime Intellect released Intellect-2, a 32B Reasoning Model like DeepSeek-R1, where the training was completely decentralized by crowdsourcing and open to track. The model was trained by asynchronous Reinforcement Learning and allows you to specify a thinking budget, i.e how many tokens the model is allowed to think before answering.
FedRAG [2]
Another example of the emerging trend of decentralized training is FedRAG, a framework that allows you to finetune your RAG application on a centralized or federated architecture.
Dynamic Byte Latent Transformer Weights [3]
This Transformer model uses raw data bytes instead of tokens. This makes the model inherently better at e.g handling different languages. The model dynamically groups bytes into patches when they are easy to predict (high propability) thereby reducing computing power and using full computer power only for difficult bytes. Interestingly, the Byte Latent Transformer scales better than token-based models.
BLIP3-o [4]
BLIP3-o is a family of AI models from Salesforce designed to unify both image understanding and generation capabilities in a single architecture. The models combine the reasoning strengths of autoregressive models (Qwen2.5-VL-Instruct) and semantically rich CLIP image features with the generative power of diffusion models in one semantic space.
Paper insights
Reasoning Models don’t always say what they think [5]
Reasoning models did not mention using hints in their chain-of-thought (CoT) when experimenting with a plain multiple-choice questions and the same questions with hints, indicating that CoT is not reliable as a source for AI monitoring and safety.
LLMs get lost in multi-turn conversations [6]
LLMs hallucinate less when provided with all information at once and prompted to answer right away then receiving them in multiple turns and at the end, indicating challenges for multi-agent systems that rely on context via multiple turns.
s1: Simple test-time scaling [7]
Researchers finetuned a Qwen-2.5-32B base model on only 1000 examples of CoT data. The model achieved accuracies rivaling that of top reasoning models by a simple trick, namely appending a “wait” token when the model was about to finish thinking, leading to a much longer CoT.
Other
Thorium Reactors was successfully refueled. [8]
Chinese scientists were able to refuel a Thorium reactor while it was online, which previous countries weren’t able to do, proving the viability of this nuclear technology. I never heard of Thorium Reactors before but find it quite interesting. Thorium is a more abundant element than Uranium, more energy efficient and is safer to get rid of.
From Large Generalists to Small Specialists
I want to talk about a paradigm shift in how models are trained and used relative to the years before. The paradigm shift is the movement from few large isolated generalist models to many small connected specialized models (agents). In the following, I will reference this weeks publications mentioned above at the end. But first of, I want to give a short recap of the last 3 years.
The rise of isolated large generalist models stemmed directly from the validation of the Scaling Hypothesis, which formed the foundation of OpenAI's remarkable success. This hypothesis proposed that increasing model size while applying more data and computational resources during training would yield better performance. This approach defined the Age of Pretraining, producing impressive results with models like GPT-2 and GPT-3.
However, this scaling strategy reached its end when training datasets expanded to hold the entire internet (approximately 15 trillion tokens). While there is still room for improvements in data quality—see the FineWeb dataset —and extending context windows, Pretraining has reached its peak because of the fundamental limitation of available high-quality data (which Ilya Sutskever termed thereby the "Fossil Fuel of AI").
As traditional scaling alone approaches lead to diminishing returns, three promising directions emerged:
Post-training (SFT, RLHF, Tool Calling)
Synthetic data generation
Inference-time scaling (Reasoning)
Post-Training: Base Models to Specialized Models
Pretraining gives you a Base Model, that has learned the semantics and relations/interconnections in sequences (Text, Image, Audio). Post-Training ist what follows Pretraining. Post-training typically consists of a multi-stage workflow:
Supervised Fine-Tuning (SFT) leverages labeled prompt-response pairs to shape model behavior toward specific capabilities (coding, reasoning, OCR, etc.).
Reward Modelling (RM), where responses of the model are sampled, ranked by Humans for preference and a model (the Reward Model) is trained to predict the reward of a certain response.
Reinforcement Learning by Human Feedback (RLHF) optimizes this reward signal by updating the model weights via reinforcement learning.
This workflow proved effective but high-quality labelling is slow, expensive and injects human biases and intuitions in the models.
The Evolution of Reasoning in Language Models
After compute and label intensive post-training optimizations, the resulting models still showed significant deficiencies in complex reasoning tasks. Researchers discovered that response quality substantially improved when models were explicitly encouraged to work through problems step-by-step before providing final answers.
Chain-of-Thought Approaches
This intermediate reasoning process, termed Chain-of-Thought (CoT), was initially implemented through specialized prompting techniques:
Prompt Engineering: Early CoT implementations relied on carefully crafted prompts that explicitly instructed models to "think step by step" before answering.
ReAct Framework: This approach combined reasoning and action by prompting models to alternate between thought traces and specific actions, creating more structured reasoning paths.
However, these early reasoning approaches suffered from significant limitations:
Brittleness: The quality of reasoning was highly dependent on prompt formulation and often broke down with slight variations in input.
Training Misalignment: Models were still primarily trained on human-intuition-based data from SFT, without explicitly optimizing for robust reasoning capabilities.
Inconsistency: The same model would produce drastically different reasoning paths for similar problems, revealing the superficial nature of the induced reasoning.
This prompted the development of dedicated reasoning models specifically trained to produce higher-quality intermediate thought processes, rather than relying solely on prompting tricks applied to models optimized for other objectives.
The Rise of Advanced Reasoning Models
Evolution of Extended Chain-of-Thought
OpenAI's "o-models" represented a significant advancement by generating substantially longer chains of thought, achieving state-of-the-art performance across reasoning tasks. However, their exact training methodology remained largely undisclosed, creating a knowledge gap in the field.
DeepSeek's Alternative Approach
DeepSeek brought transparency to reasoning model training through their innovative research:
DeepSeek-R0: This model pioneered a new approach by bypassing human feedback entirely. Built on DeepSeek-V3, it was trained exclusively through reinforcement learning without relying on human-labeled data for the critic model.
GRPO Framework: Instead of the algorithm (called PPO) with human-trained models, DeepSeek implemented Generalized Reward Policy Optimization (GRPO), which leveraged heuristics and result averaging to determine rewards. Hence no human labels involved.
Reinforcement Learning with Verifiable Rewards (RLVR)
The breakthrough came with DeepSeek-R1, which demonstrated the superiority of training models on objectively verifiable outcomes:
Objective Verification: For tasks like coding or mathematics, the model's output could be directly verified through compilation or computational validation.
Extended Reasoning: RLVR encourages longer, more thorough chains of thought by rewarding correct final results derived through sound reasoning processes.
Domain Specificity: This approach works particularly well in domains with clear verification criteria, such as code execution or mathematical proof checking.
However, some researchers argue that RLVR will suffer the fate of Pretraining and lead to diminishing returns [9]. This suggests that while RLVR represents a significant advancement, it may only be a part in the evolution of reasoning model development.
Specialized AI Model Portfolios
Nowadays, the provided model portfolio of the large providers like OpenAI, Anthropic and Google gets more and more numerous and specialized compared to early days 2022/2023. There are multiple reasoning models (o-models), generalist models, specialized models for writing or ideas or coding. I think this trend of model diversification will continue.
The Rise of Decentralized AI Ecosystems: MCP and A2A Protocols
So to summarize:
A decently trained small Base Model, that is finetuned on a small CoT dataset and reinforced for long CoT with verifiable rewards with GRPO competes or outperforms larger Generalist models on many tasks, including obvious savings of money and time in training and much less hardware requirements for hosting and inference.
At the end of 2024, Anthropic released the Model Context Protocol (MCP). LLMs which are finetuned to this protocol are able to generate API requests to call Tools that provide the LLM with external context the model needs to answer a prompt. Recently, Google released a protocol called A2A, that allows LLMs with their tools (together called an Agent) to communicate with other Agents. These protocols enable a new paradigm where specialized AI agents can collaborate to solve complex problems. MCP provides standardized, secure context for individual agents, while A2A enables seamless communication and collaboration between agents. This architecture supports sophisticated workflows where agents can delegate tasks based on specialized capabilities.
I personally like the agentic definition, that i saw from Nvidia
Agents are a system with complex reasoning capabilities, memory, and the means to execute tasks.
With the advancements in reasoning models, the inventions of the aforementioned protocols and recent developments in Agentic Memory (see mem0), we are about to see the broad application of Agentic systems that are not using large generalist models but will be based more on smaller models that are able to reason and are capable of communicating with external Tools (MCP) and with each other (A2A). Coming back to this weeks paper selection, training a rather small, but competitive reasoning model seems not that difficult [7] and much less training data intensive.
However, there are still some issues like mentioned in “LLMs get lost in Multi-Turn Conversations” [5]. Researchers from Microsoft proved that LLMs are not really able to keep coherence in longer multi-turn conversations, posing a big challenge for successful multi-agent systems. LLMs work better when they are carefully provided with all instructions they need in a prompt and have to respond right away instead of receiving specifications one turn at a time. Reasons for this is that they are filling the missing specifications themselves with incorrect assumptions and attempt to answer the question right away. LLMs generally tend to give the most attention to the first and last turn, which is commonly known as primacy and recency bias in LLMs. Also the inspection of why an Agent decided what to do by looking into the CoT seems to be not as reliable as previously thought [5].
References
[1] Intellect-2 https://arxiv.org/abs/2505.05410
[2] FedRAG https://github.com/VectorInstitute/fed-rag
[3] Dynamic Byte Latent Transformer https://github.com/facebookresearch/blt
[4] BLIP3-o https://www.arxiv.org/abs/2505.09568
[5] Reasoning Models don’t always say what they think https://arxiv.org/abs/2505.05410
[6] LLMs get lost in multi-turn conversations https://arxiv.org/abs/2505.06120
[7] s1: simple test-scaling https://arxiv.org/abs/2501.19393
[8] Thorium Reactors: https://www.neimagazine.com/news/china-refuels-thorium-reactor-without-shutdown/?cf-view
[9] RLVR will saturate; Inteconnects.ai Sycophancy and the Art of the model