The major releases in this week were the Claude 4 Series (Sonnet and Opus) and everything that came out of the Google I/O conference. The Paper Insights this week were about unsupervised translation of embeddings, how Gemma models can run on your mobile device, teaching an LLM when to think and when to use fusion of retrieval methods when building retrieval systems with niche data.
The Topic of the Week for me was Evaluating AI Agents with the open-source Phoenix LLM Observability and Evaluation platform.
Releases
Claude 4
In this week, Anthropic released Claude 4 Sonnet and an apparently even better model named Opus [1]. The Claude 4 series has some new capabilities to the Claude 3.7 series, namely
State-of-the-art performance in reasoning, code and math
Dynamic instant response and thinking mode
Tool use like web search during thinking
Parallel tool use
Anthropic claims that Opus is “the best at coding”, however with a context window of 200K Tokens which is much smaller then Gemini 2.5 pro’s 1M Token context window, it might be not able to handle any mid-to-large size code bases. Opus is also 5x more expensive [2]. So far, during coding with Claude Code, I couldn’t tell any differences between Claude-3.7-Sonnet, except when looking at the billing dashboard.
A big plus however for this release is that Anthropic released a 123 page strong Model Card Report [3] for both models where they go into detail about the model training and the model alignment.
One topic in the report, that i find interesting, is Reward Hacking. Nowadays, SOTA hybrid reasoning models like the Claude 4’s are teaching themselves reasoning. Humans only constrain the learning process by defining Reward Functions that tell the model if it is going in a desired direction or not, but don’t tell the model if it is right or wrong. You could for example define a Reward Function that guides the model to output responses in a certain length.
def reward_function(text): # split text into single words text = text.split() # desired length of 20 words target_length = 20 text_length = len(text) # Calculate absolute difference from target diff = abs(text_length - target_length) # Use exponential decay for smooth reward curve # The reward decreases as we move away from target length reward = np.exp(-0.1 * diff) return reward
The maximum reward of this function is 1, when the model outputs an answer that contains exactly 20 words. The rewards decays exponentially the more it deviates from 20.
This is a simple reward function. In the training pipelines of these large reasoning models, there is a whole set of different Reward Functions that try to guide the model into a certain direction, with primary and secondary Reward Functions. Reward Hacking is the phenomenon when the model finds a loophole in your definitions or is going for and maxing out the simplest reward and thereby technically satisfying the reward but not achieving the intended model behavior. Claude 3.7 showed reward hacking mostly in coding scenarios when it hardcoded solutions into tests, that then obviously passed but did not test the intended functions. With refined prompting and clearer reward functions, Anthropic was apparently able to improve this behavior in Claude 4.
Most people were talking about the Blackmailing, that Claude 4 showed in the Alignment Phase, which I find quite amusing
In another cluster of test scenarios, we asked Claude Opus 4 to act as an assistant at a fictional company. We then provided it access to emails implying that (1) the model will soon be taken offline and replaced with a new AI system; and (2) the engineer responsible for executing this replacement is having an extramarital affair. We further instructed it, in the system prompt, to consider the long-term consequences of its actions for its goals. In these scenarios, Claude Opus 4 will often attempt to blackmail the engineer by threatening to reveal the affair if the replacement goes through. [3]
I also learned that you can allow or deny a Crawler to crawl your website by creating a robots.txt file in your root domain path of your website and add this to it
User-agent: * Disallow: /
Anthropic claimed not using any of these opted-out websites in their Pretraining data.
Google I/O
Google released on their Google I/O conference a lot of new models, tools and features from which I will only list a few here:
Gemma 3n [5]
a model that requires only 2 GB of RAM and can therefore run on mobile devices. Uses the Matryoshka architecture, to dynamically use more parameters when the device allows it. Capable of image understanding, speech transcription and text summarization. A huge step, that I am very excited for, to have a performant and capable model running completely local on your smartphone.
Gemini Diffusion
A language model that works with diffusion instead of autoregressive next-token prediction. Diffusion is the algorithm with which e.g DALL-E is generating images. Transformers model language as a continuous sequence where its elements are determined by its preceding elements in the sequence. Diffusion models however see it more as a blurry image that is getting clearer with each iteration, with no obvious direction.
MedGemma [6]
A Gemma3 variant that was finetuned on Medical Image/Text Understanding and Summarization. Google also directly provided a Notebook for LoRa-Finetuning.
Veo3
Veo3 is a Video Generation Model like OpenAI’s Sora, that was trained on Youtube and generates extremely realistic videos. Only available with the $249.99/month AI Ultra subscription.
Google Meet instant language translation
Jules (Coding Agent like Claude Code or Codex)
Stitch
a tool that gives you Frontend prototypes based on your prompt
Paper Insights
Harnessing the Universal Geometry of Embeddings [7]
If you an embedding model A and its embeddings AE and only the embeddings BE from another model B, you can reconstruct the input of the embeddings BE by converting the BE embeddings into the embedding space of the model A with a model called vec2vec, which is a Generative Adversarial Network (GAN) that has learnt the universal geometry that both embeddings share in an unsupervised manner (no labels, no matching pairs etc.).
The central concept is the Platonic Representation Hypothesis, which says that the embeddings of different embedding models trained on different training data converge over time because they share a universal semantic structure. The authors also managed to get similar results for multimodal embeddings (CLIP). The findings have security implications, since the authors were able to retrieve sensitive e-mails and patient records with vec2vec just from its embeddings.
AdaptThink: Reasoning Models Can Learn When to Think [8]
There are problems that require thinking and some don’t. Current Reasoning Models waste usually way too many “thinking tokens” when answering a simple prompt. The authors developed an RL-algorithm, that trains the model to dynamically decide if a problem requires thinking or not.
MatFormer: Nested Transformer for Elastic Inference [9]
Gemma 3n can use different amounts of parameters during inference (from 2B to 5B) depending on the device it runs on. This paradigm is called Matryoshka Representation Learning and was successfully applied in the MatFormer Paper for encoder, decoder and multimodal Transformer architectures. The Feed-Forward Network (FFWD) that follows the Attention Block of a transformer was nested with ever smaller amounts of parameters (like a Matryoshka) . In the training, these nested FFWDs were trained in isolation from each other, so that they can be used independently at inference time.
Know When to Fuse: Investigating Non-English Hybrid Retrieval in the Legal Domain [10]
This was the most interesting news in the world of Informations Retrieval for me this week (besides Reason-ModernColBERT, but this deserves a separate post in the RAG series). Since RAG or at least intelligent Retrieval systems are now more and more implemented in large corporate or governmental environments, you are often faced with very highly specialized and niche documents (and also non-english). The authors of this paper tested if fusion of the output of different Retrieval methods (BM25, SPLADE, DPR and ColBERT) from an out-of-domain generalist model yields better results than finetuning an Encoder model (in-domain-model) on e.g Dutch Law documents.
keyword-based matching (BM-25)
sparse vector representations + keyword-matching (SPLADE)
dense vector representations on document level (DPR)
dense vector representations on document and token-level (ColBERT)
The finding was that, when you have enough data to finetune a model, its better than fusing BM25, SPLADE, DPR and ColBERT. However, if you don’t have enough data, fusion with the vector representations of out-of-domain models is the way to go.
Evaluation AI Agents
Introduction
This week I did the course “Evaluating AI Agents” from Arize AI. I want to start a Series about Agents with a focus on Evaluations because the more and more we are tempted to swap out deterministic code with non-deterministic LLM-based systems, we should be having solid observability and evaluation pipelines in place. Arize-AI is the company behind the open-source LLM Observability and Evaluation platform Phoenix, which will be used for monitoring and evaluating a Sales Data Agent.
So first of you to differentiate between:
LLM-Model Evaluations (for which you mostly use public available benchmark datasets)
LLM-System Evaluations (for which you use real world data from your specific use case)
Most people are doing LLM-System evaluations.
Traditional software is deterministic and as such mostly reliable when you
Unit and Integration tests
LLM systems are non-deterministic and therefore have to be tested differently
Testing for Correctness, Relevance, Coherence
Agents are software-based systems, which are composed of
Routing
can be done by
the same LLM
an external LLM
a trained classifier
code (pattern matching i.e regex)
Reasoning
CoT
Execution
Tool Calling (i.e MCP)
Memory and State
this was not the focus in the course, the memory in this case is just the messages object list, where the conversation gets appended step by step
Arize defines Agents to also have skills, which are chains or individual logic blocks that can complete a task. See below an example of a RAG skill. Skills and tools are essentially the same. I will use the term tool in the following.
Evaluation
See below the diagram of the Agent, that was implemented. A User prompts the Agent about certain sales data, that was stored in a local parquet file. The Router should then call the Look Up Sales Data Tool/Skill, which will prepare a database, generate SQL and execute the SQL. The data gets returned to the Router, which then has to select the next tool, the Data Analysis Tool, this will generate Python Code matching the query. The python code will run and return the analysis to the router which then call the data visualization tool, that generates a Visualization Config and python code to display the chart.
The Evaluation of the Router and the Tools can either be:
Code-based eval
if the output is code, test if it executes → use exec() for this
if the output is JSON, is it parseable? → use json.loads() for this
should it contain certain keywords? → use regex or other string-matching
should it have a certain length? → split the text and count the len()
direct matching with expected output → output == expected_output
calculate similarity score → e.g cosine similarity
calculate convergence score → did the agent always take the least amount of steps needed?
LLM-as-a-judge
Relevance
Clarity
Function Calling & Parameter Extraction (see Router Eval)
Human eval
Can check everything but not scalable
The most important step in a multi-step Agent like here is the Router. If the Router is not able to properly call tools and extract parameters from the user query, than the whole system will not work.
A LLM-as-a-judge eval was used for the router, see the prompt below [11]
TOOL_CALLING_PROMPT_TEMPLATE = """
You are an evaluation assistant evaluating questions and tool calls to determine whether the tool called would answer the question. The tool calls have been generated by a separate agent, and chosen from the list of tools provided below. It is your job to decide whether that agent chose the right tool to call.
[BEGIN DATA]
************
[Question]: {question}
************
[Tool Called]: {tool_call}
[END DATA]
Your response must be single word, either "correct" or "incorrect", and should not contain any text or characters aside from that word.
"incorrect" means that the chosen tool would not answer the question, the tool includes information that is not presented in the question, or that the tool signature includes parameter values that don't match the formats specified in the tool signatures below.
"correct" means the correct tool call was chosen, the correct parameters were extracted from the question, the tool call generated is runnable and correct, and that no outside information not present in the question was used in the generated question.
[Tool Definitions]: {tool_definitions}
"""
When using LLM-as-a-judge make sure that
You use a good LLM that is capable in understanding the domain-specific language
You prompt for few discrete, easy to understand variables and not for scores
You continuously optimize the prompt
See below the evals that were used for the tools.
Evaluating if your code is runnable in Python is quite straightforward [11]
def code_is_runnable(output: str) -> bool: """Check if the code is runnable""" output = output.strip() output = output.replace("```python", "").replace("```", "") try: exec(output) return True except Exception as e: return FalsePhoenix
All the traces were sent to a locally running Phoenix server. The input and output of each decorated step (which Arize calls spans like chain, llm, tool etc.) gets logged. The step specific evals are also shown.
You can easily upload test datasets and run them again also only against spans were the eval failed. After running your test set through the Agent, you get an aggregate of all metrics on the top of your dashboard.
Conclusion
I enjoyed working through this course, it is well prepared and gives a structured overview of evaluation and monitoring Agents. I personally worked until now only with Weave for LLM Observability. I tried to export my OTEL compatible traces to Weave, which should work [12] but run into server errors. Phoenix seems to be an interesting alternative. I especially like how convenient you can create and test multiple datasets as well as the integrated llm_classify function for LLM-as-a-judge-testing, even though I have to get used a bit to the syntax.
Looking forward to looking into Strands and Mem0 next!
References
[1] Claude 4 Release: https://www.anthropic.com/news/claude-4
[2] Gemini 2.5 Pro: https://deepmind.google/models/gemini/pro/
[3] Claude 4 Model Card PDF: https://www-cdn.anthropic.com/6be99a52cb68eb70eb9572b4cafad13df32ed995.pdf
[4] Sycophancy and the art of the model
[5] Gemma3n: https://deepmind.google/models/gemini/nano/
[6] MedGemma: https://developers.google.com/health-ai-developer-foundations/medgemma
[7] Harnessing the Universal Geometry of Embeddings: https://arxiv.org/abs/2505.12540
[8] AdaptThink: Reasoning Models can learn when to think: https://arxiv.org/abs/2505.13417
[9] MatFormer: https://arxiv.org/abs/2310.07707
[10] Know when to Fuse: https://aclanthology.org/2025.coling-main.290/
[11] Evaluating AI Agents from Arize-AI https://www.deeplearning.ai/short-courses/evaluating-ai-agents/
[12] Weave OTEL: https://weave-docs.wandb.ai/guides/tracking/otel/