Deep dive - How to Debug & Optimize¶
Here we will cover how we can apply Langfuse insights
Now that you have your observability layer running, it’s time to use it. This guide takes Langfuse from theory into practice.
(Note: This guide assumes you have already completed the Setup & Connection steps and your Drupal environment is actively sending traces to Langfuse.)
https://www.youtube.com/watch?v=EU3XTuRKZC8
Below, we’ll walk through two real-world scenarios: optimizing a slow AI Search Block, and fixing a "hallucinating" AI Agent by tracking its tool calls.
Optimizing the AI Search Block (RAG)¶
The ai_search_block module is a fantastic way to let users chat with your Drupal content. But out of the box, you might heavily rely on defaults, use stock or bad system prompt, have too many chunks / documents in the context and just rely on a random LLM model that you think should work well. However, often, this results in slow replies and sometimes those aren’t even the ones you’d expect.
The Scenario & Investigation
Let's say we set up our AI Search Block on a Dev environment. We ask it a question: "Show me a cheese-cake recipe?" (we’re using Umami + some stock recipes). It gives a great answer, but it takes 8 seconds.
Instead of guessing why it's slow, we open the Langfuse dashboard and look at the trace for this specific query.
Looking under the hood in Langfuse, we spot the problem: our vector store is returning the top 20 documents. The LLM is forced to read through a massive wall of text to find the answer, which slows down the response and spikes our token usage.
Also, we default to Gemini 2.5 Pro - which costs quite a lot, per response.
And finally, it seems it ignored the language of the recipe - and felt back to the English (request language), which is exactly how we wanted this. We can see the raw documents in Langfuse.
The Optimization (Fixing it in Drupal)
We head back to our Drupal AI Search Block configuration and make two strategic tweaks:
- Reduce the Context: We drop the
RAG final max resultssetting from 20 down to 10. This filters out the noise and only feeds the AI the most relevant chunks. - Switch Models: Because the context is now highly relevant and concise, we don't need a heavy, slow model. We switch the LLM provider to a high-speed model like Gemini Flash or Claude Haiku.
- Adjust the prompt, to stick to requested language, and hint to AI that we allow it to translate the recipes into the requested language, if needed.
The Verification
We go back to our Drupal site and re-run the exact same query. Then, we check the new trace in Langfuse.
Okay so the replies still look good, sticking to English, let’s analyze the traces now:
We dropped from 22 seconds to 5.3 seconds, AND dropped the pricing from ~0.15 → ~0.015 (~10x), per question (not only by switching the model but also by dramatically lowering the input tokens).
This was a simple demonstration, but in reality you can do the following:
- Experiment with chunks - sometimes smaller chunks can yield better results, sometimes bigger chunks yield better results.
- Experiment with the temperature, threshold, number of documents, prompts, etc.
- Experiment with reranking (there are different reranking methods).
- Enable different Vector Databases indexes - and see how they accelerate the responses (without degrading the quality).
- Experiment with other embedding models.
- Experiment with caching layers.
The Takeaway
By using Langfuse to actually see the context payload, we successfully maintained the exact same "smartness" and quality of the reply, but drastically increased the speed and lowered the cost per query.
Chapter 2: Unpacking AI Agents and Tool Calling¶
AI Agents are more autonomous than simple search blocks. They can use tools, delegate tasks to other agents, and make decisions on the fly. But when an Agent goes rogue, the "black box" makes it very hard to debug. Drupal offers Agent Explorer - which is a nifty tool explaining what’s happening under the hood (sequentially) - but when Agents delegate to other Agents that then turn around and use Tools, things can become messy and hard to untangle.
The Scenario
We have an AI Recipe Agent (yes, we’re still re-using the Umami theme), and we still ask the same question: “Show me a cheese-cake recipe?”
It seems to be working, we got a recipe, but…
The Investigation
We can clearly see that the agent replied without using any tools - just from its training. Why didn't it search the database? The trace reveals the smoking gun: The call to the Vector Store tool never happened. The LLM simply received the prompt and immediately generated a response.
The Fix in Drupal
We go back to our Agent configuration in Drupal and check our settings. We spot the issue immediately:
❌ Missing Tools: We forgot to add our RAG / Vector Search tool. Let’s add the tool to the agent.
With the tool in-place, let’s rerun the question in the AI Agent Explorer, to see what we get.
The Verification
We ask the Agent the exact same question. This time, it gives the correct answer based on our actual policy. We verify this by looking at the new trace in Langfuse.
We can now clearly see how the tool-call was made, and how the interactions trickle down.
The Takeaway
When dealing with agentic interactions, delegations or tool-calls, observability tools like Langfuse are non-negotiable. They allow you to instantly see if a tool failed, if permissions were blocked, or if the Agent simply decided to skip a step - turning hours of frustrating debugging into a 30-second fix.