Rashmi explains that because multi-agent systems are stochastic and interacting, observability must extend to replaying agent actions, tool invocations, reasoning chains, and cross-system latency. ✦ AI generated
Rashmi Shetty · The TWIML AI Podcast · 2026-04-16 · original ↗
starts at this moment · 22:52
“It what are the unique observability challenges that you've seen with multi-agentic workflows?”
All the more important for us to be able to replay agentic actions and try to understand how it function. Agent behavior needs observability along many different dimensions in terms of what are the tools invoked? How what was the reasoning mechanism that led to that tool invocation? And overall, what was its context that passed across systems?
verbatim transcript · starts at 22:52
22:52replay agentic actions and try to understand how it function. So, as I mentioned, there are several different layers that you observe, several different junctures that you observe. Agent behavior needs observability along many different dimensions in terms of what are the tools invoked? How what was the reasoning mechanism that led to that tool invocation? Um and overall, what was its context that passed across systems? Is there any
23:23potential for monitoring for latency optimization across these layers? So, observability across agentic systems, there is a standardization for sure in terms of observing agentic behavior, but there is also a standardization across the system to observe an end-to-end uh latency profile of how the different layers and systems are functioning together. So, latency today is really a cross-functional effort, right? It it it has to be observed from an end-to-end mechanism. It it cannot be
23:58a siloed approach today where just the models are evaluated for for um latency. So, from an agentic standpoint, it's it's very much like any other system except for that agentic systems have specific nuances that needs to be observed, as I mentioned, the behavior, whether it's it's hitting its goal or not, whether it is um invoking the right tool or not, what was the reasoning process that resulted in that tool invocation.
24:29And did you find that the combination of your existing observability tooling and infrastructure plus, you know, traces, you know, which are, you know, popular for with you know, observing LLMs, like did those two things provide you what you needed, or did you need to uh you know, build new to support these types of agentic workflows? Yeah, as I mentioned, uh I think one of the key strengths of Capital One has