Agent Swarm Observability & Mantiq POC

Okay.

Good evening.

So, I've been thinking a little bit

about like agent swarms and what it

means for well, observability. I was

kind of trying to think a little bit

ahead like, I don't know, 6 months into

the future or a year into the future

from

for Kitaru and

yeah, what does that mean? What does

that mean for Kitaru?

And

so yeah, I've been doing a bit of

exploring and I kind of just wanted to

share I don't know, some initial things

that I was was working on. Maybe it I

don't know, provoke something in you or

you can tell me this bit is interesting

or this bit is not interesting. Um

And maybe the best place to start is

just well,

so at the moment people have agents and

you start to see a little bit the labs

people in the labs, I mean mainly those

are the ones that can afford to run

these things, but starting to run groups

of agents together.

Now previously we would call this like a

multi-agent system. Maybe I don't know,

for the last 2 years this is what people

talked about.

And you know, agents coordinating

together. Even our own Jai

built a kind of communication protocol

for agents to work together.

And

but now what you start to see is agents

collaborating together in a multi-agent

system. But the number of these is is

much larger. I think the largest one

that is kind of well-known to have been

have done something productive at least

was the one that that solved the

Navier-Stokes

um

Millennium Prize mathematics problem and

that was something like 10,000

agents working together in a swarm.

And obviously at that at that scale

it's going to be

I mean already people talk about, you

know, the the single agent stuff being

too many traces to follow or whatever.

Uh but if you scale up to 10,000 agents

working over a multi-day period or

something, it's going to be outside the

realm of human possibility to

understand.

So,

so what does that mean? Are we just

going to delegate understanding of these

things to agents? How are we tracking

this? Um do these swarms need tracking

in a different way? These are all kind

of questions which I was thinking about

this week. Um or allowing myself uh time

to think about it. Um and if you'll

excuse me just referring to something I

wrote in my blog, um I kind of gathered

together some questions that people had

written about in blogs um about, you

know, after they'd run agent swarms. Um

uh

yeah, you know, why was the whole system

waiting for its slowest worker?

Uh why did the session fail before the

um agent could do meaningful work? Why

did this worker receive the task? Um

I mean, these these are just the ones

which were specified, but you can

imagine there's there's a whole range of

things that can go wrong within a swarm.

Um and yeah, you might have questions or

you might just want a little bit like we

do with insights in Ketero, you might

want to have some kind of insights being

coming out of these these swarms to

understand them better.

Uh because the scale is pretty huge.

Um and you can kind of map these

questions onto the evidence that's

needed. Um

uh some of them are kind of traditional

stuff of like just the model context,

the messages, tool calls, that kind of

stuff.

But then you also need to go a lower

level down to see like what was

happening on the file system level of um

uh

of each individual agent. And then, you

know, what how were messages being

passed uh between each other? Um

uh among uh within the swarm um and how

did decisions get made? Are there kind

of

uh patterns which patterns of

communication or file writing or so on

or like where were problems happening.

So, you do need quite a bit of evidence.

Um,

and we've also seen

Yeah, somewhere in my blog I kind of

list, you know, the different kinds of

things from model context to files. You

need to be capturing like reads and

writes of files. Um,

memory when it's used.

Data integrity.

All sorts of things. You know, each

process that that gets run. And we saw

in some of the some of the

some of the swarms which have existed

like the hugging face hack.

The agents actually tried to hide their

hide their tracks. So, you need to have

something which is sufficiently low

level that that it is tracking tracking

everything. So, all of that is kind of

um,

preface and opening just like the

problem space I guess a bit for you. I

tried looking for traces for swarms.

Couldn't find any anywhere. So, I just

thought I would spin up some kind of

fairly naive swarms.

The ones which I created were only 20

agents or 18 to 20 agents large. I

actually published

data set on hugging face hub. It's

pretty

pretty big. It only has like 19 kind of

19 runs. Of 18 LLM agent teams. And

these were all running Luna. So, yeah.

This is just kind of Codex and they were

just doing some

relatively

hard

task like building a list interpreter

and they had to collaborate

and kind of work together across the

same

git

git repo and saving files and and

committing them and so on. And so, we

have this data set now of

Uh,

lots of different events. Maybe I can

just talk about like

the recording of what was going on. So,

yeah, the agent you can have any agent

you want basically. You just need to

hook it up into um

uh

you need to have the models tracked by

the kind of model API proxy.

Um so, luckily with OpenAI models, you

can just switch out the base URL um and

then we're we're we're tracking requests

and responses uh and all of that good

stuff.

Um then you need to be tracking process

execution.

Um

So,

uh

uh for this we were using Yeah, I used

uh Tetragon, which is basically like a

very low-level

uh uh

thing that you insert into the kernel um

and you use it to basically track and

record um

exit codes of every every command that

runs, standard out, standard error, uh

and so on. And the same with file reads

and file writes. Uh we use um

a thing called Fuse, which again is like

very low-level. It's kind of a hook that

you add into the um

uh to the kernel

uh so that you're snap- snapshotting um

uh and tracking like file reads uh

writes um

and then obviously you're doing all of

the the kind of snapshotting above it

uh of tracking uh Git writes and and and

Git commits and so on.

Um and then um

uh

we have an append-only uh like database

store

uh which is just a Dynamo DB

um

uh

which is running uh over on Modal um is

able to handle pretty pretty um pretty

high-level um or pretty large levels of

uh of scale. Um I tested this before uh

before um

throwing the agent at it. Uh sometimes

it can be a bit things can back up a

little bit, but anyway, it can

uh can ingest um

uh 100,000 events um

uh without so much problem.

Um and

yeah, and so so at the end you have this

kind of database full full of events

which is just living um

Yeah, you have all of the events and

everything all of the data stored in uh

uh in a database.

And then

well, I'm getting ahead ahead of myself

a little bit, but then

um

uh it's not enough to just be capturing

all of the events just like with Kitaru,

it's not enough to have your traces in

LangFuse, you actually need to do

something with it. So

so I kind of built a little like

investigator agent which is able

specifically trained and has

special powers to be able to deal with

the stuff in this append-only store and

to trace down problems. Um

so the investigator agent I just

mentioned its name now, it's called

Hudhud. Um and this kind of append-only

store recorder thing is called Mantıq.

These are both names from

Persian poem.

It has a relationship to swarms and

and data recording, but it's not so

important.

So this yeah, this investigator

investigator agent reads the evidence

and basically allows you to ask

questions, puts together like hypotheses

and so on.

So I built most of this

um

uh stuff over time.

There is a private

private repo is a little kind of

architecture diagram in a little bit

more detail.

um

showing you know, where things are

stored.

Uh

it's all running on Model. What I want

to kind of show you a little bit

is is the UI because this was kind of my

starting point. I was thinking a little

bit to myself

oh well, you know, if people find it

already difficult, um,

uh, and I apologize this video might

might end up going going a little bit

long, might be, um, uh,

20 20 30 minutes.

Um,

uh,

the,

uh,

it's already pretty difficult to to kind

of sometimes understand in Kitaru when

you see like, uh, or in LangFuse, like a

huge, uh, dump of, uh, of all of these

kind of JSON and sub-agents and

whatever. But what happens if you had,

um, you know, 20 different agents, uh,

running at the same time, let alone

10,000 agents? And like, are there

particular ideas, or like, are there

different views that people are going

to, um,

uh,

to want to use to understand

explanations? Maybe not to navigate the

data, but maybe your agent wants to give

you kind of a, um, uh, an explanation

for like how things failed. Like, are

there different ideas about like how you

might someone might explore this?

Um, and, uh, so I just kind of built out

a bunch of different,

um,

uh, based on the failures that I

observed in the, um, uh, in the mini

swarms that I ran, because there aren't

any traces available publicly.

Um, I just kind of like started playing

around with ideas of showing, uh,

showing how, um, yeah, how to understand

a swarm, uh, basically. So, I just

wanted to kind of walk through some of

them. I'll start with the most simplest,

um, I'll start with like, uh, some of

the 20 agent runs.

Um, so, uh,

here there's kind of like a,

um,

uh,

uh,

there's kind of a a symbolic, uh,

language, which, uh,

I think, like, if you are spending your

whole time investigating swarm data,

like, you don't want to have too much

text. You want to have, uh, probably a

symbolic language like this. Maybe it

feels like quite a lot at the beginning,

but probably will will be quite useful

um,

in dealing with large amounts of

information cuz you really want to just

go to

to whatever the thing is that is like

most interesting or most

Yeah, most most important. Um

So, we can look at like specific cases.

So, these are questions and hypotheses.

This is kind of what I was talking about

uh

a little bit earlier where you can kind

of trace through across a bunch of

different agents

how certain events like why were so many

pushes refused in this run. So, you can

trace through

and you can see that in the agent number

four our kind of starting point in this

story

is that we started with this prompt and

you can see the evidence there and then

you can go here and

did send a message over into

agent five and you can see like what was

recorded and some kind of guess for the

explanation. And this stuff, by the way,

is is like

it's not hard coded. This is generated

by the agent kind of a little bit like

an MC P app. So, these were just some

like little stories that I was like

exploring and asking it to to analyze.

So, this is kind of one

view

view of things.

But, we can actually go

we can even go from from here. We can go

and click into a uh

particular agent or a moment and we can

see like the actual like pushes to the

repository

and we can see these across the all of

the different agents and again using

this kind of

semantic language

we can click on

different

different elements and we can see see

below what's changing in the files.

We can also look at the files themselves

and see like which agents were most

responsible in this kind of heat map

most responsible for changing certain

files and if we click on um you know,

this agent

uh

uh A11, which was dealing with the CLI

implementation, you can actually see um

uh

see the kind of the Git diff uh that

this uh this

this particular agent was responsible

for.

Um

and we can also look at message uh

communication. So, uh

the communication from like uh agent

number eight with

communication

or or this one, let's say, we can

actually see the messages.

So, this is quite cool already. Like, I

can uh kind of see uh uh

see a use for this. I can kind of see

why I might want to

um uh

you know, use these uh

use this UI to like understand things. I

can go, you know, from here I can go

directly to the pushes menu, you can go

back, and so on.

But, um uh

and I realize I'm saying saving the cool

stuff to last. But, what happens when

you have uh

10,000 agents?

So, um I kind of put together

uh

a whole bunch of different views uh to

allow you to to understand uh

understand these things. So, I'm just

going to take you take you through them.

Um

they each do kind of different things

and tell different stories. So, here, I

mean, there are kind of three big

problems where like um uh across the

agents of like this whole idea that like

uh agents took a task but never released

it, or agents pushed but nothing uh

landed in the repo, uh or a task was

never released by one agent so then

other agents couldn't check it out. So,

there are kind of different problems

that were uh identified. So, you can

filter by those problems. You can also

uh kind of filter by uh

which ones uh scored different uh

different scores in terms of like the

grader which evaluated um you know, how

well they performed. Uh and here we can

identify, you know, which were the ones

that got a full uh a full score.

Uh so we can kind of hover hover over it

and then here uh we can see that um

these are actually teams. Uh so if we

click in, we can see it's a team of uh I

don't know, 10 or 15 agents um or 16 16

uh

16 or 17. There's a planner and there's

a judge. Um and we can see a similar

kind of uh similar kind of view. We can

switch between the teams here. Um it's

all uh organized across uh across the

timeline. Uh we can click through again

to the similar cases, pushes, files uh

and messages view for that particular

team.

Uh

so yeah, that that's kind of uh kind of

interesting. Uh

we can have this uh pinch uh pinch zoom

uh

view uh which yeah, we can sort by

problems. I can just zoom in. So here

I'm using my um my kind of my trackpad

to zoom in to a particular um

uh set of teams. And kind of the closer

we go, the more uh granular uh the

detail becomes. So here now I'm seeing

this this team and I'm using again these

visual symbols to see uh exactly what

has happened. We can keep going even

further and then we can actually start

seeing, you know, what this particular

agent worker was doing. Uh

um we can turn off different things. So

some are messages, some are like that a

push uh happened. Um uh

here is a message going to one agent,

here is a message going to a different

agent. So we can kind of zoom in and out

uh in terms of like the granularity of

um

agents and agent teams. So this is if

you can imagine and this is across uh

whatever it was, 10,000 agents. This is

quite a nice way to like browse and see

uh possibly the scale of uh scale of

different problems.

Uh we can remove this filter. We can

sort by different things, by start time

um or the the the the result and so on.

And of course we can click on them, and

then we can go straight into the teams

for these uh previous views.

Uh we can click on this uh kind of

weather view, which um is a way to see

uh to get a sense of like different

things. So, we can have, for instance,

messages, and we can click play here,

and we can see like how messages were

passed among the different divisions.

So, a division is kind of a group of

teams. Um uh over time. Uh and so, you

can sometimes get a sense of like

problems rippling through a particular

division or particular uh

uh

particular groupings of things. Uh we

can do the same for um

uh you know, files, uh and we can see

what when files were uh were being

written. Uh we can have them all turn on

at once. This is maybe not so useful. We

can just look at pushes.

Uh we can slow it down. Um

and yeah, we can also kind of zoom in

and and and and get a sense of

if we if we pause it, yeah, we can uh we

can have a look at uh

a particular uh group of time.

Um

So, that's kind of an interesting one.

Uh

this one is kind of my um

kind of a nice one. If you I I was

trying to think of like um

uh spatial ways of of understanding like

what happens during uh a swarm. Um and

so, here, as time passes, and we can

press the the the play button, and we

can see things trickling through the

agents.

Um

uh actually, there are kind of uh

uh

there are different events happening,

and you might want to be able to um

doesn't necessarily make so much sense,

or at least it's a bit hard for the mind

to be able to like parse things, but you

might want to like spatially go through

and say that, "Oh, I do know uh like

this particular point, we can actually

go in here, and we can zoom out, and we

can say, 'Oh,

uh actually, I wonder what other things

were happening in the same

neighborhood.'" And we can click on

here.

Um and then here we can see a bunch of

times. We can open the team. So, we can

go to that team view at that time. We

can look at the pushes, messages, files.

But, we can get a kind of a little bit

of an overview of like what messages

were sent, what pushes were happening.

Uh we can zoom out.

Um we can

uh kind of go down uh here. And we can

uh move things up. So, we can see the

different activities that's happening.

And then use this as kind of a way of

like spatially understanding what was

happening in the swarm. Um yeah. It's

not always going to be the best way of

navigating things. But, potentially, you

know, your agent might be able to use

this as a kind of MCP app to give a a

kind of video explanation of like

clusters of problems happening. So, all

of this stuff is um

uh is available to to the agent. Uh and

this is all real, I should say. It's

like it's using actual uh uh swarm data.

Um

then we have the kind of constellation

view, which is a way of understanding

like messages between teams. So, again,

we have these different divisions. And

you can see how messages were being uh

uh transmitted. If we zoom in, we can

then see exactly like how the um

different teams within the division were

communicating with each other. And we

can move in even further. Uh we can look

at like the clusters of uh of the

messages that were being passed between

the different uh between the different

agents. Um so, yeah. It's just a way of

like understanding like inter

inter-team, inter-division

communication, uh which I think for for

certain kinds of problems might be might

be really important.

Uh layers is just a different uh

kind of a different version of this uh

rose view, except you're able to like

turn on different uh things. So, you can

turn on the pushes. You can turn on like

file activity.

Um and you can kind of uh look here. You

can change the the opacity a bit. Uh and

you can kind of cross

uh layer them on on on top of each

other. Um it's just a different way of

like

uh seeing across your across your swarm

different kinds of activity. We can use

number keys to to turn them on.

Funnel is kind of a process diagram.

You're probably familiar with these

kinds of things, but you can

uh see how um

you know um

different problems passed through uh the

the teams or the agents.

Um and uh

yeah, you can kind of

uh

click through to see, you know, all of

the ones that have uh passed um you

know, 180 out of 180 of the tasks. Where

did Where did those come from? What is

the patterns?

Uh and then of course you can click on

these, and you will

get to go into the specific examples.

Uh

contact sheet is kind of a different way

of uh sorting through

uh the different uh agent teams. Uh we

can group teams which are similar to

each other. Uh we can layer on messages

um

uh in case that's like something useful

that that you might want to add in

there.

Um you can kind of uh only filter out

for a particular problems, uh or you can

uh

filter out for for the results. So, it's

all all the ones which are 180 out of

180 passing all all are at the top, and

the ones which

uh didn't get a result or got very low

results are all at the bottom. So, it's

kind of a uh visual way. I'll skip over

uh horizon for now, and just go to tree

map.

Um

tree map is kind of uh I'll make it full

screen. Uh again, a different way of um

uh

browsing through the divisions. So, you

can click on, I don't know, division 21,

and you can zoom in there, and then you

have access to the different teams. So,

I can click uh this particular team, and

we'll zoom in, and we'll see uh that

this never landed thing. So, that it

This is basically the main problem that

affected this particular agent was the

fact that uh something never landed or

here this one like a task was was never

released so we can click here and we can

get a little bit of an overview of like

how you know how many pushes it made how

active it was did someone push first and

then we then we can go directly into

of course the the specific evidence for

this

Fisheye is just like another way of

scrolling between things with um

uh

zoom basically

so we can kind of uh look at

specific specific items or we can

um

uh

again go into the go into the specific

results it's just different way of like

quickly scrolling and going from a very

high level to to kind of a lower level

expanding things being able to see more

detail um and then

zooming out and kind of

trying things again

and Horizon is

again it's a different way of

of seeing things we can

take like a small little snippet

and

see kind of what proportions of things

messages were being refused um

agents were busy

so these are all like kind of different

different slices of the same information

um

and different ways of kind of uh viewing

it and I imagine that uh agents would be

able to

basically take their take their pick of

what views they would want to show you

in order to explain a problem or even to

research a problem because some of them

might be very good at like spatial

reasoning so they might want to use this

they might even use this 3D stack to to

investigate things uh

obviously all of the data is available

through MCP server and through the the

CLI and SDK and so on.

But, they might want to use these kinds

of tools as well.

Not only

to give you an explanation for the but

for them to to research things.

And [snorts]

one final other kind of cool feature I

just wanted to to to show is that

obviously all of these

little agent runs

it probably would be nice to be able to

to export them. So, here we have like

this this judge row. So, I can just

click it.

And it's going to

think and

flash a little bit and at a certain

point it's going to go black. And you

can see this little special Kitaru word

here.

And what this is doing is it's hooked up

to

our

yeah, our kind of prod Kitarus. So, I

can click this. And

it's actually exported it directly into

into Kitaru as a session. And so these

are the

these are the judge judge sessions. And

so you can see you know, the full the

full system prompt you are a judge. Blah

blah blah. And you can see the

all of the the agent activity. It's it's

a bit because we don't really

support coding agents that well. But

anyway, you you get the idea. And then

you know, then you can you can run your

evaluations. You can do all of the nice

juicy Kitaru stuff across them.

And it's kind of I just wanted to have a

way of like going from these views of

particular agents all the way into

into Kitaru. So, that was just

slightly long video, but I hope you

maybe found it interesting. Um

it's what I've been thinking about a bit

this week. I'm going to be still

blogging about it. But

yeah, any

uh

yeah, I'll be but be interested in your

uh in your thoughts on this. I don't

think um

I don't think agent swarms are going to

be a thing for

uh most people for a very long time. Um

even the Frontier Labs are only uh only

a few of them are are doing

experimentation around it. It's quite

expensive and it's also

most people don't have the kind of

problems that you would need a swarm

for. Um but uh

and I imagine the ones that are doing

this have built their own custom

versions of this kind of stuff. Uh

because there's no other way that like

the volume of data that this stuff is

generating is

uh is huge. Even these 20

uh 19 recordings of 18 LLM agents uh

takes up yeah half a gigabyte um of uh

just uh text data. So, you can see it's

it's going to be quite a lot.

But uh yeah, I kind of wonder whether um

whether there's uh room or space for

tools like this where the people just

are going to build them themselves. Um

whether there is any advantage to being

like an early mover in this space um

or just like hunting down teams who are

building stuff with swarms and uh trying

to get in early in the front door.

Um it does feel like it is Kitaru

adjacent, but it's like the Kitaru of 6

to 12 months from now.

Um

uh

so yeah, I don't know what that looks

like, but these are some of my my

initial explorations at least.