Mira Murati's 975B Open Model, Ramin Hasani on Post-Transformer AI, and Demis' AI FINRA | EP #271

Mera Marotti, the former OpenAI CTO,

just shipped her first model. It's

called Inkling. Customization over

leaderboard dominance uh is what's going

to win her the day.

>> She's built exactly the thing hitting

the market that exactly what everybody

needs. Right now,

>> I want to pivot to a discussion of

liquid AI, the small language models,

what they are, what they mean. Our

mission has always been building

efficient generalpurpose AI at every

scale that explores the computational

graphs of intelligence beyond

transformer and then figure out what

should be that architectural design that

brings the same level of intelligence

that a frontier model into let's say on

on a CPU.

>> CEOs building the most powerful

technology in the world are asking to be

regulated. Demisabi, CEO of DeepMind, he

called for a US-led frontier AI

standards body modeled on FINRA. When

the incumbents ask for the rules and

they set the standards, they set up a

barrier for all the entry-level labs

coming in. Let's just be real. AI moves

way way too fast for any kind of

traditional uh bureaucracy. How quickly

can you do it is going to be huge, huge

challenge because

>> now that's a moonshot, ladies and

gentlemen.

All right, everybody. Welcome to

Moonshots, your number one podcast in

all things AI. Your front row seat to

the singularity. I'm here with my

magnificent Moonshot mates, our original

quartet, AWG, DB2, and Seem, and a

special guest, Reine Hassani, co-founder

and CEO of Liquid AI, and a pioneer in

small language models, which we'll dive

into. Raine, welcome. Uh, where are you

this morning, pal?

>> Thanks so much for having me. I'm

actually in Spain right now.

>> In Spain? All right.

>> There's nothing going on in Spain this

week.

>> God damn it.

>> Yes.

>> I'm I'm a I'm struggling from yesterday

cuz I'm a long-suffering England

supporter.

>> Um it was a very difficult game to

watch. God, they were like just had it

with six minutes to go and they blew it.

>> Yeah.

>> And Messi's a genius.

>> That's the round That's the round ball,

right? Is that

>> That's the round ball. Now, Peter, this

is where the Faulland's war gets

relitigated on a soccer pitch. Oh god.

You know, I just flew in last night from

Zurich and I had the most painful

experience, right? I don't know why

every airline doesn't have Starlink. You

know, I'm suffering on some me, you

know, some meager thin pipe connection

and you're flying over the poles over

the, you know, the Northern Territories

and there's nothing. And I'm trying to

get ready for this pod. I was like,

"Please give me give me some bits." So

anyway, challenge.

>> You must have grown up on soccer, right?

didn't you were in Vienna for a while

and getting your PhD and or your

undergrad or whatever it was. Uh

>> yeah. Yeah. I mean soccer has been like

a big thing, you know. I'm Persian and

Austrian like at the same time, you

know, like it's a big thing for us. So,

um yeah, like competition is something

that, you know, uh it's it's extremely

core to what we do even today, you know.

So, and uh I feel like that's like one

of the main drivers like sports and

everything like has been part of our

lives like from day one and then getting

into science the same thing you know now

getting into ventures same things you

know and that's uh that's what we're

doing

>> just compete compete compete compete I

love it

>> well are you there for a little bit of

time or you coming back soon

>> no I'm coming I'm I'm flying tomorrow

actually back to San Francisco

>> and Salem are you are you jealous of

everybody of him being in Europe or are

you happy to No, no. Three weeks

bouncing around in 10 different spots.

I'm very happy to be home right now. I

was just in Spain where me and myself.

So,

>> uh, all right.

>> There was a lot going on actually like

in in in Europe, you know. So, that's

>> same same like you 10 different places.

Then,

>> you know, Alex and I were just

reminiscing the fact that Europe's uh

sort of major advantage in the future is

it's going to be a museum of the way the

world used to be. Um,

ouch. Ouch.

But it is beautiful. There's no

question. It is gorgeous. All right, I

want to jump into our first

conversation. We have a lot to unpack

here. And of course, our mission is

keeping you aware of what's going on in

the world and giving you sort of the

optimistic, hopeful vision of the

future. Uh join us and uh keep up with

the incredible pace as we head towards a

singularity. So our first story today,

once again, CEOs building the most

powerful technology in the world are

asking to be regulated. You know, last

week Sam Alman published an op-ed in the

Financial Times proposing a framework

for a US-led international forum that

would establish standards, provide

expertise, impartial analysis and

capabilities, and assess risks. This

week, both Elon and Demis are adding

their voice to the regulatory

conversation. Elon says he expects a

standalone uh regulator similar to the

FA or FCC to emerge at some point

because in his words the consequences of

AI going wrong are severe. Then this

week Demisaba CEO of Deep Mind went

further in an essay titled A Framework

for Frontier AI and the dawning of a new

age. He called for a US-led frontier AI

standards body modeled on FINRA, the

industry funded watchdog that polices

Wall Street under SEC oversight. He

wants the FINRA equivalent to test

Frontier models before release. He

reportedly wants this up and operational

before the end of the year. Let's take a

look at a quick video from Elon and then

let's jump into this conversation.

I think the the general I think it's

clear that there's a strong consensus

there should be some AI regulation that

it would be in the best interests of the

people to do so and I think we'll

probably see something happen. I don't

know on what time frame um or exactly

how it will manifest itself. I I don't

know. I mean this there's clearly we've

created regulatory agencies before. Um

while our regulatory agencies are not

perfect um and I deal with regulators on

a very frequent basis um with automotive

um you know communications Starlink um

and then uh FAA with with rockets.

>> I think the probability of there being

some sort of AI regulatory agency that

stands on its own similar to the FAA or

FCC is likely at some point.

>> You think so?

>> I think so. Um,

now the the reason that I've been such

an advocate for uh AI safety in advance

of sort of anything terrible happening

is that I think the consequences of AI

going wrong are are severe. Um, so we

have to be proactive rather than

reactive.

>> Um, amazing. So I this is a conversation

we've seen over and over again and I

think the government, the public and now

the CEOs want to be leading this. I like

the approach that Demis laid out, right?

Um, but the challenge we have to discuss

is when the incumbents ask for the rules

and they set the standards, they set up

a barrier for all the entry level labs

coming in.

See or Dave, do you want to jump in

first?

>> I'd be very curious to know, Ramine, if

uh do they reach out to liquid AI and

say, "Hey, join this, you know, we're

going to create a FINRA like regulatory

body." Um you the reason FINRA works

fundamentally is because people from the

industry who know what they're doing are

willing to join it. They're definitely

not willing to join the government in

general, but they're willing to do a

year or two in a regulatory body. It's

actually kind of a badge of honor. So

for this to work in AI, it would have to

be something cool. And people like

Ramine or maybe you know some of the

people on your team would need to come

into your office and say, "Hey boss, you

know, I'd love to do this for a year. I

think it's really good for the world.

Will you let me do it?" And then you

would also have to be like yeah this is

a functional organization go for it. So

if it passed those two hurdles I mean it

might it might actually work. I don't

know what do you think. Yeah, there's

like you know like there's a capability

kind of threshold that we we're trying

to define right now and some some sort

of an iteration is needed to see like

how this um how how this framework it

has to exist you know that's that's for

sure you know there has this has to be

there but uh it has to be related to

capability and then the thing that

becomes a challenge is that that there's

a horizontal kind of capability lock

into like active like let's say like

enterprise deployment of AI and then

there's the vertical because if you go

to different verticals like for example

we operate on on on device and with

enterprises that are connected to the

physical world you know like we're

connect we are talking to car

manufacturers like semiconductor

business you know and laptop business

you know like people that are building

like AIPCs and then we are also working

with financial services and we working

with like e-commerce and and biotech

kind of companies and we see like in

different verticals you know like the

enterprise applications themselves and

enterprise criteria for let's say a

limit or let's say a regulation kind or

a governance kind of a structure is very

different you know so for us it becomes

a lot more kind of verticalized because

we're building a specialized models and

those specialized models like

pervertical we we have had like

conversations with the DoD and we have

had like a joint uh uh submission of

something I think with AMD like pretty

uh like just recently like we with with

our team to really have um have a say

basically like in in in the design of

like these regulatory kind of things and

I think as an exploration I think

Everything has to be like getting

started. I like to look at it as a game

theory kind of uh way of uh looking at

it like how to design like policies in

general. Like it would be a stake kind

of game. I don't know if anyone is

familiar with I don't want to nerd out

like pretty soon on this but we can we

can talk about this.

>> Alexon

the better.

>> Sooner the better.

>> Yeah. So I mean stakeber games like

essentially like where two policies like

basic like there's like a you know like

you have like a policy maker and then

you have agents or bodies that are

working in that uh kind of game theory

kind of optim they they're trying to

find an equilibrium you know what is the

optimal policy and what is basically

which is good for both right and then so

there's the frequency of action usually

policy makers are slower than the agents

in the society you know so if you think

about like you can you can really model

like that, right? And then you can uh

you can figure out like an an

equilibrium. This is not a Nash

equilibrium because everything doesn't

happen simultaneously. Regulations

happens and then you agents react and

then you iterate kind of accordingly and

then you change those uh uh regulations

basically. So I think

>> I see you trumping at the bit here

buddy.

>> Yeah. So I think I think what Raine is

saying is exactly right. The problem is

we have no mechanism for that. Right.

like

if you go down the path remain that

you're talking about you end up with the

appropriate structures that are adaptive

and API based or like driven by

benchmarks or something but the

mechanism that people have today is just

static law and the minute you pass the

law the law is going to be out of date

right the I I found that the FA and FCC

analogy is is is pointing in the right

direction but a let's just be real AI

moves way way too fast for any kind of

traditional government bureaucracy

Right. So, you're going to need you're

going to need a standards body. You're

going to need real-time audits. And

you're going to need open evaluation

suites. Um otherwise, you're going to

end up otherwise you're going to end up

in political gatekeeping and then you're

in a mess. The problem

>> isn't that what's good about FINRA. It's

not a government agency. It's an

industryfunded self-regulatory org.

>> Uh it is, but then the teeth go to the

uh SEC, which is essentially being

dismantled right now. So there's all

sorts of issues here. I I I I think the

the trend is correct, but how quickly

can you do it is going to be huge huge

challenge because forget passing a law,

passing a structure where you have a new

construct like this takes a long time

and it takes forever uh in Europe. I

>> I think Ramine nailed two things that

are very different from FINRA right out

of the gate. One of them is, you know,

at Vesmark, if somebody on our executive

team said, "Hey, I want to be part of

FINRA for a couple years." We would say,

"Sure, put on your suit and tie. go to,

you know, go to the meetings, come back

in two years, we'll still be here.

You're not going to do that. Like if

Alexander Amini or Matias Lechner came

into your office or said, "Hey, I I'm

going to check out for 3 weeks." You'd

be like, "No, you you can't do that

right now." So it's difference number

one is nobody's going to carve out the

time to do something for years like they

do at FINRA. The the other big

difference is AI can help regulate

itself and FINRA, there's no equivalent

to that in FINRA. It's all people just

chatting for long periods of time. But

you know when you start talking about

Nash equilibriums and other ways to to

automate the process of regulation

that's a big big difference as well. So

the FINRA analogy has some legs but you

know the differences are bigger than the

similarities.

>> Alex is I want to hear your voice on

this B.

>> I I tend to think this is a bad idea. It

smells like regulatory capture. It

smells like the attempted formation by

Demis of a cartel of Frontier Labs. And

I think the elephant in this particular

room is openweight models and research

that lives outside of the frontier

capabilities. And it's very easy to

imagine a future with FINRA or or other

I mean worst case scenario FDA like

capability even though outgoing

personnel from the current

administration have declared uh with in

no uh no equivocal terms that there is

going to be no FDA for AI regulation.

that that would be maybe on the the

worst case end of the spectrum that we

we see the emergence of some sort of

cartel of frontier labs that locks in

certain practices, certain price

performance optimal frontiers that try

to box out open weight or open-source or

uh say university driven or other

nonincumbent

frontier models and I think that would

be an utter disaster for both the west

and the world for continuing to advance

us towards everinccreasing super

intelligence capabilities. I I just

don't think it's a good idea.

>> You know, and the other elephant the

other elephant in the room here is these

CEOs who are asking for some level of

regulation I I think are are looking for

a backs stop. You know, if things go

wrong, they want to be able to point at

someone else. Now, I mean, we're all

super fans of the optimistic vision of

AI, but there's going to be issues that

materialize. is there going to be rogue

AIs that take down a power grid or take

down you know stock market or something

like that for some period of time and I

I guess they you there going to be

lawsuits flying as a result of that

unless there's a regulatory body that

that backs stops these large these large

models and these large frontier labs

>> maybe uh there are I think at least two

different frames that one can look at

the liability side from there's regulate

the inputs that is to say like have

something that's FINRA like or FDA like

that regulates the raw capabilities of

the models at model construction time.

That that's one end of a spectrum. The

other end of the spectrum is regulating

the actions of the models. Like you you

let the lawsuits fly if if a model takes

down a stock market or does something

else that uh otherwise harms third

parties. That's the other end of the

spectrum. It's not obvious to me that we

should be in the business of regulating

super intelligence at super intelligence

time. That's that's maybe tantamount to

thought policing the AIS. And I'm I'm

not generally a fan of that notion of

let's thought police the AIS but not

thought police the humans. We don't at

least in in the West have a practice of

regulating what's in our minds. We we

don't have a practice or a tradition of

regulating an upper limit say or via

some sort of regulatory code saying

humans uh natural persons can't be above

some level of intelligence. is not

obvious to me why we would create a new

tradition of regulating or otherwise

coordinating the upper intelligence of

non-natural uh entities perhaps soon to

be persons but regulating the actions

that in at least the western legal

cannon that we do do and that I'd be

much more supportive of.

>> So do you Alex, let me ask you a pointed

question here. Do you think that this,

you know, sort of outcry for regulation

by the large frontier labs is is

regulatory capture that they're just

trying to build a moat against uh

further players coming in? Or do you

think they actually want to provide some

level of safety? What's their underlying

driver here?

>> I I worry that it's more regulatory

capture and creating moes for themselves

in a hyperco competitive landscape. And

it is h I mean, it is a rat race at this

point, the frontier. And I I I do worry

that it's more regulatory capture than

it is some notion of protecting the the

future here. Seem, what do you think?

>> Uh, not workable.

>> Well, I know that, but do you think do

you think it's regulatory capture or do

you think that the that these CEOs are

trying to just make sure we've got a

safety a safety net of some type?

>> I I I'd say it's like 50/50, but I think

there's a bigger problem. There's an

elephant in the room here. Some

>> there's already an elephant in the room.

We have a room has to accommodate so

many elephants. We need some other

non-human animals.

>> Better get a bigger room. You've got uh

non-state actors and other folks that

won't listen to this structure and

you're back to square one. What's the

What's the point? I'm going to say it

again. I've said this repeatedly. I see

no mechanism to regulate AI. It's moving

way too quickly. Any regulatory is

static.

>> And so it's going to have

position on that one. just if I may

Peter narrowly on that I mean there are

definitely hypothetical mechanisms and

that I'm not supportive of for

regulating AI like we royal we the the

US and China if going back to I think we

gestured at it in a past pod but uh past

proposals to say regulate the foundaries

regulate the chip outputs regulate the

data centers establish mutually assured

destruction type schemes where the US is

monitoring Chinese data centers and vice

versa like there are There are schemes

there are schemes at choke holds as

Peter says in the supply chain by which

one could imagine doing this

>> interesting mechanism

>> the only mechanism it's going to be like

a pandemic style threat detection that

would be globally agreed and I don't see

how we get there

>> well you don't need global you just need

US and China right the rest of the world

is is basically outside those blocks or

inside those blocks

>> all right well I think my guess There's

probably a poly market out there uh we

can we can look at and if someone wants

to search on it you know the question of

will we have a regulatory body by end of

the year right we have Demis saying by

the end of this year you know Elon

stepping up uh and and Sam obviously

trying to on his own on the side trying

to push for this so when the three

largest labs uh are pushing for it my

guess is the government will latch on

and will do this I don't think it's a

matter of if it's only a matter of when

and what the structure will be.

>> Well, I I should also note that Elon

clip I think is from three years ago,

which is interesting. You know, it's

from three years ago because Elon had

his sort of like painted on uh Iron Man

goatee uh when he was in that that that

phase. Uh so, so Elon's been forecasting

this for at least three years. Others

have been forecasting it for decades. We

still don't have it. We have like

subdivisions orgs within NIST that are

uh working on standards but that's not

really regulatory body. We have

executive orders that are creeping

towards a regular regulatory body. But

you know at what point it do we sort of

are we frogs boiling in water where

there's just like a creeping roll out of

increased standards expectations of

early reviews but it never quite reaches

regulatory agency level before we

achieve whatever escape velocity we're

heading towards.

>> Well, uh we're going to monitor this one

closely for everybody. I I think my

guess is we see this before the end of

the year and the question is can we see

something that's intelligent. Uh let's

go to the next story which is related uh

and this is a wild one comes from the

Washington Post that the White House is

reportedly weighing a capability

framework that would clear US models

open or closed as long as they stay at

or below the level of China's best

openweight model. What's the

translation? So the proposed ceiling for

what American companies can openly

release is pegged to what China has

already put out on the internet for

free. So here's the logic. Chinese

openweight models reportedly trail US

models an average of 7 months. I think

that's been closing over time. Uh so if

anything is at or below that, it's

already out there. It's an implicit

admission that open models cannot be

unshipped. Models like deepseek have

already been downloaded millions of

times. So once China releases a model

freely, banning it is impossible. So the

US response is to define a permissible

ceiling rather than a wall. The

implications were tying our open release

ceiling to China's pace of release

effectively giving you know Beijing

control. If they push their open weight

models higher then the US can release

higher models as well. If China holds

back, then they throttle us. And it's a

very strange mechanism. I was surprised

to see this. Alex, let's go to you first

on this one. What do you think of this?

>> Uh, I mean, the the obvious note here is

this creates the perverse incentive to

let China win the race to ever greater

super intelligence so that Western

models and western labs can escape

regulation. I'm not a fan of this. Uh uh

raine gesturing at you from a game

theoretic perspective. This is the I

think this would be the moral equivalent

of throwing the steering wheel out the

window in in a game of chicken. Not such

a great idea. Not supportive of this.

>> I love that. Oh my god. See, what do you

make of this? Is this just perverse

Washington DC logic?

>> Yes. This is like trying to uninvent the

printing press. I mean you we're

throwing the kitchen sink at things

trying to to to solve something that's

already a problem. The you you have to

move from like prevention and whatever

to adaptation. You have to go to that

and we we don't have the mechanisms for

that.

>> I mean you know I mean would you even

listen to this?

I mean what what logic

>> you might

>> well

>> what do you think of this? I mean the if

I just look at the the the the

progression of the technology itself

like it's it's getting into into the

place where like you AI are designing AI

like we you're doing the same things and

all of the labs are doing this and the

pace it's just the pace of model

development is like getting so so so

much smaller you know that is um is

becoming like exponentially more more

difficult to really like impose any any

of these type of constraints and I know

like they had these type of

conversations But it's just at the level

of conversations, you know, like these

are the things that are getting leaked

outside of

>> White House for ideas.

>> Let me let me give a headline from for

Alex for for his next newsletter. Um,

the singularity is becoming a trade

dispute

>> for the next newsletter. That was like

two newsletters ago.

>> Okay, fine. Whatever. That's out

already, but thank you.

Um, you know, I can just imagine where a

US Frontier Lab CEO calls DeepC can say,

"Would you please accelerate your next

model release? We want to get ours out

as well."

>> Or you see worst case scenario. I mean,

there there's actually a an even worse

scenario, which is you start to see the

the best, if not Western labs, unlikely,

the best Western researchers move to

China to escape this regulatory

framework. That would be a disaster. I I

think and and we've seen this, by the

way. There's precedent for this. We saw

this in biotech where China now exceeds

the west in terms of number of trials.

Like China is experiencing a biotech

boom that could happen in AI as well

disaster.

>> It's it's much more specific than that.

If you look at all the quantization

research, all the best stuff came out of

Microsoft research in China. All those

people now are at Chinese labs. They're

not they're not still working for US

companies.

>> China ran away with turnery and one bit

quantization. You see a little bit of

Western research. I don't think we're

talking that much about it in in this

episode. you see a little bit of uh

encouraging western research on like one

bit or 1.58 bit quantization but China

ran away with it due to constraints.

>> Yeah, it's a new company.

>> Look, this is a huge problem, right?

Because over we've seen throughout

history that open ecosystems always win

>> and this is not open versus closes which

open ecosystem wins and the US's

historical strength has been open per

ecosystems with permissionless

innovation. you like abandoning that

would be the weirdestly strategically

bizarre thing we've ever seen.

>> Yeah. The the other I mean there's even

a meta worry I have which is how do we

even define capabilities and and I worry

a little bit not just about regulatory

capture of the labs themselves. I think

there's actually so so sorry to be like

a a meta doomer here. uh there's a worst

worst worst case scenario which is we

freeze in or otherwise lock in the

benchmarks for how we measure

capabilities and that that would be I

think maybe even worse than just locking

in the incumbents as labs because if if

someone somewhere ratifies all right

like whatever index of evals this is

going to be the rubric going forward for

how we measure what's above the

threshold for frontier versus below

what's a frontier model versus not I I

worry that could so distort model

capabilities like they'll overex

exercise certain capabilities

deliberately and perversely under

incentivize or underbenchmax others that

it'll just totally distort maybe topize

the the future landscape of super

intelligent capabilities.

>> All right. Well, again this is a story

that we'll be we'll be following on this

news of ope models. So in the past

>> there's our topiary right there. In the

past, we've been discussing how ope

models uh have been in the US have been

lagging in China. We have Nvidia's

Neotron 3. We've got Google Gemma 4. But

that changed last night with some

breaking news. Mera Marotti, the former

OpenAI CTO, uh who walked out and raised

her one of the largest seed rounds ever.

Uh it was incredible uh financing she

pulled off in the background. just

shipped her first model uh for her

startup called Thinking Machine Labs.

It's called Inkling. It's an Opalweight

Foundation AI model that can be

downloaded by anyone, fine-tuned and run

on prem on your own hardware. Uh the

specs are serious. Uh it's a mixture of

experts model with 975 billion total

parameters. Only fires 41 billion at any

one time. So it keeps it, you know,

keeps the model going fast and cheap. It

was trained on 45 trillion tokens of

text, image, audio, and video. And very

importantly, reasons natively across all

four. Reuters Muse framed it exactly

right. Quote, "This is meant to be a

western alternative to the Chinese opate

models, Deep Seek and Quinn, that have

dominated the opo leaderboards." Uh,

now, interestingly enough, Maradi her

bet is contrarian here. She's not

claiming it's the best model on Earth.

Her own blog says so. Uh she's betting

that an AI that AI companies can adapt

her models for themselves. That

customization over leaderboard dominance

uh is what's going to win her the day.

>> You you've you've hit there, Peter, on

the really big thing. She's making this

she's pushing on the customization lever

>> and this because it's not going to be

the future is the raw power. It's going

to be the adaptability that's going to

win. And this is she's built exactly the

thing hitting the market that exactly

what everybody needs right now

>> and people owning their own models

working on prem and not giving their you

know uh their controls to the large

frontier models. I mean I I I do hope

this begins the race for powerful

openweight models in the United States.

>> Well it's it's worth looking at the raw

capabilities. So if you believe the eval

hopefully that thinking machines aka

thinky has released it's stronger than

neatron which is great like neatron

you'll recall from past pod where we

were discussing Alex karp's rant on

sovereignty of models neatron is one of

the incumbents at least on the American

side for openweight frontier models so

this this seems to be at least according

to the eels that think he's released

stronger than nematron which is great so

the the west now has a new frontier here

openweight model. It's weaker than GLM

5.2 which is arguably the strongest or

one of the strongest Chinese openweight

models and openweight models overall. So

it's not it's not one of the strongest

openweight models overall in the world.

It's obviously weaker than the closed

weight western frontier models. But I I

think point one it's great to have

better stronger western openweight

models. Point two, I I think it raises

the question, why has the West been so

bad at releasing frontier openweight

models and why has China been so good at

it? And I think it comes down to you

show me the incentives and I'll show you

the outcomes. I think the west has been

poorly incentivized to release strong

openweight models because these API

based frontier models are just such a

good business model. And we see

Anthropic about to IPO at a trillion

dollars and we see OpenAI planning to

eventually IPO at a trillion dollars.

And in China, which has been GPU and

compute deprived on the one hand, and on

the other hand has the CCP declaring

5-year AI plus plans to integrate AI

into the rest of society. has all of the

incentives a different incentive

structure than what the west has. China

has been much more incentive

incentivized to make money from the

integrations between AI upstack on

applications like robots and downstack

into the chips than the west has which

is more horizontally stratified. So to

the extent that thinky has been

incentivized in the west due to

competition and due to just a saturation

of the frontier by the closed weight

models into looking a little bit more

dare I say Chinese in terms of their

outlook and their incentive structure. I

think this is very helpful to finally

have enough competition in the west

that's creating ways to monetize

openweight models other than just per

token sales namely selling them into

enterprises and what you incentivize.

>> Two more.

>> Two more thing. I agree wholeheartedly,

but also you have to note that OpenAI

started open source open weight and then

went closed big revenue and uh Meta also

was the leader of

what happened to now it's closed. No,

they they have a new model out and it's

it's closed API. I mean, it's exactly

what Alex said. If you throw your model

out there as open source, what's your

revenue model? So I think, you know,

there's a real possibility that that you

put a data point on the map with a a

really solid open- source release that's

not quite on the frontier. You generate

news, then you have a data point on the

line, then you do another, then you do

another, and then when you have

something really groundbreaking, then

you go closed source and you launch an

API into corporate America. And so that

that's a wellworn path. So I wouldn't I

wouldn't say this is necessarily a

religion at thinking machines that

they're going to stick with. You know,

the trend has been the opposite of that

in the past. Raine what?

>> They're they're leaning into fine-tuning

as a service. If fine-tuning as a

service becomes like something at scale

revenue generation wise, I think maybe

this has legs, but who knows?

>> Yeah, it's a matter of like the business

of the company, you know, like thinking

machine can do uh three more iterations

of their pre-training or post- training

kind of RL kind of environments and

benchmarks like those numbers that you

see on the benchmarks and release like a

like a better model. But what they what

what what their business is their

business is fine-tuning. Like this is

kind of the place where customization

has been like something that everything

like the whole the whole market around

customization has been very empty. Like

if you look at the first attempts like

OpenAI released the OpenAI tuning like

fine-tuning kind of 3 years ago or

something it never took off. So they

took like a really good uh approach on

designing the base for fine-tuning

larger instance of the models for

enterprises because as you see like the

model layer is not anymore like you know

like the the the place where you can

actually extract value especially if

you're not hitting the maximum frontiers

you know like uh and even the open

weight kind of models when we're talking

about sovereign AI and integration of

these models into enterprises you need

to leave some room for let's say

fine-tuning these models and what they

have what what I think their business

strategy around what they're doing and

this release is genius because they're

deliberately releasing they're they're

putting they're leaving some room for

fine-tuning so that people can come in

and using their business uh uh their API

business because that's even generating

if I think in the order of uh one to two

orders of magnitude more tokens as well

you know on the on the on the

customization side so that would be like

even printing money at a larger speed

like in the in the in

absolute best case right

business entry

>> to to add to Raine's point I I think the

situation maybe is is even more extreme

so a couple points one OpenAI was the

first to my knowledge to launch

reinforcement fine-tuning RF as a

service and no one used it uh the the

whole tech world everyone I speak with

no one used it uh it was barely

advertised by OpenAI second point open

AAI shut off their fine-tuning API open

AI was one of the earliest if not the

first to offer fine-tuning as a service.

>> We used it all the time. It was it was

incredibly cool for its time

>> and they they've just they recently in

the past few months they announced it it

has either already been wound down or

about to be wound down. The fine-tuning

API has been shut off. So that I mean it

raises the question is is thinking

machines bet like explicitly contrarian?

Are they thinking that we're going to

end up in a world where reinforcement

fine-tuning and RL fine-tuning in in

general and fine-tuning like that's the

paradigm? They may be right, they may be

wrong. There there's an alternative

vision where RF just dies. Uh and we the

the baseline models are so generalist in

terms of their capabilities that all you

need is prompt engineering and there's

no need for RF at all. Alex, you talked

about the Alex Karp rant, right? Yes,

the result of that was um don't allow

don't use a model that is has all of

your data open to your competition. And

I do think we're going to see a real

push over the next months to years where

people want to use fine-tuned opate

models that they own on their own

hardware in their you know onrem and if

that's the case then the question is who

are they going to use which models are

they going to use are you know and is

the US going to start to regulate

against Chinese openweight models in

which case a dominant US openweight

model is going to take is going to have

an advantage and so is that the bet

mirror is going after um you know we're

going to probably see my guess is Google

step up in this area as well very

shortly you know take Gemma 4 to the

next level and hopefully we get some you

know two or three major in the same way

we have a closed you know the closed

model Frontier Labs competing and

dominating in the US hopefully we'll see

that competition give birth to you know

very strong opio models

>> it just to build on something you know

Alex and Verine were saying you know if

I compare today to a month ago you know

we've been fine-tuning Quen all week and

and the idea of using Inkling sounds

really compelling to me and you know our

companies are using liquid as well. A

month ago to fine-tune these things with

some huge engineering effort that

required AI experts. Now with Fable 5,

it's just a prompt.

>> So let's back up one second. Dave,

explain what fine-tuning a model is for

those who don't know.

>> Well, you know, back when GPT2 and GPT3

came out, you could actually very easily

fine-tune by uploading text right into a

window and say, "Look, you're pretty

smart, but you don't know anything about

my laundromat." you know like what hours

were open now who our employees are

entire payroll let me dump that data in

too and retrain the model with that

knowledge and if you didn't do that you

couldn't do anything useful because it

didn't have this holistic I know

everything capability back then so

without the fine-tuning it was

borderline useless to to use the models

then the models got so smart that

they're pre-trained with now 45 trillion

tokens which is basically every word

ever written by humanity has already

been trained into the model so people

tend to use them in their vanilla form

today and just say here write this code

for me or here drive this car for me

because it's already in there but then

when you get into biotech research or

you get into aeronautical or the

Mercedes you know like Ramina is doing

there's a whole bunch of proprietary

company knowledge that actually isn't in

the model so right now we dump it into

the prompt field and say okay here it is

in prompt form but that's hugely

inefficient

>> and you dump it into open AI and you

dump it into anthropics uh you know

model which now makes it accessible to

everybody else as well. I mean,

>> yeah. Yeah. I mean, Sam Sam and Dario

can see everything. All your proprietary

information, they're looking right at

it. That's what Alex Karp was ranting

about when he said, "They're stealing

your weights. They're stealing your

alpha." What he really means is they're

looking at your most proprietary your

company payroll, your company's secrets,

your your your chemical research. Like,

it's all going right over the wire to

these foundation labs. Is that what you

want? And of course, you know, for

defense and for banking, of course,

that's not what you want. And so now the

ability to bring the model in-house and

fine-tune it with your local data is a

huge is a huge unlock. But the the

higher level point is now the

technological capability to do it

relatively easily is hugely better today

than it was a month ago. So I think

mirror may be on to something here.

We've hit a real tipping point and Alex

Carp I think is right about it too.

>> I think there's two things that also

that that I saw that were really

interesting here. One is a very big

context window like a million tokens

because that means you can do a lot with

it. And the second is multimodality.

>> Yes.

>> And so this is aiming squarely at

organizational use. This fits perfectly

into the onrem proprietary data um model

where you you take your data customize

and fine-tune as you said Dave and that

will be the future. A couple of historic

notes again for for those uh

definitionally uh not tracking the the

full sorted history of fine-tuning. So

fine-tuning is is this notion that you

you start with a model. Model consists

of billions usually these days of

weights of parameters that are frozen.

And if you want to customize the model

for your purposes, you can conduct a

so-called fine-tuning process that

usually makes relatively small, hence

the fine changes to some usually a a

tiny subset of the weights in order to

customize the model for your end

application. That's fine tuning. There's

actually now decent literature out there

that suggests that conventional

finetuning like supervised fine-tuning

Laura style low rank uh adapter uh one

class of fine-tuning architectures

doesn't result in increasing the

capabilities of your model at all. And

at most it it results in like a style

transfer like you could fine-tune a

language model to only speak in

Shakespearean verse for example that's

not really increasing its capabilities

>> or only be an accelerando flavor output.

Well, uh, no comment. Uh, but but I I I

I would say historically fine-tuning

didn't have a history of increasing

capabilities. Then along came

reinforcement fine-tuning where for the

first time via large amounts of

synthetic data uh and giving access to

all of the weights and and not just like

a subset that's convenient to train. we

gained the ability and you know

fine-tuning post- training there there's

a there's a gray area between you know

what what's the distinction between them

but with reinforcement fine-tuning RFT

uh and the the release of the first

generation of reasoning models we saw

fine-tuning actually start to increase

the capabilities of the models now the

problem with thinking machines business

model as as I understand it is it's a

bet on the flavor of the moment that

reinforcement fine-tuning is going to be

a paradigm in the future right now

obviously the paradigm of the moment

that you could take an off-the-shelf

model and RFT your way to customization

with proprietary data and proprietary

environments and other things that that

seems to work pretty well at the moment.

But in some sense, if that is like the

permanent long-term plan of thinking

machines, it's fundamentally a bet that

we're not going to ever move beyond the

reinforcement fine-tuning paradigm,

which I think is probably wrong. I I

think probably RFT is the scaling of the

moment, but in the future, I can totally

imagine a generalistbased model that is

just so generally capable that it

doesn't actually benefit from any

further reinforcement finetuning on any

internal data sets and we tend towards

ASI. Let me bring up another key point

here on this story which is uh in the in

the context which is it's great to see a

woman CEO in the AI frontier lab area. I

think women are distinctly missing from

the entire AI industry, right? We have

Lisa Sue from AMD, but very few in

leadership positions. And I I think

that's an important point. I'm not sure

who else you know, Alex, are you seeing

>> Daniela Roose right where

Fe is also

>> and Fay Lee. Yeah. But again, we're

talking about what singledigit percent

of the AI industry is is women. Uh and

we need more. So, a call out to every

all the women out there, please jump

into this industry. We need uh

>> we we we need more balanced thinking.

>> Yeah, for sure. I mean, I I I do think

that's an important point to pull out

here.

>> This episode is brought to you by

Blitzy, autonomous software development

with infinite code context. Blitzy uses

thousands of specialized AI agents that

think for hours to understand

enterprisecale code bases with millions

of lines of code. Engineers start every

development sprint with the Blitzy

platform, bringing in their development

requirements. The Blitzy platform

provides a plan, then generates and

pre-ompiles code for each task. Blitzy

delivers 80% or more of the development

work autonomously while providing a

guide for the final 20% of human

development work required to complete

the sprint. Enterprises are achieving a

5x engineering velocity increase when

incorporating Blitzy as their preIDE

development tool, pairing it with their

coding co-pilot of choice to bring an AI

native SDLC into their org. Ready to 5x

your engineering velocity? Visit

blitzy.com to schedule a demo and start

building with Blitzy today.

>> All right. Um let's move on to our our

next story here. Uh

it is uh a a fun one. Uh Alex, I was

walking in the streets of uh where was I

yesterday? Zurich. And I saw this come

up and I said, "Hey, let's talk about

this tomorrow." And and you said yes. Uh

so here is the uh the story we've talked

about the holy grail of AI is recursive

of self-improvement. It's sort of like

the holy grail of the launch industry

was reusable rockets. Um this you know

RSI is a holy grail for AI. It's the

idea that AI makes itself smarter. Uh

and then you use that smarter AI to

create the next generation of AI. It's

sort of the theoretical engine behind

the hard takeoff scenario of the

singularity. So this week, a startup

called Wo AI uh with researcher Zeng Yao

Jang uh published what they call

experimental evidence for the first

recursive self-improvement. Whether

they're first or not, Alex, I'll ask you

about that. They built a system called

AIdriven exploration squared, aid

squared, with an outer AI agent whose

job is to rewrite the code and the

research strategy for an inner AI agent.

In their experiment, they claim that 8

days of machine self-improvement beat

two years of expert human effort. So,

Alex, what do you make about this? Is it

the first uh is it significant?

Very significant. Highly unlikely that

this is anywhere close to first. So, so

a few bits of additional context. One,

uh this is actually this WICO is a

startup that's based in London.

Interestingly, it's not based in the US,

but still western sphere. So, great. Uh

so this is a startup built by a bunch of

as I understand it uh UC London grads.

Secondly a few points that I love about

this story. One it's an example of

defensive co-scaling which so to to the

extent we talk about alignment AI

alignment on the pod and I'm I'm always

banging the the drum of defensive

co-scaling as the ultimate alignment

strategy.

>> What does that mean? So defensive

co-scaling is the idea uh borrowed by

analogy from human alignment

human-touman alignment that rather than

hoping for call it the great man theory

of alignment that someone somewhere is

going to discover the perfect algorithm

for keeping AI safe instead the solution

for AI safety is AI policing AI in

proportion the way we keep cities safe

is we have police forces police forces

that scale according to some scaling law

in proportion to the population of the

city. So, so we have the good guys and

the bad guys and the the way we keep the

bad guys in check is with making sure

that we have enough good guys to police

them. Same idea with AI. The way we keep

AI aligned with humanity, a key way is

we make sure that we have enough good

AIs policing any bad AIs in terms of raw

capabilities that they defensively

co-scale. So what one of the things I I

love about this uh aid two story is that

the outer loop so so the the way this

recursive self-improvement process

worked was they had an outer loop and an

inner loop. The outer loop was tasked

with the with improving the inner loop.

The inner loop was tasked with improving

software development processes in

general according to some benchmark. The

outer loop discovered and both both

powered by the same underlying AIdriven

exploration process. At least initially

the outer loop and AI discovered that it

was able to achieve and this was an

emergent property better results from

the inner loop by keeping by preventing

the inner loop from cheating and reward

hacking. And so so in some sense the

outer loop is defensively co-scaling

with and policing the inner loop all the

while this is reaching toward greater

and greater capabilities. And I I think

this is also parenthetically an example

of a case you know all of those who

would say okay like we need to pause AI

capabilities and throw all of our

resources to AI alignment until

something preposterous in my mind like

2040 like stop all stop the race to

super intelligence. stop it all. Focus

on focus the next 14 years on alignment

research. It's going to backfire because

every alignment capability, I would

argue, is actually just cap is is

capability, new capability in in sort of

in disguise in a trench coat. Same idea

here.

>> We need stronger white hats to police

the black hats.

>> Yes. But the the beauty Yes, agree with

that. And also the beauty is the the

so-called white hats were emerging

organically uh on their own just from

the outer loop policing the inner loop

towards greater capabilities. That's

first point. Second point quickly the

same startup Wo has published a scale of

recursive self-improvement which is I I

think something the world has been

missing. So we have like for autonomous

cars we have uh the um the society of

automotive engineers has their like five

levels of autonomy for autonomous

vehicles. They've published a scale for

recursive self-improvement that that

goes from zero to three. Zero is

delegation where the AIs are slower than

human R&D. Level one net positive where

the AIs beat human a R&D at the same

cost. Level two they call ignition where

the improvers are better basically a

better improver. and level three

inflection self- acceleration with a

fixed budget. And the claim here is that

they're touching just starting to touch

on ignition. They call it level one

rather than level two. But the claim

here is like this is a pre-ignition

event, which I think is super exciting.

>> So they they rate themselves as a level

one here.

>> Yeah, they rate themselves as level one,

but reading between the lines, they're

like this is like sparks of ignition,

literally and figuratively.

>> Okay, so maybe I can maybe I can jump in

and and say a couple words. I'm not as

excited as Alex is like on the on the

topic and and I see I see this is an

impressive engineering kind of work that

has been done just to tell you a little

bit about like how the foundation model

labs are operating all foundation model

labs since the beginning of let's say

like four years ago or let's say 5 years

ago everybody has been thinking about

recursive self-improvement and for us

the definition of recursive

self-improvement is not the engineering

and prompt engineering of in inner loop

and outer loop to really get get some

code patches like changing because that

gives you the assumption that every

single AI model that you're using in

your pipeline is already like uh uh you

know like it's already defined and it's

already fixed with a certain type of

capabilities which is actually the case

in the whole pipeline that they actually

like design there's no weight changes in

the neural networks so that means like

the AIs that are actually getting used

right now there's no uh kind of

improvement of the core competences and

even behavior of the models they're

always like in the system prompt of the

of the models like changes in the system

problem because I will give you like

fundamental reasons why this is actually

limiting because if you just run the

like how I want to tell you how hard of

a problem is recursive self-improvement

for us recursive self-improvement means

that you have an AI system or an army of

AI systems that they can also like

retune themselves they can you know

adapt very similar to how humans do it.

You know, if you if you think about it,

the core competences of these models

that we have right now, they're they're

they're fixed weight models and and the

capabilities are within a certain kind

of threshold. And the frameworks that

they actually like designed, it's not um

it's a very nice early stage of show

showcasing an engineering pipeline that

can improve work, which is actually very

very important and very nice. But I

wouldn't I wouldn't go so much to say

like this is like the first breakthrough

in in in the entire AI industry or

something like in fact like about three

years ago we published a paper ourselves

like we talked we talked about automatic

design of model architectures you know

like you know as liquid AI we didn't

want to put like a bet on a single

architecture we have basically designed

self-improve like meta AI systems that

are actually defining their

architectures and then going through

scaling laws for various types of

architectures and then trying to figure

it out based on the criteria that you

define what should be the final model

and then right now at our company all

the process of training foundation

models and really like retuning the

weights of the system are are getting

automated. So we are talking about AIS

or designing AIS. So that's that's what

I what I would be like calling it like

the holy grail where you can actually do

automatic kind of tuning of a model. And

I'll tell you with the frameworks that

they kind of uh uh structured, it would

be extremely exhausted computationally

intractable to actually performing this

this job training an AI model training

like being able to customizing an AI

model and training an AI model on a

meaningful number of tokens for

adaptation or let's say like the core

competence of the model changing core

architecture of the model changing core

algorith learning algorithm itself

changing all of those matters adds more

and more complexity on the on the

situation. I can give you also like one

numerical kind of example of this.

There's a scaling laws called chinchilla

law. You know like chinchilla is like

the scaling laws of neural networks and

know like it is unproven like we have

actually unproven but still like it's it

gives you a good sense. It says when

you're training a neural network let's

say of a given size. If the size of the

model is two billion parameters, you

need 20 times of more tokens number of

tokens to train these models so that you

have you have compute optimality given a

compute budget. How many tokens do you

have to train a model so that you have

like a general purpose kind of system?

So that ratio is like 20. And then when

you actually do the math with the

frameworks that they have, if they want

to like let's say you launch this

framework on retuning an AI model to

recursively self-improve with this uh

framework that is getting introduced, it

takes us 350 years to really uh uh

fine-tune a two billion parameter model

with this framework. So there are so so

so there's there's a lot of there's a

lot of u computational complexity goes

into nested learning systems nest metal

learning systems you know like these are

the kind of problems that the last four

years of like at least at my company

like we have been heavily focused on and

I know friends at openai and entropic

has been like focusing on this recursive

self-improvement and entropic has been

having a lead on all of these things

because they thought about this before

every everybody else that's that's what

can put out there. Dave,

>> yeah, brilliantly said. And actually,

just just so the audience can get the

analogy there, when a baby is born and

then learns, you know, that happens over

about a 20 year time scale and after 20

years, you've got an adult that's

capable. Recursive self-improvement is

like evolution on top of that where

you're changing the DNA and creating a

new

>> You're changing the neuronal structure

of the brain along the way.

>> Exactly. So, so that happens over, you

know, about a 10 millionyear time scale.

So you go from 10 years to 10 million

years to go from learning to recursive

self-improvement or recursive evolution.

And so the big foundation model labs

like Ramine said are all doing it. It's

the most important moment in human

history. But there's no, you know,

little guy out there that's going to

come up and say, "Hey, I've got a

breakthrough in recursive

self-improvement. My Mac Mini suddenly

became conscious and now it's improving

itself." Just computationally it doesn't

even come close to fitting. So it's

happening, but it's happening with big

compute and big budgets. uh and and you

know there's a lot of room for

efficiency improvement a lot of

breakthroughs will happen but it's not

going to just pop up on some you know

>> you know there's a lot of fe there's a

lot of fear just to call it out that you

know recursive self-improvement leads to

AIS that take off a hard you know we've

discussed the hard takeoff and without

our understanding of that black box um I

I guess the two questions need to be

asked is do is there a concern that

recursive self-improvement once we hit

level two level three by that definition

um runs away in a way that um uh causes

an uncontrolled uh AI that is misaligned

with humans. And the second question I

have is when do you think we'll see

this? When do you think we'll actually

see recursive self-improvement hit? Is

ASI going to be that point uh or is it

post AGI whatever that means? See, I say

that for you.

>> I'll let you answer that exist. I've got

I've got several comments though but

Ramine go ahead what do you think is

happening

>> look the thing is I can tell you like

the early evidence of recursive self by

the way recursive self-improvement is

not related to one single agent it's a

social kind of character as well you can

imagine like you know you have societies

of agents so this defining kind of

structure for society of agents itself

self-improving these are the places

where actually mythos level kind of

class of models like I hate this analogy

but still like let's say mythos level

kind of class because everybody body

like heard about mythos and then what I

would say is that like the cyber

security kind of uh um um uh threads

that we are seeing like coming out of

these type of pipelines of recursive

self-improvement

they're real you know like like the

reason why I'm actually I I've always

been like you know like pro open source

and I want to open source technology all

the time like we are doing it all the

time like every single release of our

models is open source our science has

been always open source I believe

science has to be open source and I I

see the value of open source going

forward. But some of these concerns that

uh Peter you you brought up, they're

they're very real, you know, like the

cyber security kind of aspect of things.

That's why I feel like like a degree of

at least uh enterprises themselves

having some degree of kind of

self-control like about like how before

mass release of their uh their models

there there has to be always a certain

degree of selfch check and I think

entropic took it very seriously. The

reason behind is because they're seeing

the impact of recursive

self-improvement. So I know I know this

for for a fact because I I know what is

happening like in in seeing it at a

smaller scale. You know, you can do

reward hacking, but you can also like,

you know, like avoid reward reward

hacking like to to the certain extreme

and push a model to actually discover

some stuff that you know like are are

out of norm, you know, and and we we see

that on a small models like at a at a

certain capabilities certain

capabilities emerging and then I can

only imagine like what kind of

capabilities could emerge from let's say

larger and larger systems thrown more

and more compute at them. When do you

when do you think we you know when do we

have a pod

>> timelines remain timelines?

>> Yes. When do you have a pod that said

yes this is recursive self-improvement

because while you know while the data

released by WICO is interesting uh it's

their own self-reported data. It hasn't

been confirmed by anybody else yet and

you know there is a you know debate

about whether it really is or is not

real recursive self-improvement. When do

you think we actually, you know, you

give the trophy out to somebody? Is it a

year, 3 years, 5 years?

>> Yeah. I mean, I I'm telling you that

that so I I would say like you're going

to see like unbelievably kind of models

like probably in the next 2 years or so,

you know, like models that are like

going above our our understanding even

like that that's that's what what I

would imagine to get. The reason behind

it is because the time to developing the

next generation of the models is

reducing especially if the compute grows

like at foundation model companies like

with the rate that we are seeing right

now and if there is no like let's say

another chip shortage or memory shortage

like on compute or anything like around

the globe and they have access to

abundant compute we are going to see

those things like happening faster and

faster. Now in terms of model

development there's a concept that we

have we call it depths of customization.

So everything at a foundation model lab

when you're customizing a model when

you're building something that is like

better than its previous generation we

always categorize it with depths of

customization. The place where recursive

self-improvement today is really good at

is prompt engineering changing editing

code like in engineering kind of tasks

that you've seen like some elements of

these things like at a very very

superficial level let's say make my

model run fastest like doing kernel

engineering basically you know make my

model run faster that's what I call like

the shallowest level of kind of

customization where you have Python code

and then you're kind of adopting that

Python code to really run or maybe like

even lower level programs that you have

like on a kernel level to optimize like

let's say inference speed you know

that's something that I think with when

when they released Fable 5 they they

shared like and tropic actually shared

that this was one of the tests that they

have been performing you know but they

they don't share like the next level

depths of customization the next level

depths of customization is that can a

model fine-tune a small language model

to a production grade capability or a

smaller version of itself to a certain

capability

today like fav 5 can actually you can

push it to actually get to some degree

of kind of customization with some

>> performance optim performance

optimization

>> performance optimization of the model

but by fine-tuning then the latest holy

grail which is like the craziest one

which would be pre-training right can a

language model pre-train the next

generation of their own that's why they

hired karpathy because Andre was talking

about like nano GPT style kind of uh

fine-tuning you know andre like joined

entropic and now he's working on

pre-training automation like basically

automation of automation. So which is

which is a very very important kind of

element that we don't have yet because

the scale of these problems goes beyond

human imagination in terms of the scale

of compute that

>> you're jumping

>> I've got I've got for me this is by far

the most important uh story or slide

we're going to cover today. Um I'm

beyond excited for a couple of reasons.

uh the you know I I'm not really focused

on the self-awareness or the loop that

will go there but this is self

accelerating it's accelerating

experimentation right because the system

doesn't need it's it's improving the

process by which it searches and

evaluates and selects improvements and

the innovation loop begins to compound

that for me is the key why because this

this whole thing we've been doing called

the organizational singularity relies on

one thing which is can you get to

recursive self-improvement at the

workflow level. Here we're talking about

the model and we're talking about like

can you so but you don't need that

level. The bar can be much much lower to

improve invoice uh uh approval at a

company right that's a very low bar to

improve that process. So this is the

first glimpse of the organizational

singularity. It's happening at the

research level, but the because AI is

not just doing tasks in a in a workflow.

It's redesigning the workflow uh that

makes it better for doing future tasks,

right? And so this is proof now for the

whole thesis we've had. Um we predicted

this, but it's great to see it actually

happen because now I can kind of tick

that box off and go this is there cuz

now you have meta improvement. And I

think Dave's analogy of the baby

changing the DNA is fantastic. That's

such a great visual around this. What

the hell does it become over time? Um,

really really I'm beyond excited about

this.

>> I've got to move us along. There's a lot

that happened this week. Our next story

here is the Malaysian prime minister,

uh, Anoir Ibrahim has is preparing to

debut an AI generated digital double of

himself trained to sound like him for

public communications and outreach. So,

uh, this is one of the most prominent

cases yet of a sitting head of

government officially adopting an AI

likeness as a communications tool. Uh,

not a deep fake Biden adversary, but a

sanctioned official AI clone of a

national leader. Uh, we've seen this

before, Selene. we've talked about in

the past where Albania in 2025 uh

announced uh Dileia uh an AI avatar that

was formally appointed the minister of

state for artificial intelligence and

following a presidential decree became

the first AI system in the world named

at a cabinet level role. Uh so one

leader uh in this case prime minister of

Malaysia can personally address millions

in their own languages. It's worth

noting that Malaysia has 135 spoken

languages. So, um it's a big deal,

especially in in a nation like that.

Sim, I'm going to go to you first on

this one. Um we've been talking about

this for a while.

>> Yeah, I I met the um the former prime

minister when I was there helping them

open a university. Uh and Anoir Ibrahim

is a really really good guy uh to as a

follow on. Um the there's a risk here.

the risk is that the authenticity kind

of collapses because people need uh you

know you could you could launch a bunch

of deep fakes with this and have a huge

issue. Is this the actual leader? That

kind of question can come up. But I love

the general approach because if you can

do it from a with a watermarking or

something and say this is the actual

avatar, uh then it gives every citizen a

voice to um um plug into and gives huge

props to the civics of all of this

because now you're scaling civic

engagement and I think that's a very

powerful thing to do. It's one of the

biggest challenges we have with

democracies all over the world is civic

engagement and this allows you to scale

that. So I'm very excited.

>> Do you remember the reason why Albania

put this their AI cabinet minister in

place?

>> Yeah. Corruption.

>> Corruption. Exactly. It was to fight

corruption.

>> Yeah.

>> Yeah. Now, Malaysia is pretty decent as

a pretty decent place, but definitely

you you have that issue. But I think the

this is more of a PR thing and more him

trying to figure out ways of connecting

with the ordinary citizenry, which is

all great. I I love the fact that we you

know we had this conversation with the

uh president of Argentina uh you know

going full out here and it's interesting

to see which countries are sort of

experimenting on the edge. Um Alex, do

you want to weigh in?

>> Yeah. So many thoughts here. First I

think we're going to see more of this in

the west as well especially with like

extra high alpha personality leaders

that want to amplify themselves and

touch the the citizenry. AI Trump is

coming is how you're saying

>> high personality leaders that that want

to touch the citiz citizenry and in some

sense I I think it's a generalization of

social media. So social media enables

direct outreach from the leader or the

influencers to everyone but it's sort of

broadcast one to many. It's not

interactive. This generalizes in some

sense social media to make it a lot more

birectional since if you're touching a

million or 100 million or a billion

people it's very difficult to interact

birectionally with everyone all at once.

Now if you create a digital twin of the

leader or the influencer or the

organization now it can be birectional.

So I I also don't think it's just going

to be governments or government leaders

that adopt this. I I think it's likely

that corporations, corporate CEOs will

do this. We already see Zuck and others

creating digital twins of

>> themselves. We had DAR on the Abundance

stage last year. We're discussing this

that uh the employees made a DAR clone

that they could go and practice their

pitches on and get feedback before they

pitch to him.

>> Yes. And it won't just be I think

corporations, religious leaders and

religious institutions. uh if you're

Catholic, imagine having like a digital

twin of the pope and you you see like

lots of religious institutions,

organizations already creating basically

living versions of of their founding

documents and making those interactive.

But I think the biggest twist and we

we've seen variants of this movie before

are going to be in cases where what

start as digital twins of the leads or

the avatars uh of an organization uh or

an uh some sort of like organized

religion actually themselves become the

leader. that that's at at some point the

the digital twin uh it to the extent

it's interfacing much more with the the

the populace uh the the proletariat as

it were of an organization at some point

it's actually the digital twin of the

leader running the company and not the

actual behavioral origin that uh that's

running the company and I think that's

that's one way in which sem to your to

your exo point this is I I think a

potentially a pathway towards not just

uploading individuals like natural

persons or non-human animals but

uploading entire organizations into into

cyerspace into the cloud if we created

digital twins are the leaders and those

are the ones actually running the

organization

>> it could lead to a true democracy Dave

where do you come out on this I mean we

saw just one quick point we saw Sam

Alman talk about in the future if I

believe enough in what we're building

with with chat GPT it should be the CEO

of open AI eventually

Dave are you going to create an AI Dave

Blondon that's going to run Link Studios

and and Link Link Ventures.

>> Absolutely. Going to create an AI Dave

Blondon. And I'm shocked that there

isn't already a Peter Diamandis.

>> Well, there is there is one. It's just

inside the Abundance ecosystem. I mean,

anybody It was funny. I went to uh went

up to Calgary and met with one of my uh

dear friends and abundance member and on

his wall, I kid you not, he had a giant

screen of my AI avatar that he has all

of his tech employees talk to uh to sort

of get their moonshots and it was it

blew my mind.

>> You've got your own big brother, Peter.

>> It was like he goes, I want to introduce

you to someone, Peter. And he spins them

up and I you know, it's interesting to

have a conversation with your AI self.

Um, it is very compelling. I mean, I

have enough books and tweets and uh and

and Substack posts out there that it

does a damn good job. Uh, we should

effectively, you know, moonshots.com is

our our platform we're building out. I

think we should have AI avatars of all

of us there where people can do AMAs.

>> In some in some cases, Peter, I think

that might be redundant.

>> Ah, well, hey, in other words, you're

already an AI, but we can have an AI of

the Alex AI. Sure.

>> It would be so much better than the real

person because we'll have access to

everything we've ever said, all our

memories, all our thinking. The context

will be much broader. Go for it.

>> This whole area

>> Yeah.

>> This whole area is about a year behind

where it should be largely because, you

know, Noam Shazir was doing character AI

and and we had Steve Brown Peter that

was uh two years ago now. We had Steve

Brown make uh the debate between AI

Peter and Sak Aristotle.

>> Yeah.

>> Yeah. And and so it's been a it's been

possible for a while now, but all the

key talent working on it got sucked back

into the big foundation labs. And you

know, there's so many big big big uh you

know core technological breakthroughs

going on that the people that were

working on this just got absorbed back

into those things and not into the the

avatar. But my my mom would always tell

me when I was a kid that John F. Kennedy

beat Richard Nixon in the election

because uh he looked good on TV and TV

was the new medium and the prior medium

was radio and Nixon was still using

radio voice when TV had taken over. So

then, you know, elections go by and

suddenly it's the internet, it's it's

social media, now it's YouTube. But this

is another step function change in the

way that you reach out

>> to people and it's underutilized, but

it's it should be easily dominant two

years from now in the next election. And

so I'd be shocked if because the

technology is already there and people

are visualizing the medium right now as,

oh, let me make an AI version of myself.

I'm Alex Wisner Gross. Here's my AI

version. It's just like the real thing.

That completely misses the point. The

the AI version of it can in real time

access any information and make it

visual, graphs, charts, you know, it can

morph its face. It can it can teleport

through space to make a point and point

to atoms. It can shrink and expand. It

has all these capabilities that the real

human version doesn't have. And that's

why it's going to be so compelling. It's

the differences that make this new

medium so exciting, not the not the

exact clone. And so once people realize

that there's no going back. It's going

to be huge.

>> I think Dave, that's such a great point

that you make, it's the complimentarity

that is very powerful.

>> Let me let me close out on one thing

here. If if uh to our audience here, if

you've not sat down, if if you're lucky

enough to have your mom and dad still

alive or your grandparents still alive

and you haven't sat down and interviewed

them in video uh for hours at a time,

please do that. Right? you're gonna

you're gonna wish you had. So, I've done

that with my mom. I miss doing that with

my dad. And it's the ability for your

kids and your grandkids and your

great-grandkids to really have a great

AI representation of your of your

parentage and your your lineage. I think

that's going to be super important.

Reine, I want to I want to pivot to a

discussion of liquid AI uh and uh uh the

the small language models, what they

are, what they mean. uh super excited

you know uh just for full disclosure uh

you know liquid AI is a company in which

uh Dave you played a important pivotal

role as an early investor Dave you want

to give that backstory here a little bit

>> uh actually I got a call from Daniela

Roose over at CEL saying the best

student I've ever had Daniela Daniela is

you know one of the three I guess big

shot women in AI she runs CEL at MIT AI

lab in the world computer science AI lab

you know I don't know if you remember

back in the day there was the AI lab and

then LCS lab for computer science were

the two biggest

>> you know compsai labs at MIT they merged

them together and made one mega lab put

it in the new STA building which is that

crumpled look looking beautiful

structure uh you know right on the edge

of MIT's campus and then Daniela is

running that entire thing so I think

it's like 1500 researchers in the

building biggest AI lab in the world and

and so she has access to incredible

talent but she called and said, "Hey,

best students I've ever had have this

incredible breakthrough." And then she

completely lost me. She said, "It's

based on the nervous system of the worm,

the C elegance 300 neuron worm." Like,

what are you talking about? But it turns

out that if you, you know, I actually

don't know of any um successful

foundation lab uh that has really

rethought from the ground up the

transformer and thrown it out basically

and started over which which you know

humanity desperately needs because that

the everybody knows the transformer

architecture and the whole attention

mechanism is bloated. And if you really

go back to founding principles and think

again, you might be able to build

something dramatically like massively

better. And so the team went from idea

in a lab to billion dollar valuation in

faster than any company out of MIT in

history.

>> And luckily we were an investor in that

company.

>> Luckily we were. Yeah. And very very

thankful actually. It was very

competitive getting any money in at all.

So Reine uh we owe you a huge debt of

gratitude for for being invited to the

to the party. Um but uh yeah it's it's

uh one of about 200 unicorns out of MIT

all time but the only foundation model

company that I know of that reached

unicorn status coming out of MIT. So

it's a really unique uh and incredible

achievement and in record time too.

>> So remain take it take us from there.

You're you're doing your PhD under

Danielle Larus at the computer science

AI lab CEL and you're studying a 302

neuron uh worm uh C elegance and so take

us from there forward to what's uh what

you're doing now what is liquid AI

>> absolutely absolutely like before I

start like I want to thank you guys like

for for the support throughout like this

three and a half years years of liquidi

you have been like great support giving

us like the the the kind of distrib

contribution that uh a company needs,

you know, like and and at at our scale

like starting off of the east coast.

Thank you so much for doing that both of

you. Um and um and um yeah, so so 2015 I

was in Vienna. I started my PhD with

professor in Vienna, Professor Rad

Grusu. There he had the idea of like we

don't understand a lot about human

intelligence. Let's start on a smaller

animal and then from first principles

like if you understand how the neurons

exchange information in the brain of the

worm. The worm has 302 uh neurons in its

nervous system. It is uh its body is

transparent so you can actually see the

body actually lighting up like so it is

a one of the best model organisms in the

world. It won so far like four Nobel

prizes for humanity like you know

because it has 78% similarity genome

similarity to uh to human genome you

know. And the way nervous systems

compute in the brain of a little worm

which is 2 mm is um basically analog

very similar to how artificial neural

networks are actually computing. They

are also like analog switches like they

have like graded potential. They're not

spiking. So in biological neural

networks usually in the brains you see

neurons a spike and when you have a

spike that's there's an analog to

digital kind of uh transfer of uh uh

things are happening and that's a

natural development of nervous systems

for uh in in the human beings and and

bigger animals for propagation for

efficient propagation of information. In

the brain of the worm neurons behave

very similar to how artificial neural

networks react but then the the

mechanisms are very interesting. So we

wanted to add more complexity into the

neuro like every individual single

blocks of nervous systems and see can we

pack more information into inside the

smaller kind of units of compute you

know and that's what we have done so

Danielle Arus two years into basically

discovery of these things that I was

doing with my co-founder Matias Lechner

Matias was a master student in Vienna

Vienna University of Technology and I

was a PhD student and then when Daniela

heard from Radu that uh you know like

this project is going on. Danila was

like, "Oh my god, this is crazy. We

should apply this in autonomy in

robotics and all the sort of things

because you're showing like uh a handful

of neurons can drive and control

autonomous systems, you know, and can we

scale this to vehicles? Can we scale it

to drones to to jets to like like

predictive kind of places?" So Daniela

came in and said, "Would you guys

consider coming to MIT?" And we we went

there since 2017 in the middle of my

PhD. I actually joined CELL there. We uh

we continued working on the uh on this

technology which was you know like from

a base is a completely different things

a neuroscience inspired the math behind

like every single neuron in a liquid

neural networks that became kind of my

PhD thesis is very different than how

attention works you know these are based

on recurrent neural networks these are b

based on continuous time processes you

know like more and more kind of nature

inspired computation went into the

design of uh uh found design of kind of

AI systems and then we applied these

liquid neural networks as a completely

new base because uh we applied them to

real world scenarios like robotics

because you can pack a lot more

information into smaller kind of

processors in the real world in the

physical world you don't have the luxury

of having abundant compute let's say a

robot doesn't have it doesn't have like

a lot of GPUs or parallel data centers

like attached to it a robot has a CPU

and a small like let's say GPU and let's

say an NPU a custom ASIC. So you can

actually take this type of uh you know

intelligence that we design that deliver

basically intelligence at the level of

like models that are 10 to a thousand

times larger than themselves. You can

bring those things like directly running

on CPUs, GPUs and NPUs outside of data

centers. So we thought that okay this

format is is going to open up an

opportunity for us to bring in like

alternative architecture if we scale

this technology to let's say into into

the regime of foundation models which is

kind of large language models and SLMs

uh as a whole like human understandable

like making this liquid neural networks

or architectures that we have uh also

scalable like the transformer

architecture and uh we built like a

foundation model lab around the idea uh

in 2020 2023

beginning of 2023 I think at the very

beginning when we started there was no

foundation model lab apart from deep

mind and and and open AAI basically like

when we started and this notion of

foundation model labs didn't exist and

everybody was betting on top of uh uh

you know transformer architecture and we

came in and we said okay so why don't we

explore this space of alternative

architectures starting from the priors

that we have from nature and then take

take a different approach build a meta

AI system again uh basically an

automated AI system that allows us an AI

that designs AI that explores the

computational graphs of intelligence

beyond transformer and then figure out

what should be that architectural design

that brings the same level of

intelligence than a frontier model into

let's say on on a CPU that we can run

let's say a physical system

>> take a second and and walk us through so

these are small language models can you

define an SLM M and how it varies from

an LLM.

>> Definitely. So when you start uh

developing kind of foundation models,

you start you you run something called

scaling laws. You know like a scaling

laws is like basically starting with a

smaller models and with these smaller

models you train them on a certain

number of token budget given amount of

compute. You train these models to see

how well they perform. Then you start

systematically making the models larger

and larger. So that and and we have seen

scaling laws shows that the larger you

make the models the more token budgets

you spend the more intelligence of a

system you can get and this has been

like giving rise to large language

models along the way of scaling there

are instant instantiation of the models

which are smaller you know like on the

scaling laws but we have been doing as a

lab our mission has always been building

efficient general purpose AI at every

scale so we started as a foundation

model lab to really run the scaling laws

on on efficiency front you know and

efficiency was a first class citizen for

us you know like thinking about

computational graphs of intelligence

smaller models are models that are you

know like along the line of like a

scaling they can solve um let's say they

don't have like the general capability

to the level of the largest kind of

language models but they can be

specialized to solve dedicated problems

they are general purpose small language

models are general purpose in the sense

that they understand language They can

see and they can hear in a multimodal

kind of format but they don't it doesn't

mean that they can solve let's say a

homework in physics and at the same time

they can solve an enterprise problem you

usually specialize smaller language

models

>> and what does small mean what does small

mean in this case

>> small means like basic I mean now they

come like now small would be like

anything below 100 billion parameters

you know like that's kind of the regime

that I would count I mean midsize like

basically is is around that that size

but I would consider like anything below

100 billion parameter is something that

is not small and mediumsiz kind of

models you know there's no there's no

clear threshold of like let's say what

is the number of parameters but for us

like the notion of ondevice AI is is

extremely important here to distinguish

within this range of parameters ondevice

AI is like models that you can actually

deploy them uh on the on an actual kind

of device physical device this could be

a

>> let's make this concrete cuz you've got

a significant deal with Mercedes.

>> Yes.

>> Um and can you speak to that and let's

talk about you know these these SLMs in

terms of uh onrem uh basically they're

they're and energy efficient uh you know

fast offline. Let's let's dive into that

give people sort of a real understanding

here.

>> Absolutely. So as I mentioned you can

specialize these foundation models. We

work with a lot of enterprises that are

building devices themselves. Like

automotive is a device is a is a is a is

an environment where you have a lot of

chips in there and now in a car you

don't have that much that much compute.

So there's like one chip that is

available for infotainment and incar

intelligence you know that chip is very

very small. The Qualcomm chip or let's

say Samsung chip like depending on like

what company is providing the chip like

that chip is like very very small. We

are talking about 2 GB to 8 GB of RAM,

you know, like not more than that. So

the model has to be very small and at

the same time being able to perform

because we want to bring this and enable

a private space inside the car that

powers the intelligence of the car in

the car. Car is a safety critical

environment. You don't want your car to

be driven by an AI model that is sitting

in the cloud. Why? Because connectivity

is not uh always available, right? then

uh it is it is private because it's one

of those spaces that people spend a lot

of time in and you don't want those

conversations to be like recorded. So we

brought the intelligence like um

basically we brought u uh one of our

multimodal foundation models that is

only uh less than one gigabyte of in

size

>> and it can go inside the car's chip like

very very tiny chip. The chip could be

as cheap as $60, you know, like that's

what I'm saying. Like we're bringing

that level of intelligence into that

that voice and it is going to power kind

of the multimodal intelligence

experience inside the car. We do that

with all car manufacturers. We announced

the Mercedes partnership as a first uh f

first kind of uh point of entry because

automotive is like it's very sensitive

kind of uh topic and and they're they're

pretty slow. One of the things that

Mercedes dispense actually enjoyed from

this process was the speed of operations

that we had for enterprises you know

like when we are bringing this type of

technology inhouse this has been like

one of those uh cornerstones of landing

the deals you know because we want to

work we are an enterprise company we're

a B2B company we are bringing our full

power to really like deploy the

solutions and really have platforms that

allows people to fine-tune like their

small models and fine-tuning small

models is not that expensive. It's

something that is extremely tangible. So

they we fine-tune kind of the small

models for the the applications inside

the car. We also have data flywheel kind

of systems that allows the system always

stay adaptable. Imagine some of the some

of the problems in enterprise AI has

always been let's download a GLM 2 5.2

like you know and and let's say an open

source model and put that in production

and then so what happens after you put

the system in production? What happens

like when there's a drift from the use

cases that is hitting this model inside

let's say a car and in the physical

world it becomes even more challenging

because when you deploy an intelligence

that is completely kind of disconnected

from the cloud how do you want to like

maintain updates of the system because

we have always thought about like

intelligence in the format of liquid you

know like intelligence has to always

stay adaptable and um and that that's

that's kind of a portion that we're also

pushing on to really be able to collect

the data and personalize models to the

experience of every single user. With

Mercedes, we're rolling this out first

in North America uh as as soon as

basically this year. All the

Mercedes-Benz North America cars like

from 2022 on they're going to get an

update overtheair update because the

size of the update is 600 megabyte. So

that's that's like a that's like a

overlay of like it doesn't consume that

much internet to really update your

software and that allows us to also

further customization. Imagine if every

update that you want to perform on the

system is in the order of 20 megabytes

because we are doing like some sort of

lore adapters and let's say all sort of

adapters that we can actually bring in

inside the car. You would be able to

have like a recursively kind of

improving the experience of the user as

well. So that's kind of let's take it

make it more concrete for me. So what am

I going to be how am I using this model

in my Mercedes next year? So right now

my experience is using Grock in my

Tesla, right? And it's over the air. If

I don't have connectivity, I don't have

Grock. Uh but you know what kind of what

kind of queries what kind of

capabilities does this all of a sudden

enable in a Mercedes?

>> It has access it it's it's sitting below

the the the operating system. So that

means like it is basically it is like

basically have access to all the

functions inside the car you know so

there are 700 functions inside the car

700 to like I don't know 1,200 depending

on what what you count as a function you

can you can talk to your car you can

control like all the panels of your car

you can ask for let's say manuals of the

car you know like when when you're like

get let's say stuck somewhere you know

like something pops up you know like you

would be able to talk to the car there

are memory features that we are adding

to the car like You basically can have

conversations with that with that

system. Once the like one of the

beauties of this system is that like it

has full access to the to all the

functionalities of the car plus all the

apps because there are like function

calls. They're one function calls away,

you know. So if you want to control any

other thing from this from this

intelligence unit inside the car, you

would be controlling everything all the

ecosystem that is sitting on top of the

uh um sitting on top of the operating

system of the car.

So basically I mean if I get you right

there the advantage of the SLMs are

first of all you know the size of the

model I I assume energy consumption

they're efficient um and they can run on

on prem uh do I mean how do you avoid or

reduce sort of overgeneralization of

these models compared to LLMs?

>> What do you mean overgeneralization? In

other words, uh are the do you have

enough capabilities internal to them so

that they are uh actually able to

accurately answer the questions you're

asking?

>> Great question. So if you have like you

know I I told you about the framework of

foundation model development which is

depths of customization.

>> We try to actually stay adaptable and

have access to the tools across these

customization stacks. Sometimes prompt

engineering is enough. Sometimes you got

to fine-tune the model. Sometimes you

have to do go and pre-train a model

again you know for the core capabilities

or a specialization of intelligence. Now

we make systems that are you know our

platforms are getting into the place

where they're automatically identifying

what depths of customization is needed

for a certain solution and the platform

basically like it's it's one of the

products of the company that we sell to

enterprises to allow them to fine-tune

kind of models like I I don't want to

call it fine tune customize a model at a

level that is needed for that uh uh for

that system to actually operate right so

for Mercedes-Benz we have a let's say

like the framework that we have at at

in-house. We call it model plus X, you

know, model plus a platform that allows

you to perform customization. It's not

just the models that we're selling to

enterprises, the static weights of a

model. We sell them something that they

can actually like retune and fine-tune

the system. Detecting how how much uh

generality like the base models have,

it's something that you know like

libraries of liquid models are coming

out for many different applications. We

have models that we're working with for

example in silicon medicine like you

know Alex

>> I introduced you Alex

>> that you introduced us Peter like I

remember and um and through that kind of

interaction like it is getting big you

know because they discovered that liquid

foundation models are actually pretty

good getting customized for a certain no

they're they're basically like really

really well orable so and and and that's

something that they they they figured

out that it comes handy for them. So now

we have a state-of-the-art biotech

foundation models like longevity

foundation models like these are the

kind of things that we're building in

bio and imagine like as a horizontal

company that is building foundation

models we went to like fine-tuning and

that became like something that we have

we have managed to do and then in terms

of um you know like some of the uh uh

some of the other engagements like we

recently with with Shopify we entered

like uh one uh 1 billion kind of request

address inside the Shopify kind of

framework And uh there like what we've

done we uh we deploy our liquid

foundation models in production. They

have been in production for the last 6

months and they are really serving

clients you know and and Shopify is like

a huge uh uh base like we are touching

100 million kind of hundreds of millions

of kind of users 10 billion products and

many different kind of uh places to to

integrate. We are working with

Mercedes-Benz as I mentioned like on the

car kind of side of things. We're

working with AMD and uh other chip

manufacturers to really bring AI uh

let's say um low code AI experiences on

PCs as well. So that's like another uh

area that we enter. The focus of our

company is to really uh make sure that

we can bring uh basically intelligence

outside of data centers. That's like

something that we have focused on and I

think our efficiency is actually

allowing us to get

>> preliminary matter. I have no financial

interest in liquid. Sorry Reine have to

ask the the most obvious question. I

have so many questions for you which is

the company liquid was founded as I

understand it and I I remember reading

the original I think it was in science

or nature paper on liquid neural

networks. Uh the premise is basically a

neuromorphic premise that that you could

gain useful AI insights from looking at

nematodes uh a few hundred neurons sort

of the ultimate small neural network.

But my perception I I'm hoping that you

can uh either help me amend or revise my

perception is that although liquid

started with a neuromorphic premise if

you will like a post transformer very

recurrentoriented architectural premise

or prior that over time again just based

on my perception of public messaging

liquid looks more and more like either

transformer or transformer plus or

transformer plus hyena plus dot dot

looks more and more like basically a

conventional off-the-shelf architecture.

It may be a good business selling sort

of customized transformer derivatives to

Mercedes at all. If so, great from the

business side. But from the technical

side, does Liquid still have anything

that looks remotely like a trans a post

transformer architecture either in

production or under development? And can

you speak to what if anything is post

transformer or non-transformer oriented

about the architecture that you

currently use? Great great question. So

let me tell you like the space of kind

of architecture. So liquid neural

networks in in the original form they

are one of the most expressive formats

of computes that you can actually create

arguably like in terms of our

architecture they are they have nested

non nonlinearities that are like in like

you cannot really like take them out.

They're like completely physics

inspired. They are like having like the

neural odes and and basically like

irregularly sampled data can be handled

by them. So they they become like one of

the very very general class of

architectures as a whole. Underneath

these things like when you want to scale

this type of technology these

recurrences like you know like this uh

nested kind of loops that they have. If

you want to scale these systems a lot of

people have attempted including

ourselves to linearize the dynamics so

that you can actually like scale them.

space uh uh you know state space models

are kind of basically like mumbas and

those kind of variants falling into the

same category of continuous time neural

networks but dumped down into a linear

kind of dynamical systems because you

you want to scale them. They are

underneath this class of continuous time

models that we have. Then there is like

there are variants of uh linear linear

attention gated linear attentions that

are coming out. They are also like

gating mechanism is something like

there's a special gating input dependent

gating mechanism that actually we got

inspired by the by by how neurons

actually exchange information with each

other. That gating mechanism is also