22. Emergence and Complexity

Stanford University.

OK, so we will pick up on the one topic that was not

covered from two days ago because you guys needed

to go play around with these cellular automata first.

So I will work with the assumption

that everybody here has now spent

48 hours playing with those.

But presumably because of the sleep deprivation,

you've forgotten much of it by now.

So we will cover some of it.

OK, back to that issue of fractals and butterfly

effects and that whole business that, by the time

you look at chaotic systems that are

determinist but aperiodical, that, when

they seem to be lines crossing, getting back

into the same spot, look closely enough

and they're not going to actually be touching.

And the centerpiece of why that matters

was that whole business of, these

both appear to be the same.

And take them out a decimal place

and they're actually different.

And a gazillion decimal places out.

And the entire sort of rationale for thinking in that way

is the notion that a very small difference here

can make a difference one step to the left.

And a million decimal places out there, a small difference,

will make a difference one before that.

In a scale-free way, a fractal, this here, a difference here,

a million decimal places out is just as likely

to have consequences for one over as this one for one

over fractal, scale free, all of that.

But the critical thing that is encompassed in this

is the notion that tiny little differences

can have consequences that magnify and magnify

and amplify into a butterfly effect.

So cellular automata are a great way

of seeing this principle along with a number of others that

are relevant to all of this.

OK, so we start off with the very first one.

And this is the one that you no doubt first

discovered is a pattern, which made you deeply happy.

And if you follow the rules, it was

starting there-- was this-- which way is this facing,

starting at the bottom.

OK, this is starting at the bottom.

And what you see is these very simple rules.

And out of it emerges a whole complex pattern.

And we'll be seeing shortly the features

of this that perfectly match what the requirements are

for emergent complexity.

But we'll see as the elements are lots of constituents,

lots of building blocks.

The building blocks being very simple.

They're binary.

Either they are filled or not filled.

Extremely simple rules as to how the next generation gets

formed.

And in terms of the extremely simple rules, none of the rules

have anything to do with other than the next generation.

It is all local rules built around what the neighborhood is

like for each one of these.

So you put it together, and out come

these very structured patterns like these.

And this is great.

This is very exciting.

Except this isn't what you usually get.

In most of these cellular automata systems, where

you start off with an initial condition and a simple set

of local neighbor rules for how you get reproduction

into the next generations, in most cases

the patterns stop after a while.

In the vast majority, they stop, they

hit a wall, they go extinct.

Aha.

Two terms that I've already stuck

in here that are biological metaphors

start to seem less metaphorical after a while.

First off, the notion that going from here to here to here

to here represents each next generation.

And the notion that, as we were just now,

that the vast majority of these cellular automata systems

go extinct.

They fail after a while.

So it's a very small subset.

What you then also see is-- in some ways

the critical point in this whole business--

is the relatively small number of starting states

that succeed produce a remarkably small number

of mature states that all look very similar to each other.

In other words, you can start with a whole bunch

of different conditions, and you will wind up

with a bunch of smaller number of stereotypical patterns.

Half of the cellular automata that

wind up taking off look something

like this with this pattern.

What are we seeing?

Convergence.

Convergence.

The notion that you can start with different forms and they

will converge over time.

What's this?

Well, you just proved you look at the mature form

and you can't know the starting state.

The other thing is, starting at the beginning just looking

at this line, there is no way you

can tell what it's going to look like 20 generations from now.

You've got to march through it.

In other words, the starting state

gives you no predictive power about the mature state.

This is a nonlinear system.

The cellular automata encapsulates

this, this business that most of these go extinct.

Only a relatively few number of mature forms exist.

It shows convergence.

Very different starting states can converge

into the same sort of patterns.

And minor differences in the starting state

can extend into very different consequences.

It shows, in other words, butterfly effects.

OK, so appreciating this a bit.

So what we did was then go to example number

two, where we changed the starting state

just a little bit here.

We shifted around some of the boxes.

And what you see is something that looks roughly the same,

but it's not exactly the same.

But it's the same general feel to it.

So that's great.

But then we started an exercise of starting off

with the initial boxes.

This goes that way.

The initial boxes evenly spaced with one space between them

and applied the rules from there.

And this is what you get.

Totally boring, static, inorganic, inanimate.

This is what it does for the rest of time.

What this exercise then did, going to number four,

is what if we now spaced two boxes

between each one of these?

And here we have an extinction.

This is one of those where it hits a wall,

and all the next lines are empty.

OK, how about three boxes in between the starting states?

OK, another form of extinction.

OK, how about four boxes between the starting states?

And suddenly, something very dynamic takes off.

Applying the same rules, and all you've done

is change the spacing between the starting states.

And look at, for one thing, how close this was to going extinct

up there on top, how asymmetrical the pattern is

that comes out.

And this particular one will stay asymmetrical forever.

And the ways in which that generated

something very unexpected.

There is no way you could sit there a priori

and say, hm, one box in between generates something that

looks inanimate.

Two boxes, not going to work.

Three boxes, yeah.

Somewhere around four boxes in between.

That's when dynamic systems suddenly take off.

There is no way to have known that

before without marching through this and actually seeing.

Starting state tells you nothing about the mature state.

Then we space it even further.

And what we get is something similar again.

This one is symmetrical.

It is somewhat different from the previous one,

but it's the same sorts of patterns

that come up over and over.

So what we've seen here is, by starting state,

minor differences, big divergence

between going extinct versus being a viable pattern.

Minor differences in starting state, big divergence

between symmetrical and asymmetrical patterns.

Tiny differences, butterfly effects.

OK, next.

Looking at the consequences here of introducing some asymmetry

from the very beginning.

The one on the left up on top has four boxes and four boxes.

It has eight boxes.

The one on top has eight boxes on the left, and the one on top

on the right, just adding in one extra box on the side,

so it's four and five.

Adding a little asymmetry, and what you see

is a very different pattern.

And one of the things you tend to see

in these pseudo-animate living pattern systems is starting

states of asymmetry produce more dynamic systems,

more dynamic patterns than even symmetrical ones.

That's one of the only rules that comes out of there.

So we're seeing now minor little differences producing

major different consequences.

Divergencies, butterfly effects.

Now showing this in a different way.

And what we've got here are four different starting state

conditions.

The one on the far left is, in fact,

the one from the previous one, the four and four.

Four different starting state conditions

where they're not enormously related to each other.

The first one against the other three, but the other three

have minor differences.

And the whole thing is, two of these

are identical after the first 20 generations or so.

This one and this one.

The two of them are identical, and for the rest

of the universe they will produce

the same identical pattern.

And looking at the mature state, you show up on the scene

somewhere halfway down, and you could never ever know

what the starting state was.

Did it start like this or did it start like this?

A convergency here.

And in this case, it's another one of those rules.

Knowing the starting state doesn't allow

you to predict the mature form.

Knowing the mature form, you don't

know which particular starting state brought it about.

And the only way to figure it out

is to stepwise go through the whole process

because you can't just iterate by a blueprint.

There is no blueprint.

Finally, the last one was giving you,

instead of different starting boxes

in each case with the same reproduction rule,

the last one was the same starting pattern

of boxes with different, slightly

different, reproductive rules.

And what you see here are totally different outcomes,

depending on which variant.

We have the beloved one on the top left.

And you see here, by slightly changing the nearest neighbor

rules, if and only if there is one

neighbor with this property, if and only

if there's two neighbors.

And working through that way, and you

see remarkably divergent outcomes for one thing.

You see that the majority of them

produce something very boring, either boring extinct

or boring repetitive in a very undynamic way.

Only a small subset produce lively, animated,

living systems.

So we're seeing a whole bunch of biological metaphors

here over and over and over, which is the starting states.

You don't know the mature, the mature,

you don't know the starting state.

The generations, very simple rules for generations,

going one to the next.

What we see also is the vast majority

go extinct and produce, either go extinct

or some repetitive, very boring, crystallized type structure.

A small subset, a tiny subset, produce,

instead, dynamic patterns.

And knowing what the starting state is is not

going to give you any predictability whatsoever of,

is this going to produce a dynamic pattern or not?

Nor does it allow you to look at a bunch of the starting states

and say, those two are going to produce

the same mature pattern.

And these are all properties of the evolution

of different living systems.

So you begin to see, OK, cellular automata.

Do you see some of these principles?

The simplest level out there in the natural world

is looking at all sorts of shells, seashells and tortoise

shells by the seashore and whatevers.

And they all have patterning on them

that is derived from that first cellular automata

rule, producing patterns that look a whole lot like these.

And go online and look for them because I didn't get around

to it in time, but producing all sorts of patterns in nature.

Very common ones.

What does that tell you?

Very simple rules for generating the same complex patterns

and different starting states, cellular automata

and properties here.

Another thing in a living biological system

that begins to suggest this.

OK, so I do this research in East Africa.

And every now and then over the years

I've gone to this mountain called Mount Kenya, which

is on the equator.

It's about 17,000 feet.

It's got glaciers up on top.

So this is an equatorial glacial mountain.

And you go up to about the 15,000-foot zone,

and there's like-- it's more land.

Almost everything is dead up there from the cold.

And there's basically only, like,

four or five different types of plants up there.

Oh, already a very small number that survive

in that environment.

And each one of them is very bizarre and distinctive

looking.

There's one of them that looks like

a little, like, rosebud thing, except it's

about 5 feet across.

And then there's another one that

has sort of a sprouty thing like this and then

a big central cactus-looking thing that isn't really cactus.

So there's a few of these really distinctive,

bizarre-looking plants.

And in some way or other, that's what

it takes to survive up there.

So I have this friend who does research up in the Andes.

And he does botany stuff up there.

And he goes into this one range there that is on the equator

and high enough that there's glaciers up there.

Ah, a glacial equatorial mountain on the other side

of the globe.

So one day I'm sitting around and looking

at some of his pictures there.

And suddenly I look and say, that's the exact same plant.

That's the big rosebud plant that is in Mount Kenya.

And oh, my god, that's the tall sprouty one.

And say, it's the exact same plant.

How can that plant be over there?

And we go rummage around in his botany taxonomy stuff,

and they are completely unrelated plants.

They are taxonomically of no connection whatsoever.

But what they've done is converged onto the same shape.

And in some mysterious way, if you're

going to be a plant growing on the equator at about

15,000 feet, there's only about four or five different ways

of appearing.

There is massive convergence.

And there's only four or five ways

that you can survive an environment like that.

You get organisms in very dry environments,

and there's, like, only four or five ways

that you can go about being an organism that's super

efficient at retaining water.

And those are the only ones you see amongst them.

Desert animals, and completely unrelated ones,

have converged onto some of the same solutions.

There's only a very finite number

of ways to do legs and locomotion.

Two is good, four, weirdo things that fly of six,

creepy things have eight.

You don't find seven.

You don't find three.

You find some of the solutions here are immensely different

starting states and have converged.

What we see is in these living systems over and over stuff

that look like cellular automata,

where slight differences magnify enormously butterfly effects,

where you are modeling living systems in a very real way.

Most of them go extinct.

Divergence, convergence.

And where each one of these you get the smaller number,

reflecting the fact that there's only a limited number of ways

of doing rain forest, temperate zone rain forest

in the Pacific Northwest.

There's only a limited number of ways

of doing tundra in Miami Beach.

There's only a limited number of ways.

In all of these, this convergence,

and always reflecting that these are cellular automata.

OK.

So hopefully you are now feeling desperately regretful

that you didn't spend the last few days doing this

because these are so heartwarming.

If you want to read a book that nobody in their right mind

should read, it's a book by this guy named Steve Wolfram, who's

one of the gods of computers and math

and was one of the sort of people who first developed

cellular automata.

And by all reports probably one of the largest

egos on the planet.

And he published a book, self-published it,

a few years ago, which he can because he

is grotesquely wealthy, from some of his computer programs.

And just showing what a low-key sort of humble guy

he is, he called the book A New Kind of Science,

just showing that he wasn't just going

from some little piddly new way of viewing the world,

but here was his new type of science.

And the book is about 1,200 pages.

And I suspect not even his mother

has read the thing it is so impenetrable.

And it sold a gazillion copies.

And almost all of them are sitting in people's garages

now, weighting down drain pipes because no one can actually

read this thing.

But an awful lot of what the book is about

are patterns in nature coded for by very simple local rules.

And the simple fact of that, you've

got a lot of very smart people doing this cellular automata

stuff.

And they can't come up with rules

where you could look at something beforehand a priori

and know this one is going to survive,

this one is going to go extinct, those two

are going to turn the same, these two

that differ by a slight smidgen are

going to turn out to be enormously different.

There's no rules for it.

And the book has all sorts of cool pictures

of a cellular automata looking things out in nature.

And go buy it for somebody's birthday

and see if they're not grateful for the rest of their lives.

But his whole argument there is, these

show ways in which you can code for a lot of the complexity

in the natural world with small numbers of simple rules.

This whole business of emergence.

This sets us up now for beginning

to look at some of the ways in which we

hit a wall the other day, ways in which the reductive model

of understanding the universe stops working after a while.

One version being the problem of not having

enough numbers of things.

Not having enough neurons to do grandmother

neurons beyond Jennifer Aniston, that whole business

that you simply don't have enough

neurons to do that beyond just the rare ones now and then.

And what do you get instead?

What has the solution turned out to be?

This field that people focus on now called neural networks.

And the point of neural networks is that information, again,

is not coded in a single molecule, single synapse,

single neuron, one neuron.

This neuron knows one thing and one thing only,

which is when there's a dot there, instead,

information is coded in networks,

in patterns of neural activation.

And just to give you an example, and this is

one that's in the Zebra book.

And anyone who is in Biocore I do this one,

and I do it because I at one point

learned the name of three impressionist painters,

except they're not coming to mind right now.

OK, so you've got two layers.

Here's what a neural network would look like.

A two-layer one.

These neurons on the bottom are boring, simple, Hubel and

Wiesel-type neurons from the other day,

where each neuron knows one thing and one thing only.

This one knows how to recognize Gauguin paintings,

this one recognizes Van Gogh, and this one Monet.

OK.

Each one of them is-- obviously there is no Hubel and Wiesel

neuron on Earth that's like that.

But just for our purposes.

They now project up to this next layer.

Note this neuron projects to one, two, and three.

This to two, three, and four.

This to three, four, and five.

So what does this neuron know about?

This one knows how to recognize Gauguin.

It's only getting information from this neuron.

It's another one of those Hubel and Wiesel type,

I know one fact and one fact only.

This one here is another one of those.

What does this neuron know about in the middle?

That's the neuron that knows how to recognize

Impressionist paintings.

That's the one that says, I can't tell you

who the artist is, but it's one of those Impressionists.

It's not one of those Dutch masters.

It's an Impressionist painting.

And this one does it because it is getting information that

is not available to these guys.

It is getting information at the intersection

of all these specific examples.

These ones, number two and four, those

are ones that recognize Impressionist paintings also,

but they're not as accurate at it as number three

because they've got less examples to work off of.

This is how a network would work.

And what that suddenly begins to explain

is something about the human brain versus a computer.

Computers are amazing at doing sequential analytical stuff.

Like, you get calculator things inside Cheerio boxes

that can do more things than the human brain can

do computationally.

But what we can do is parallel processing.

What we can do is patterns, resemblances, similarities,

metaphorical similarities, physical similarities.

And that's why you need networks like these.

You don't need neurons that know one fact and one fact only.

You need neurons where each one of them

is at the intersection of a whole bunch of other inputs.

OK, example.

So now suppose you've got a network.

There's one neuron which fires, and there's

a whole bunch of neurons sort of sending projections into it.

And this is a neuron for remembering

the name of that guy.

What was the name of that guy?

That guy, he was that Impressionist painter.

So suddenly your Impressionist painter network

is activating and firing at this neuron.

So it's sitting there.

So this is-- now you've got your whole Impressionist network

that's activated.

What was the name of that guy?

He was an Impressionist painter.

He painted women dancers a lot of the time.

So people who painted dancers.

But it wasn't Degas.

OK, so your "it's not Degas" circuit going in there.

And what was that guy's name?

God, I had that seventh grade art teacher

who loved this guy's work.

If I could remember her name, I would remember his name.

Oh, remember the time I was at the museum

and there was that really cute person who seemed to like it

and I had to pretend I liked this guy also

and it didn't work out nonetheless?

And going through.

And oh, what's the name?

There's that stupid pun about the guy, he's really short.

And something about the tracks being too loose.

Ah, Toulous-Lautrec.

And suddenly it pops out there, and you've

got enough of these inputs coming in there.

And this is tip of the tongue wiring.

This is how you may not be able to just remember

the guy's name.

Wait, he was the short guy with a beard who hung out

in bars and Parisian bars.

And here was that time in seventh grade.

And enough of these inputs, and suddenly out pops

the information.

And what this begins to tell you is,

this is ways of getting similarities.

These are ways of getting things that vaguely remind you.

This is a world where humans can now do stuff

like have a piece of music that reminds

them of a certain artist because they

both have similar coloration.

And that's something that makes sense to us.

That's something that can work because, what you then

begin to see is, every one of these neurons, this one,

for example, Impressionist neurons.

This one may also be at the intersection of another network

that's going this way, a network of French guys

from the last century.

And it may be part of another network of people

whose names are hard to pronounce

so you're anxious about saying them in a lecture.

Or the intersection-- and each one of these

is going to be an intersection of a whole bunch of these.

All of these networks, what does that do?

That's what you can do that a computer can't.

You see similarities, similes, metaphors.

And somewhere in there you get something really important,

which is the ones, the networks, that have wider expanses that

connect to a broader number of neurons

in a very simple, artificial, idiotic way.

That's kind of what creativity would have to be,

networks that are spreading far wider

than in some other individual.

It is literally making connections

that neurons in another individual does not.

And suddenly you have a world where everyone

knows this one is a face.

And it was only a limited number of people who ever

decided that this one's a face.

And in some level Picasso had a different network, a broader

one, as to what could constitute a face.

A broader network in some way is going

to have to be wiring that is more divergent.

And at the intersection of a bunch of networks

that are acting in a convergent way.

So what's some of the evidence that it actually

does work this way?

You go and you stick electrodes into neurons in the cortex,

and what you see, if the world was entirely made up of,

like Hubel and Wiesel, one piece of knowledge only, what

you would see is you would find neurons that each one responds

to one single thing.

All these grandmother neurons.

And instead, what you see by the time

you get to the interesting part of the cortex

past the first three layers of the visual cortex

and the first three layers of the auditory.

Once you get into the 90% that's called

the associational cortex-- and it's

called that because nobody really

knows what it does-- then what you see

are neurons that are multimodal in their responses.

All sorts of things stimulate them.

And here we have a neuron that's being stimulated

by a type of painting, by the knowledge of French guys,

by something phonetic, by all sorts--

and they're multiresponsive.

So that's what you wind up seeing.

The majority of cortical neurons,

when you record from them with an electrode,

they're not grandmother neurons.

They're at the intersection of a bunch of nets.

More evidence for this.

This was-- one of the grand poobahs of neuroscience

around the 1940s or so, a guy named Karl Lashley.

And obviously a very different time

in terms of thinking about specification of brain

function.

And what he did was a very systematic attempt

to be able to show where in the brain

individual facts were stored.

And the term for it at the time, this jargony term, was engrams.

He was searching for the engram for different facts.

And what he would show was, he would

destroy parts of the cortex in an experimental animal.

And he couldn't make the information disappear.

He would have to destroy broader areas.

And some of the knowledge, some of the memory

was still in there.

And he concluded in this famous paper

in the search for the engram that, according

to all the science he knew, there

could be no such thing as memory.

And the reason why was he was working

with a model of being able to-- there's

a single neuron where, if I could ablate it,

I should be able to now show in that rat

that it's just lost the name of its kindergarten teacher.

And instead, you see networks going on.

You see the same thing clinically in something

like people with Alzheimer's disease.

Early on in Alzheimer's, you will lose, in these networks,

you'll lose a neuron here or you'll lose a neuron there

when you're just beginning to lose neurons.

And what you see is, clinically, in people

with Alzheimer's, early on, it's not that they forget things.

It's not that memory is gone.

It's just harder to get to.

And you do this with all sorts of testing,

neuropsychological testing, where

you try to give the person cues to pull it out.

Example.

You're giving somebody, potentially with Alzheimer's,

a classic orientation test.

You ask them, OK, do you know the name of the president?

OK, they manage to get that.

Do you know the name of the last president?

No idea.

So now you give them a little bit of cuing.

OK, let me help you a little bit.

It's a one-syllable word.

Still not there, even though you've now

activated the one-syllable word network, obviously artificial.

Still can't say.

OK, let's make it a little bit easier.

It's things you could find in a park, in a city park.

So you're activating that.

No, still not coming out.

And then you give even more explicit priming there.

You give them a forced choice paradigm, is what it's called.

OK, so is it President Tree or President Shrub or President

Bench or President Bush?

Bush, Bush, the kid with the father also.

It's still in there.

It was still in there.

It just takes more work to pull it out.

What you're seeing there is not the death

of individual memories.

You're seeing a weakening of a network, a network that

is now taking stronger priming to pull it out of there.

And just to show how subtle network stuff can be,

here's something that would work with a lot of individuals

with early stage dementias.

What you do is another type of priming.

So you're eventually going to ask

them the name of the previous president.

And they first come in and you say, oh, great to see you.

Come on in.

What a beautiful day.

I walked here by way of the park.

The bushes were so beautiful this morning in the park.

Some of them had flowers, some of them didn't.

But bushes are so nice to look at when you're

walking through a park because bushes

are one of my favorite forms of [INAUDIBLE].

And then five minutes later, they

are more likely to remember the name Bush

out of a whole different realm of more subtle networks you're

tapping into.

So all of this is the beginning of a way

of solving the problem we had the other day of not

enough neurons for them to be grandmother neurons.

More solutions.

We then went to our next realm of trouble,

which was the problem of, there's not enough genes.

There's not enough genes in that specific realm

of explaining bifurcations.

And there can't be a gene that specifies, OK, this

is where you bifurcate if you were this particular blood

vessel and a different gene for this particular bronchial

and a different gene for this branch

of a dendrite and a single-- it can't work that way.

There are not enough genes.

What this introduces is the idea of there

being fractal genes, genes whose instructions

are ones that are scale free.

What do I mean by this?

OK, here's what a fractal gene might do.

So we've got a tube.

And this is a tube that's going to be part of a blood vessel

or a dendrite or a lung or whatever.

We've got a tube.

And the fractal rule here is, grow this tube in distance,

grow it until it is five times longer than it is wide.

The width, the opening, and that's the simple rule.

And the rule is, when it's grown five times longer, bifurcate.

So what's going to happen at that point

is just gone five times longer.

And it bifurcates at that point.

And what you've got is now, because this is split in two,

the cross-section is going to be shorter.

But you apply the same rule.

Now with the shorter cross-section,

you have the same rule.

Grow five times the length of that cross-section

until you split.

And what you wind up getting is, this

is one simple fractal rule that will generate the tree

patterns.

That the branchings get shorter and shorter,

the distances between the branch points

get shorter and shorter because the cross-sections are

getting-- one simple rule and you could generate

a circulatory system, a pulmonary system,

and a dendritic tree by giving a fractal instruction,

in this case, one that is scale free.

That is, independent of what the unit is here.

And this could work within the single neuron

or within an entire circulatory system.

So all of that's great.

That's totally hypothetical.

Ooh, fractal genes.

Well, you know by now that's got to translate

into a protein in some way or other.

How might this actually look in a real system?

So suppose-- OK, so a gene coding for a protein.

This is one copy of the protein, this is another,

this is another.

They bind to each other in a way so that they form a tube.

And they bind to each other in a way

that's just pure mechanical reality of, these are not

bits of information, these are actual proteins.

So it's going up in the tube there.

And suppose that the forces are, as the tube goes up

it gets more and more unstable.

And when the tube is high enough,

it gets unstable enough that these bonds

between the proteins begin to weaken, and it begins to split.

The splitting there is a function

of the length of these.

So it's split.

And now the next one has half the number

of proteins in this one, and thus it's that much weaker.

So you only have to go a shorter distance now

before it begins to split.

This doesn't exist.

It is no way it's like this.

But what you could begin to see is,

here's how you could turn a scale-free set of instructions

potentially into what it would actually

look like with mortar and bricks in terms of proteins.

How it might actually work.

Now the notion of fractal genetics, of fractal genes,

and fractal instructions begins to solve another problem,

and this is that space problem of how much stuff can you

jam into a space.

Here's the challenge here in terms of how dense things are.

In the body-- amazing factoid-- there is no cell in your body

that is more than five cells away from a blood vessel.

OK, you could see why you would want to do that.

But that is not an easy thing to pull off.

How do you do that with the circulatory system?

And amazing other factoid to factor in

with that is, the circulatory system comprises less than 5%

of your body mass.

How can this be?

You've got this system that's everywhere.

But it's taking up almost no space.

It's within five cells of every cell out there,

yet it's less than 5% of the body.

And, OK, forget it.

I'm not going to put that up.

But what this begins to-- OK, you convinced me.

So let's do this.

So what you begin to do is transition

to a world of fractal geometry.

You've got all your Euclidean world of nice, smiley, strange

things there.

You've got this whole world of shapes

that are constrained by classic Cartesian geometry and all

of that.

And what fractal geometry generates are objects that

simply cannot exist.

Here up on top, eventually, you will

see the first example of this.

And this is out of the Chaos book.

And this is this cantor set.

What you do is you start with a line.

Start with a line, and you cut out the middle third.

Now for those remaining two ones,

you cut out the middle third.

For those remaining four, you cut out the middle third.

And there it is.

And you just keep doing this over and over and over again.

And what do you do when you take it out to infinity?

What have you generated?

A set of an infinitely large number of objects,

lines, that take up an infinitely small amount

of space.

It's not possible for that to work, yet,

as you go more and more in that direction,

you get this impossible phenomenon

of something approaching having an infinite number of places

that something appears while taking up almost an infinitely

small amount of space.

And what this winds up being is, it's not quite a line anymore

at the bottom, but it's kind of more than a dot.

It's somewhere between one and two dimensions.

It's a fractal.

Its dimensional state is somewhere

one point something or other.

It is somewhere between dots and a line,

and it does this impossible thing,

which is it's everywhere without taking up any space.

Or you could then push it to the same thing

in the next dimension.

And this is this Koch snowflake.

And it's the same sort of rule.

You start with the triangle there.

And the rule is, you take the middle third

and you put a little triangle out of it.

And then take the middle third of that

and put a little triangle out.

And a middle third.

And you just keep doing it forever and ever and ever.

And you wind up with something that is impossible,

which is an object that has an infinite amount of perimeter,

an infinite amount of surface area, within a finite space.

That's impossible.

But it begins to approach this.

And what you see here, this is a way

of just iterating over and over and over to jam

a huge amount of surface area into a tiny space.

And thus it's somewhere but different, sort of like a line,

but it's sort of like a plane by then.

And it's got a fractal form somewhere

between two and three.

It's got a fractal quality of two point something or other.

It's an impossible object that is solving

this problem of being-- in another version,

having surface area everywhere without taking up any space

and being within a finite area.

Next, finally, this Menger sponge,

which is the same exact concept.

Again, you start with the box up there, or the ring,

and you take out the middle third

of each of those segments.

And then you take out the middle third

of each of those segments.

And if you are doing this with what starts off

as a three-dimensional cube, eventually you get something

that cannot exist, which is an object that has an infinitely

large amount of surface area while having no volume.

That's what it produces at the extreme.

And we got something here that's somewhere between two

different dimensions, a fractal again.

And what you see is, this is how the body solves the packing

problem.

Because all you need to do is make the circulatory system,

the circulatory system some version of this,

some version of splitting the ends of the capillaries

over and over and over or making the lungs, with their surface

area for exchanging oxygen, looking something like this.

And this is how you generate a system that

is everywhere and taking up virtually no space.

Obviously, it's not taken out to infinity.

But this is how you can have a circulatory system that's

five cells away from every cell in the body,

yet takes up less than 5% of the body.

This is a fractal solution.

All you do here to generate these

is taking some of these qualities

over and over and over and over, and you

can begin to produce absolutely bizarre, impossible things

in terms of surface area and perimeter and volume

and all of that.

This is how you can use a fractal system

to solve the packing problem.

Now of course, as soon as you're coming up

with the notion of something like fractal genes,

you, of course, have to consider the possibility of there

being fractal mutations.

What would a fractal mutation look like?

And again, most people, most geneticists

and molecular people, do not think

about this in these terms.

But there are people who do who actually talk about things

like fractal gene mutations.

What would it look like?

Suppose you've got a mutation, and it produces a protein

that's slightly different.

And as a result, its got bonds here

that are slightly weaker between different proteins.

So on a mechanical level, what have we just defined?

This is a tube that's going to grow these proteins where

it's a shorter distance before it begins to split.

Because these bonds between them are not as strong.

There is a mutation now where, instead

of growing five times the cross-section, maybe

you're growing 4.9 times the cross-section.

And thanks to that mutation, the entire branching system

is going to be compacted a bit.

It's not going to reach the target cells.

And these would be catastrophic mutations where

the pulmonary system doesn't develop,

the circulatory system doesn't develop.

And what you would see in those cases is,

the mutation is something that has consequences

that are scale free.

Another hint when you see some fractal gene mutations are

a small number of diseases that they're

about spatial relationships in the body.

For example, there's a disease called Kallmann syndrome, where

you get stuff that's wrong with midline structures in the body.

Something is wrong with the septum

between the nose, the nostrils.

Something is wrong in the hypothalamus.

Something is wrong in the septum of the heart.

This is not three different mutations.

This is some sort of fractal mutation messing up

how that embryo did symmetry, how the embryo does

midline structures.

So you begin to see ways here in which

you can solve this and, within the biological metaphor,

where you could begin to get solutions for these problems

and also mutations that can put you up the creek.

OK.

So that is another realm for beginning to solve this.

Another domain.

And here we begin to move into the realm of emergence,

emergent complexity.

Which we will first look at a couple of crude passes at it.

First, emergence driven by biophysical properties.

And do not freak out if you don't

know what I mean because I have no idea what I mean by that.

So I will explain in a more accessible way.

And this was something that was explained endlessly

by a guy who used to be in the bio

department, a developmental botanist named Paul Green, who

died about 10 years ago way too young from cancer.

He was a really good guy.

He would give this famous lecture

where he would start off and he would

describe some sort of disk.

And the point is that the disk, the material inside

was of a softer material than the material on the perimeter.

And he'd be putting up math at this point

that I didn't understand.

But it was sort of a disk like that.

And then he would show that what happens if you heat the system.

What happens if you put heat on a disk like this?

And what he would wind up showing,

going through agonizing amounts of math,

is that, when you heat a system, the only solution

for this system that's trying to respond to the heat

but in different ways on the perimeter versus the inside

is to come up with a double saddle, a double saddle shape.

And the math proved this.

And I had no idea what he was talking about when you come up

with a double saddle shape.

And then what he says is, so that's how

you get a potato chip.

You take a slice of potato, where

there is more resistance on the perimeter

and less on the inside, and you heat it.

And the only solution to the problem

is to come up with a double saddle potato chip shape.

And if you change the outside, the force of it,

if you take one of those great organic,

"give you the runs" type potato chips,

where it's going to have the skin left on the outside,

it's going to be a somewhat different-shaped double saddle.

Because there's only one solution

mathematically to that.

And then you sit there, and you deal

with a very simple, important fact,

which is, that slice of potato knows no biophysics.

That slice of potato didn't fit.

There's no gene that instructs potatoes

to respond to heat in this way.

This was the inevitable outcome of the biophysical properties

of a slice of potato.

And what he then shows is, in plant systems

after plant systems, they develop

where two shoots come out this way

and a little higher up two shoots

this way and two this way and two this way.

They're all double saddles.

And this winds up being a mathematical solution

to a packing problem there.

When plants are growing their stems,

there is no gene specifying it.

You don't need genetic instructions.

It is an emergent property of the physical constraints

of the system.

Another example here that's sort of proto-emergent, somewhat

simpler versions, this phenomenon

of wisdom of the crowd.

And this is one that was first identified

by Francis Galton, who was some relative of Darwin

and started eugenics and was bad news in that regard

but famous statistician.

And being an Englishman somewhere in the 19th century,

he spent huge amounts of time going to state fairs and county

fairs or whatever.

And he was at this fair one day where

they had some oxen up there.

And they were having a contest that, if you could guess

the exact weight of the oxen, you would

get to milk it or something.

I don't know what the prize would be.

And there were hundreds of farmers

around filling out little pieces of paper

where they were guessing.

And what he discovered at the end

was that nobody got the answer right.

Good.

So the owners of this get off easy

without having to give up any of their oxen milk.

But he then did something interesting.

He collected all the little slips of paper,

and he averaged all of them.

And it came out to the correct weight within an ounce.

In other words, no individual in that group

had enough knowledge to be able to truly accurately tell

what this thing was.

But put them together in a crowd,

and out comes the right answer.

Another version of this.

And this one is deeply important in terms

of Western intellectual tradition.

Back to-- is that program Who Wants to Marry a Millionaire?,

does that still exist?

[INAUDIBLE]

In reruns?

In-- OK, so it was this one.

They give you questions, and if you answer them

they give you money and it's great.

And at various points, if you're stumped you've got three things

you could do.

One is, they could eliminate-- you've got four choices.

They can eliminate two of them to make

it a little bit easier for you.

Another is, you have this expert who you can call up.

And the third option is to ask the audience what

they think is the right answer.

And all the audience there has these little buttons,

so they can choose A, B, C, or D of the multiple choice there.

And what the logic is supposed to be is, cut it down to two.

Your chances are better if you have to guess.

Talk to your wise expert, who's sitting by on the phone there.

And they're going to be wise and be able to hopefully answer

this question.

Or ask a whole bunch of people.

And they would all vote.

And any smart contestant would choose

whatever the audience chose.

Because, when the audience was asked, 91% of the time

they got the right answer.

They got the majority of people voting for the right answer.

And this is more wisdom of the crowd.

And this was a much better hit rate

than whoever the expert was on the other side of the phone.

One person could be extremely expert,

but they're not going to be as expert

as a whole bunch of somewhat decent experts thrown together.

This is the notion behind a field called prediction markets

where what you do is you are trying to predict some event.

For example, the Pentagon is very

interested in using prediction markets

to try to predict where the next terrorist attack might be.

And what you do is you get a whole bunch of experts,

and you ask each of them to think about whatever

the parameters are and take a guess as to how long it will

be before the next one occurs.

And what you do is, you average them up

and assume there is a wisdom of the crowd thing going on.

And that will give you lots of information.

Great case of this a few years ago.

There was some submarine or something

that sunk somewhere out in the Pacific, in the ocean.

And nobody knew where it was, but they kind of

knew where the last sighting, the last recording,

was from it.

But they had a whole bunch of naval experts.

And they had all of them sort of bone

up on the knowledge of what was the water temperature and wind

speeds and where they were on the last sighting

and what was on TV that day or whatever.

They got all the information, and each one

made a guess as to where it would be on the map.

And you put them all together.

And they had guesses covering hundreds

of square miles of ocean floor.

And they put it all together, and they came up

within 300 yards of the right location.

So what we have over and over here is this business of,

put a lot of somewhat decent experts together on a problem,

and they will be more accurate than almost any one single

amazing expert at it.

Under a few conditions.

The collection of these partial experts can't be biased.

Or if they are, they all have to be

biased in a random scattering of directions.

And they need to really do be somewhat expert.

If you get a whole bunch of people

off the subway in New York and ask

them to guess the weight of the oxen,

they are not going to wisdom of the crowd their way

into being able to milk the thing afterwards.

You've got to have people who have some experience with it.

And you wind up seeing wisdom of the crowd

stuff going on in all sorts of living systems.

For example, here is an ant colony.

And here's a dead ant.

And they're trying to get the dead ant back

to the ant colony.

And when you look at these things,

they know how to get it, or they get some dead beetle

or something to eat, and a whole bunch of ants

push it over back to their colony.

Oh.

Does each one of them know exactly where

they should be pushing?

No.

What you have instead is, each ant

has somewhat of the right idea as to where

they should be going.

And there are more ants that have a reasonably accurate

notion, a smaller number that are somewhat off,

and a really small number that are way out of whack

because in general ants are kind of experts

at finding ant colonies.

They're pretty informed.

And what you do is you put them all together

and you do this vector geometry stuff.

And it moves perfectly in that direction.

And no single ant knows exactly where the colony is.

You've got a wisdom of the crowd thing here going on.

OK.

Where are we?

Five-minute break.

If you have a chance, could you email me that website

so we could post it in the CourseWorks?

That's great.

OK, picking up.

So now we are ready to take some of those building

blocks, wisdom of the crowd stuff, biophysical potato

chips, and begin to see it more formally

in this field of emergent complexity.

What is that about?

What we've already alluded to.

It's systems where you have a very small number of rules

for how very large numbers of simple participants interact.

What's that about?

Here's what emergence is about.

You take an ant and you put it on a table top

and you watch what it's doing and it

makes no sense whatsoever.

You take 10 ants and do it and none of them make any sense.

You put 100 and they're all scattering around.

And somewhere around, I don't know, 1,000 ants or so, they

suddenly start making sense.

And you put in 10,000 or 100,000 or whatever it is,

and suddenly, instead of some little thing wandering around

aimlessly, you suddenly have a colony

that can grow fungi and regulate the temperature of the colony

and all these things.

And suddenly, out of these ants emerges

an incredibly complex, adapted system, an adaptive one.

And the critical point there is, no single ant

knows what the temperature should be in the colony.

Or if this is time to go out foraging

in this direction instead of that direction.

It all emerges out of the nature of ant interactions.

You've got very simple constituent parts.

An ant, much like one box that's filled

in the cellular automata.

You've got very simple rules for how

they interact with each other.

Ants have, I don't know, maybe 3 and 1/2 rules.

Don't tell Deborah Gordon in the department, who's

an ant obsessive.

But that I may be inadvertently dissing the ants.

But they have a small number of rules as to how they interact.

If you bump into an ant and you do this with the pheromones,

you go this way, and if you go that way.

And I'm just making it up.

They have a small number of rules.

And as long as you've got a lot of ants doing this, out of this

can emerge hugely complex adaptive patterns.

And this is what an emergent system is about.

Simple players, huge numbers of them,

simple nearest neighbor rules.

And you throw them all together, and out comes patterning.

And there is no single ant that knows what the blueprint is,

and there's no blueprint.

There is no plan anywhere that says

what the mature form of the colony should look like.

There are no instructions.

It is bottom-up organization rather than top down.

And you see all sorts of versions, then,

of emergent complexity built around, again,

lots of elements of things with a small number of very

simple rules about how neighbors interact with each other.

We need that board.

OK.

Here we have two, four, six, eight different cities

or eight different places where ant

can find good food or eight different something

or others, eight different locales.

And you were trying to do something efficient.

You need to go to each one of them to sell your product

or to see if there's good food there or not.

You need to go to all eight of them,

and you want to do it as efficiently as possible.

You want to find the way to have the shortest possible path

to go to all of these places.

And this is the classic traveling salesman problem.

And nobody at this point can solve it.

There's no formal mathematical solution.

And by the time you get to, like, eight locales,

there's, like, hundreds of billions of different ways

you can do it.

So how can-- you can't come up with the perfect solution.

But you could come up with maybe kind of a good, decent one.

There's two ways you could do it.

First is to have an unbelievably good computer that just,

by sheer force, cranks out a bazillion different outcomes

and in each case measures how much you're doing it.

And you can get something close to an optimal answer.

The other way of doing it is to have yourself

some virtual ants in something that is now

called swarm intelligence.

Here's what you do.

You need to have two generations of ants.

The first generation, you stick them all down,

different numbers of them, and they all

start off in these different cities,

these different locales.

And their rule is, each one of them goes to another city.

Each one of them goes to another destination.

But here's the follow-me rule.

The ants are leaving a pheromone trail, pheromone trail,

and they stick their rear end down.

What is it?

Head, thorax, abdomen.

And they stick their abdomen down.

And they've got a gland at the bottom

there, which releases a pheromone

and makes a track, a scent track, of the pheromone there.

And a very simple rule, they have a finite amount

of pheromone in there to expend on the entire path they're

making.

In other words, the shorter the path,

the thicker the pheromone trail is going to be.

Now what you do is deal with the fact

that the pheromones dissipate after a while.

They evaporate.

And thus, the thicker the path, the longer

it's going to be there.

You now take a second generation of virtual ants,

and you throw them in there.

And what their rule is, they wander around randomly.

And any time they hit a pheromone trail,

they join the trail one way or the other,

and they lay down a pheromone trail of their own

with their abdomen.

They reinforce the markings on this trail.

And let 10,000 virtual ants do that

for a couple of hundred thousand rounds of generations,

and they solve the traveling salesman problem for you.

Because it winds up being, the short paths,

the more efficient ways of connecting locales,

will leave larger, thicker trails,

which are more likely to last longer and thus increase

the odds that an ant wandering around randomly

will bump into it and reinforce it.

And what you see is, initially, there

will be every possible path.

And as you run this over and over,

it will begin to fade out, and out will

emerge the more efficient ones.

You can optimize the outcome doing it this way,

just asking virtual ants to do it for you.

And this is exactly how ants do it out in the real world.

When they're foraging in different places,

there is a first wave of them that comes out,

and they go to locales leaving scent trails.

And then there are the wanderers that come in,

and when they hit a trail they join it.

There are now telecommunications companies that use swarm

intelligence to figure out what's the shortest length

of cable they need to use to connect up eight different

states' worth of telecommunication towers,

whatever they're called.

And they can sit there and do math

till the end of the universe trying

to figure out the cheapest way to wire them up.

Or they can use swarm intelligence.

And that's what a lot of them do at this point.

It works.

What are the features of it?

This is not wisdom of the crowd.

This is not that every ant knows a solution to the traveling

salesman problem, except none of them have the perfect solution.

But put them all together, and they all

get to vote on outcomes.

The ant don't know from traveling salesman problems.

The ant knows nothing about trying to optimize this.

All the ant knows is one of two different rules.

If I'm walking from one of these to one of these,

the longer I walk, the thinner the pheromone trail.

Or rule number two, if I stumble into one of these,

I join it and put down my markings there.

Two simple rules, one very simple type

of sort of unit of information in there, an ant.

And all you need to do is make sure there's enough of them,

and they solve the problem for you.

This winds up explaining another thing.

How do bees pick a new nesting site?

A bee's nest, a bee's-- hornet's nest, a bee's nest.

Every now and then the bees need to leave and pick

a new place to live.

And how do they figure out the good place?

And there's all sorts of criteria of nutrients.

And so all sorts of bees go out there, and what they do

is they look for food sources.

And they look for a place that will have a lot of food.

Maybe that's a place to go and move the colony.

So we know already, the bee will go out and find its food there,

its food source.

It will come back in.

And here's the colony cut in cross-section.

And what you wind up having is this ring of bees.

Here is the entry.

And you have the bee dancing going on

that we've heard about in the middle of the dance floor

there.

And we've already heard it's this pattern of this figure

eight while shaking the rear end.

And we know what the information is,

which is the angle tells the direction to go out there.

And the extent to which it's wiggling

its rear end is how long you're supposed to fly for.

But the final variable is, the better the resource,

the longer you do the dance.

So you've got bees coming in from all over the place that

have found good resources, that have

found so-so ones, all of that.

And so there's bees doing all this dancing stuff here

of different durations.

And the ones who have found the good solution

to where do we want to live are dancing longer.

The ones who have found the most efficient path

are leaving a message longer.

So now you bring in your second generation.

And the rule is among bees, if they

happen to bump into a bee that is doing a dance,

the bee responds and goes where it tells you to go.

So a bee may randomly sort of bump into one of these guys

and then off it goes.

Actually, I'm sure it's more complicated than this,

but it's along the lines of there's now

random interactions.

If one of the peripheral bees encounters, bumps into,

one of these bees that has information,

it joins in in that bee's group.

And it then goes and finds the food resource

and comes back with the information.

So thus, by definition, if you have found a great food source,

you're going to be dancing longer,

which increases the odds of other bees randomly bumping

into you, which causes them to go and find the same great food

source and come back and dance longer.

And the ones with lousy ones are coming in and dancing

very briefly, and thus there is hardly any odds

of somebody bumping into them.

And what you begin to do is, you suddenly

optimize where the hive is supposed to go.

Again, it's not wisdom of the crowd.

It is an emergent feature of one generation with information

based on some very simple rules, and one information

that generates some random element

and out comes an ideal solution.

More versions of this.

Another domain where some very simple rules out of it

emerges something very complex and adaptive.

OK, so the themes here are two generations,

the more adaptive the signal, the stronger it is

and the longer it lasts.

And then the randomization element.

Another theme that comes through in a lot of emergence, which

is to have your elements in there, your ants,

your bees, your traveling salesmen,

whatever the constituents are.

And now what the rules are are simple rules

of attraction and repulsion.

Which is to say, some of the elements

are attracted to each other, and some of the elements

are repulsed by each other.

Some are pulled together, some are pushed apart

like, for example, magnets.

Magnets are polarized in the sense

that magnets only have two ways of interacting

with each other, simple nearest neighbor rules.

They're either attracting or repelling,

depending on the orientation.

So here's what you do now.

You take a system and something very simple.

You've got some simulated SimCity sort of thing

where you're letting the system run to design a city.

You want to do your urban planning in your city

that you're going to construct there.

And what you do is, you can sit there

and you can study millions of laws about zoning and economics

and all of that to decide something very simple.

Where are you going to put the commercial districts?

And where are the residential districts going to be?

Or you can have just a small number of simple rules.

Which is, for example, if a market appears

in some place, what it attracts is a Starbucks.

And what it also attracts is a clothing store

or some such thing.

So a bunch of rules.

But then you have repulsion rules,

which is, if you have a Starbucks,

it will repulse any other Starbucks.

So the nearest other Starbucks can be this far away.

If you have a competitor's market,

it can't get any closer than this.

That sort of thing, these simple attraction/repulsion rules.

And what you wind up getting when

you run these simulations are commercial districts in a city

where you get clusters of commercial sort of places that

are balanced by attraction and repulsion

where you have thoroughfares connecting them.

And the more elements there are in the two neighborhood

commercial centers, the bigger the connection is going to be,

the bigger the street is, the more lanes, the more powerful

the signal coming through there.

And you throw it in.

And out pops an urban plan that looks

exactly like the sort of ones that the best

urban planners come up with.

And all you need to do instead is run these simulations

with some very simple attraction and repulsion rules.

So you do that, and it winds up producing stuff that looks

like cities.

You do that with a bunch of neurons.

You take a Petri dish, and you throw in a whole bunch

of individual neurons.

And they have very simple rules.

They secrete factors which attract some types of neurons.

And they secrete factors which repel other types of neurons.

And all of them are having some very simple rules.

When I encounter this, I grow projections

towards where it's coming from.

If I encounter that, I grow projections

in the opposite direction.

Simple attraction and repulsion.

And what you do is, at this point,

you throw in a whole bunch of neurons, each one where

you throw into a Petri dish.

And at the beginning they're all scattered evenly all

over the place.

And you come back, and you come back two days later,

and it looks just like this.

You have clusters of neurons sending projections,

and you have all these empty residential areas in between.

And if you just mark this in a schematic way, looking

from above you're not going to be able to tell,

is this the commercial districts in a big city?

Or are these neurons growing in a dish?

And you get areas of nuclei of cell bodies and areas

of projections, and it winds up looking exactly like that.

And amazingly, there was a paper in Science earlier this year.

And it was looking at one of these versions,

again, in this case attraction and repulsion

rules with ants' colonies setting up foraging paths.

And they explicitly compared one colony

to the efficiency of the distribution of the train

stations in the Tokyo subway system.

And what they showed was very similar solutions,

but the ants had gotten a more optimal one.

And the subway system had people sitting there salaried

to figure out the best way to do it.

All the ants had were very simple rules

of, if it's someone from the other colony I stay this way,

if it's someone from mine, simple attraction

and repulsion.

And out comes something that looks like this as well.

So here you see that happening with a remarkably small number

of rules.

Now you put it into a really interesting context, which

is something we bumped into back when first introducing proteins

and DNA sequence equals shape equals function, all of that.

Molecules have charges on them.

Some of them were positively charged, some of them

were negatively.

Whoa.

Attraction and repulsion.

Positively charged molecules are attracted

to negatively charged ones.

Same charged ones repulse.

Here we have a system with very simple attraction and repulsion

rules.

And that's the logic behind, when one thinks about it,

one of the all-time important experiments, something

that was done in the 1950s by a pair of scientists, University

of Chicago, Urey and Miller.

Here's what they did.

They took, like, big vats of organic soup

stuff that just had all sorts of simple molecules in there.

Little fragments of carbon, carbon, little fragments

of-- all sorts of inorganic molecules in there,

little ones in there, floating around in this organic soup.

And what they did was they would pass electricity through it.

And they did this vast numbers of times.

And eventually what they saw was,

they would come back and check, and these random distribution

of these things, of these little fragments,

had begun to form amino acids.

Whoa.

Metaphor.

The organic soup, just the evenly distributed sort

of world of potentially organic molecules

in a world in which electricity passes through, lightning.

What had these guys just come up with?

Some in your, like, kitchen sink experiment

of the origins of life.

And what people have done subsequently is show,

you don't need the catalyst.

There's a whole world of researchers

who study origin of life.

And the basic notion is, you put in enough simple molecules

in there that have attraction and repulsion rules,

and you get perturbations and spatial distributions

of certain ways, and they will begin

to form rational structures after a while.

Here's another version of this.

And I used to do this in class, except I can never

pull this one off, and it just became chaotic.

Kid's toy, you've got these magnets.

You either have-- you have magnets like that.

And then you have little metal balls that

can go onto the magnet here.

And you've got vast numbers of them.

And you can piece them together.

Whoa.

This is starting to look kind of familiar here.

So we have these constituents with very simple rules, which

is the magnets repel each other.

They bind.

These things.

And here's what you would do.

Here's what I would attempt to do.

First off, I would get somebody to show me

how to get the video thing on here to project it.

But you would put up a whole bunch of these magnets in rows,

not too close to each other, nice and symmetrical.

And what you do then is you take a handful of the metal balls

and fling them in there.

And if you do that 400 or 500 times,

eventually they will bounce around.

And amid all the pieces flying, you're

going to get a pyramidal structure like this.

One of those just like that, it's three dimensional,

you know that.

You are going to get one of those that will simply pop out

of this because that's the nature of potato chips solving

their math problem with double saddles.

That's the nature of throwing a whole bunch of elements

with simple attraction and repulsion rules.

And given enough chances, throw in enough perturbations there,

and structures will begin to emerge.

And it's the same exact principle there,

these same ones over and over.

So we've got some very simple versions where

you get emergent complexity.

One is this version of a first generation

has directed searches and the intensity

of the signal that it leaves afterward

is a function of how good of a search

they've done, random wanderers.

Then you have the attraction/repulsion world

of putting these together, lots of elements.

And you begin to get structures out of it.

Next version of this, or next domain

of where you begin to see the fact that these rules are

underlying an awful lot of things.

Suppose here you were studying earthquakes.

And apparently there's just, like, little earthquakes going

on 20 times an hour or so all down on the Richter

scale of, you know, one quarter or who knows what.

But you get enough of these, you get a huge database,

and you can begin to graph the frequency of Richter 1.0

earthquakes and how often do you get the Richter 2.0 and Richter

3.0 and all of that.

And you graph it.

And it's going to look something like this,

a distribution like that, which is obviously

there's a huge number of number one categories.

And it drops off until the extremely rare at this end.

There's a distribution, which mathematically

can be described, something called a power law

distribution, with a certain angle to it.

OK, so here's the relationship between how often

do you get little teensy earthquakes and the big ones.

Now instead, what you do is something much more

different from that, which is, you look at 50,000 people,

and you look at their phone calls

over the course of the year.

And you keep track of how far the phone

call was, how distant the person is that they called.

And now you map the distance, the very shortest calls,

the very longest, and the frequency.

And it's the exact same curve.

It's the same power law distribution.

Next version of it.

This was a study that was done, which was-- I don't quite

know how these guys did it.

I always get lost in the math on these.

But in this one, what they did was

they took a whole bunch of marked dollar bills,

and they started in the middle of-- I don't know where,

I think it was at Columbia, something--

and they were somehow able to keep

track of how far the bills had traveled a week later.

And asking, OK, how many bills had

traveled no more than a mile?

How many five miles?

And it was the exact same curve.

And people now have been showing this same power law

distribution.

Here are some of the things that have been shown.

The number of links that websites

have to other websites.

The number that have only one link.

Power law distribution.

Proteins.

The number of proteins showing certain degrees of complexity

and the numbers dropping off with the same power law.

Here's one which is the number of emails somebody

sends over the course of the year.

This is the one that was done at Columbia.

They got access to everybody's email records.

I don't understand how they could have done this.

But it was a couple of million over the course of the year.

And what they showed was the frequency, how many people

were making this small of a number of emails over--

and the same power law.

Then there's this totally crazy one,

which is, OK, do you guys know the Kevin Bacon, six degrees

of separation thing there?

OK.

Someone went and did a study about this

that they got, like, every actor that they

could find who was in a film in the last two years.

And they got all of their filmographies.

And they generated their Kevin Bacon degrees of freedom,

degrees of--

Separation.

Sing it out.

OK.

And they figured it out, the number for each individual.

And then they graphed it.

How many people were six degrees of separation away,

how many were five, so on.

And it's the same pattern.

And this one keeps popping up, this power law business.

And what you see intrinsic in that is, it's a fractal.

Because some of the time you're talking

about what's happening with the tectonic plates on Earth,

and some of the time you're talking about phone calls,

and some of the time you're talking about how molecules

interact with each other.

There's something emergent that goes

on there, which is an outcome of some

of these simple attraction/repulsion rules,

an outcome of simple pioneer generation

and then random movement ones.

And out come structures like these.

This winds up being applicable in a very interesting domain

biologically.

OK, so now we go back to the traveling salesman problem.

And we're having now a cellular version of it

in terms of networks.

You've got a whole bunch of nodes here.

And the choice that each node has to make,

in effect, is how many connections it

will make in the network to other nodes and how far

should those connections be.

Should it only connect with ones way out there?

What does it want to do?

That's nonsense.

In terms of optimizing a system, what

do you want your distribution of connections of nodes

in a network to be?

What is it you want to optimize?

You want to get a system that has

very stable, solid interactions amongst clusters of nodes

but nevertheless occasionally has

the capacity to make long-distance connections

there.

And what you wind up seeing is, if you generate

a power law distribution in terms of, OK, all

of my projections are going to be within this distance

and within this same power law distribution

so that the vast majority of the nodes in the network

are having very local connections.

But still there is a possibility now and then of very long ones.

You get a system that is the most optimal for solving

problems most cheaply, cheaply, and whatever the term is there.

And this solves it for you.

And then you look at brain development.

So you've got neurons forming in the cortex,

in the fetal cortex, and you've got neurons.

You've got all these nodes.

And they have to figure out how to wire up with each other

and how to wire up in a way that is most efficient.

What's most efficient in order to be

able to do the sorts of things the cortex specializes in?

And you now begin to look at the distribution of projections.

And it's a power law relationship.

Most neurons in the cortex are having

the vast majority of their projections very local.

But then you have ones now and then

that have moderate ones, even rarer ones, that

have extremely long ones.

And you look, and this is how the cortex is wired up.

It follows a power law distribution.

And what this allows you to do is

have clusters of stable, functional interactions.

But every now and then, you can talk to somebody way

over at the other end of the cortex to see what's happening.

Interesting finding.

Autism.

Autism, people have been looking for what's up biologically.

And the initial assumptions would

be, there's not going to be enough

neurons in some part of the brain

or maybe too many in another.

What appears to be the case so far

is there's relatively normal number

of neurons in the cortex.

But then some people started studying the projection

profiles of neurons in the cortex of individuals

with autism post-mortem.

Very rare to get these.

And you see a power law distribution.

But it's a different one.

It's a steeper one.

What does that mean?

In the cortex of autistic individuals,

way more of the connections are little local ones.

There's far fewer of the long-distance ones.

There are way more local ones.

What does that produce?

Little pockets, little modules of function

that are isolated from other ones.

And that in some ways is what's going on functionally

in someone with autism.

There is a lack of integration of a whole bunch

of these different functions there.

And that's what happens when you have maybe a mutation or maybe

some epigenetic something or other prenatally that

changes the shape of the power law distribution.

Interesting.

There's a gender difference in the power law distribution

of wiring in the cortex.

Which is, in the typical female brain,

if this is the power law distribution.

And in the male brain it's a little steeper.

Male brains are more modular in their wiring.

What's the biggest part of the brain?

OK, we're running out of space here.

There it is.

There's the brain in cross-section.

And you've got cortex here and cortex there.

And famously, here's all the cell bodies.

And when projections are going from one hemisphere

to the other, it goes across this huge bundle of axons

called the corpus callosum.

The corpus callosum is thicker in women than in men,

on the average.

It is thicker in females than in males

because the power law pattern is such

that there are more long-distance connections

in female networks, and thus it's a thicker corpus callosum.

The same thing is playing out with connections

like this, and connections.

But this is the big honker one.

You get a thinner corpus callosum in men.

You get an even thinner corpus callosum in people with autism.

Again, that hyper male notion there of Baron Cohen's.

What you have here is a perfectly normal number

of neurons, probably even perfectly normal number

of connections between the neurons.

But they're more local, they're more

isolated in the autistic cortex.

There's less integration of function.

It's more isolated islands of function there.

OK.

More examples of where you can get

sort of patterns coming out.

Another version of it, which is bottom-up quality control.

You start a website, you are selling some product,

you are selling books or whatever,

and you're asking people to rate the books.

And you have a board of experts that read all your books,

and they're editors and they're wise and they're learned.

And they write your book reviews and recommend

which ones should be bought and which ones not.

And you get this very successful business

going so that you're selling more and more different

kinds of books.

And as a result, you need to hire

more and more of these experts to read the books

and produce their ratings.

And eventually that just becomes too top heavy.

And what do you do?

The whole world that we completely take for

granted now, you have bottom-up, bottom-up evaluations.

Everybody rates things.

And that's the world where you punch in a book into Amazon

or you look at something in Netflix and when you return it,

it will give you, people who liked this movie tend

to like these things as well.

There are no critics, professional critics,

sitting there doing top-down evaluations.

This is another realm of expressing

attraction and repulsion rules.

I liked this.

I didn't like this.

And all you need to do, then, is throw

in elements of randomization, and you've

got bottom-up quality control.

And that's a completely different way

of doing these things.

What's the greatest example out there of bottom-up systems

with quality control?

Wikipedia.

Wikipedia does not have gray-bearded silverback elders

there writing up the Wikipedia knowledge

and sending it on down to everyone else.

It is a bottom-up self-correcting system.

It is very easy to make fun of some of the stuff

that winds up in Wikipedia, which is, like,

wildly, insanely wrong.

But when you get into areas that are fairly hard nosed.

Very interesting study about five years ago

that Nature commissioned, which was getting a bunch of experts

to look at Wikipedia and to look at the Encyclopedia Britannica

and look at the hard-nosed facts in there

about the physical sciences, the life sciences.

And what you got was, Wikipedia was

in hailing distance of the Encyclopedia Britannica's

level of accuracy.

And that was five years ago.

And it has five years of self-organized correction

since then.

This is amazing.

The Encyclopedia Britannica is like written-- there's,

like, 30, like, elderly, stuffed British scholars that they,

like, have locked in a room for years

who produced the encyclopedia.

And these are the law givers and the knowledge--

And you just let a whole bunch of people

loose with somewhat differing opinions

about whether Madonna was born in 1994 or 1987

or whatever it is.

And you throw them all together and you

do wisdom of the crowd stuff.

And out comes a self-correcting, accurate, adaptive system

with no blueprint, just with some very simple local rules.

Very simple ones, which is looking for similar patterns

shared between different individuals,

and self-correcting.

Where you get even more efficient versions

of that is with a lot of websites,

where not only does everybody get to put in their opinion,

but people whose opinions are better

rated have more of a voice in evaluating somebody else.

You're putting in weighted wisdom

of the crowd-type functions in there,

and out comes incredible accuracy.

These are great.

There is one drawback with those systems, though, which

is, with ones like Netflix, where it tells you

you're going to like this if you like this, that sort of thing.

It's a system that is very biased towards conformity.

It's not good at spotting outliers and sort of taste

and such.

What you really want to do in those systems is,

here are the movies-- of the movies that are out right now,

here are the ones that have 10% of the people think

it's the greatest movie they've ever seen

and 10% think it's the worst movie.

That's an interesting movie to see.

That's when you want to be able to get

a way of bottom-up information about the extremes.

Movies that generate controversy.

Everybody's going to love whatever it is,

and that doesn't take a whole lot.

This is a way to break the potential for conformity

in these bottom-up systems.

Nonetheless, overall it winds up solving a problem

without professional critics, without a blueprint,

without top-down control.

So how do you wire some of these up?

Back to the cortex.

And the adult cortex has these power law distributions,

and they're great because they optimize.

They've got lots of stable, local communication,

but there's still the ability to do

creative long-distance connections.

So that's great.

But how do you get that?

How does the nervous system wire up this way?

And it does swarm intelligence.

The developing cortex does a swarm intelligence solution.

When the cortex is first developing,

what you will have is a first generation, a pioneer

generation, a pioneer generation of cells.

The cortex surface, all of that, that there is a pioneer

generation of cells that basically grow

processes up like these.

And these are called radial glial cells.

What they are, they're the ants with the first generation

of setting down the trail here.

They're the first bees coming in.

And what you then have, the neurons

are the second-generation random wanderers.

And what they do is they come in.

And as they begin to develop, they

have rules that, when they hit a radial glia,

they grow up along it.

They migrate along it, they throw up connections.

And you do that with enough of the cortex, which

is hundreds of millions of billions of neurons in there,

and you get optimal power law distributions.

All you need are some very simple local rules.

And out of that emerges an optimally wired cortex.

And it's the same simple emergent stuff going on.

OK.

So how do we begin to really apply this stuff to humans?

Because it winds up being very pertinent and making

sense of some of the most interesting complex things

about us.

So what's the difference between humans and every other species?

Nothing all that exciting.

From a neurobiological standpoint,

you've got this real challenge, which is, you look at a neuron

from a fruit fly under a microscope

and you look at one from us and it's

going to look kind of the same.

Looking at a single neuron, you can't tell

which species it came from.

We have the same kind of neurotransmitters

that a worm uses in its nervous system.

We've got the same kind of ion channels,

the same sort of excitability, the same action potentials.

You know, minor details are different.

We have not become humans by inventing

new types of brain cells and new types of chemical messengers.

We have the same basic off-the-rack neuron

that a fly does.

Oh.

We have very similar basic building blocks.

What's the difference, of course,

is we've got 100 million of them for every neuron

that you find in a fly brain.

And out of that comes emergent properties.

Great story.

Garry KAS-pah-rof, kas-PAH-rof, I never

remember which syllable to emphasize.

Grandmaster Russian, chess grandmaster in the '90s.

And apparently he's rated as one of the strongest of all times.

And he was the person who wound up

participating in this really major event, which

was this tournament with this chess-playing computer

that IBM had built called Deep Blue or Big Blue or Old Yeller.

What was it called?

Deep Blue, Deep Blue, Deep Blue.

And they played against each other.

And apparently what happened was, in the first game,

Kasparov won perhaps.

And the computer was able to modify its strategy

and then proceeded to mop the floor with him.

And this was a landmark event in computer science.

This was the first time that a computer

had beaten a chess grandmaster.

Amazing event.

Not surprisingly, afterward Kasparov

is all bummed out and depressed.

And his friends were trying to make him feel better.

And they go to him and they say, look, all you got done in by

is quantity.

All you got done in by is the fact

that that computer could do a whole lot more

computations than you could in a set amount of time.

I'm told, apparently chess grandmaster types

can see five, six moves ahead.

And they can intuit where the interesting ones were.

And Deep Blue could calculate every single possible outcome,

like, seven, eight moves in advance.

And every time, it would simply pick the one

that was the best outcome.

It was like generating solutions to the traveling salesman

problem.

Kasparov didn't have a chance because the computer

could simply generate enough solutions

to pick the right one.

So all of them are saying to him,

you should not be depressed because all that computer had

going for it was quantity.

And what he said in response was,

yeah, but with enough quantity you invent quality.

And that's the exact equivalent of one ant

makes no sense and 10,000 do.

That's the exact equivalent, with enough of these elements

here, you optimize.

We do not have fancy neurons that

are different than in any other species.

We've just got more of them.

And simple nearest neighbor rules, and you throw a million

of them together and you get a fruit fly.

And you throw 100 billion of them together

and you get poetry and you get symphonies and you get theology

and you get all of that.

And it's the same building blocks.

With enough quantity, you invent quality.

And this is the punch line that came out

of really important work a few years ago.

OK, we're now, what, 10 years, I think, into having

the human genome sequenced.

And about five years ago they sequenced the chimp genome.

Soundbite.

Everybody learned from whenever back

when is that the human and chimp share 98% of its DNA.

So finally you had these two gigantic rolls of print-out.

And here is the entire human genome

and here's the entire chimp one.

And somebody could finally sit there

and compare them and compare them and see, indeed,

is it 98% shared?

And that winds up being the answer,

even though what that number actually means is debatable.

But that brings up the question, of course, what's the 2%?

What's the 2% that differs?

And what has come out of that have been some very interesting

findings.

Some that were mentioned earlier on,

which is, they are disproportionately

coding for transcription factors and splicing enzymes

and, OK, that amplification of network stuff.

It is preferentially coding for non-coding regions, differing

but non-- all the stuff from back,

that lecture for getting macroevolutionary changes.

That's how you get a different species coming out.

But how about other types of genes?

What were some of the key differences?

Here was one big difference.

We have about 1,000 fewer genes for our olfactory receptors

than chimps do.

They've been inactivated in us.

They're called pseudogenes in us.

They don't express.

And that's about half of the difference in the genome

between humans and chimps.

If you want to turn a chimp into a human,

you're halfway there if you just give it a lousy sense of smell.

That's half the genetic differences?

What other differences are there?

There were ones having to do with morphology, bone

development, probably bipedalism versus being

a partial quadruped.

There are ones having to do with hair development, which

is why chimps have all the hair on them and only those,

like, disturbing people with the hair on their shoulders

have that much hair.

So that's-- there's differences in some reproductive-related

genes.

You don't want to mate with them, all of that.

And then you say, where's the genes

having to do with the brain?

Are there any differences there?

And there turned out to be very, very few.

And they turned out to be very, very logical.

The handful that differ seem to have something

to do with cell division.

Have something to do with how many rounds of cell division

these cells go through.

And what you have is, the human versions

go through more rounds.

And calculations have been done looking

at the average number of neurons that each progenitor

cell generates, say, during cortical development.

And if you start with the number of neurons

that you find in a rhesus monkey brain

and have it do three or four more rounds of cell division,

you get a human brain in terms of the numbers.

Qualitatively, it's the exact same neurons.

All that differs is quantity.

And you put enough of these together

and you go from tools, which are meant

to get little termites out, into human technology,

the difference between us and them is one of quantity.

Throw enough neurons in there, and out begins

emerging all these distinctive human things.

So what does that do?

That begins to, for one thing, underline

what the main genetics are about in terms

of the genetic differences in the brain between us

and, say, chimps are genes that free you

from genetic influences.

Because those are not specifying what sort of cells

you generate in larger numbers in the brain.

They're not specifying connections.

They're just specifying larger quantity.

And all this stuff goes to work and out comes a human brain

instead of a chimp one.

OK, so what does this whole subject get us?

The chaos stuff, the complexity emergent stuff.

What are some of the themes that come through with all of it?

The first one is this emphasis on quantity.

You want to get a very, very fancy system.

You don't necessarily have to invent a new type of ant

or a new type of 0 or 1 in a binary system

or a new type of neuron.

You could do it with quantity.

You get quality, you get excellence, you get complexity,

you get adaptive optimization with huge numbers of elements

with the very simple rules.

What's the next theme that comes out of it?

One that is totally counterintuitive.

Once again, like this whole subject

that shoots reductionism down the drain,

totally counterintuitive.

The simpler the constituent parts, the better.

Fancy, complicated ants that are specialized and have

all sorts of different rules.

They are not going to generate swarm intelligence

as effectively as do systems with the simpler elements.

The more simple the building blocks are, the better.

Something else that is intrinsic to all

of this, which runs counter to all sorts

of rational intuitions, which is more random interactions

make for better, more adaptive networks.

You want lots of random noise thrown

in there because that's how you stumble onto optimal solutions.

Randomness is a good thing.

And remember, right at the time that we're making new neurons

in the cortex, that's when you induce the transposable events

in the genome.

That's where you juggle the DNA producing randomness there.

Randomness is a good thing.

Randomness adds to the excellence of networks

What else?

Next thing as a theme that comes out of it

is the power of gradients of information.

Things that guide you, you a cell, you an ant,

you a commercial district.

Things that can guide you towards things,

things that can repel you, gradients

of attraction and repulsion.

And that's exactly what's going on.

There is a gradient in magnets when

they're this close and the power that they have is dropping off.

As they move, gradients provide a lot

of the optimization in these systems.

Very, very important as well is nearest neighbor interactions.

These are not just a handful of simple rules

about how you're interacting with somebody in Chicago.

These are all how you interact with another ant,

another bee when you bump into it, a glial cell.

Local interactions with simple rules.

Something else.

Another one that runs totally counter

to intuition, which is generalists

work better in these systems than specialists do.

Generalists are more likely to come up

with these adaptive outcomes.

OK, so what does all of this mean on a larger level?

And what I think is that this is where

the complexity of human brains and human behaviors come from,

these emergent properties.

And this is now a generation or two into people

thinking about this stuff.

And it is incredibly hard to think about.

And most of the work I do and my peers do

is reductive stuff that is very limited.

And like, I don't understand how to think

about it in this other way.

And the odds are, you guys are not

going to be good enough at it either.

You're good enough that you were a first generation growing up

that knows, if you want to find out

if you're going to like a movie or not,

you don't need to have somebody with expertise

and a label on their forehead and a blueprint and top down.

You don't need critics anymore.

You have bottom-up systems.

You guys who are first generation growing up

thinking in that way.

What's a consequence of that?

You are beginning to get better at this stuff.

And my guess is, it's not until, like, your grandkids

that you're going to have people thinking

so much in the emergent systems that we're finally

going to be able to figure out what the brain is doing.

And where you see there is all sorts

of things that can happen.

If there was more bottom-up communication

in the trenches in World War I, they

would have stopped the war.

All these emergent things bottom up.

We've now had revolutions when Marcos

was overthrown in the Philippines

back when that was basically bloodless.

When the Czech revolution occurred,

it was called the Velvet Revolution

because there was no violence.

All they had to do was get enough people in the town

square in the capital and paralyze the country,

and they took it over.

I will predict that within our lifetime

there is going to be a revolution

in some country at some point where nobody

leaves their living rooms.

All they do is do something online

with some emergent bottom-up thing

and they collapse the government and do it in and no one

will have to leave their living room because it will be

all emergent things coming up.

The final couple of points here.

First one is, all that chaotic, strange, attractor stuff, all

of us spend a lot of time thinking

about how we're not quite up to the ideal this or not.

We're not at the ideal appearance.

We're not at the ideal intelligence.

We're not at the ideal choice of perfume.

We're not at the ideal anything.

What strange attractors and chaos shows

you is the notion that there is an ideal,

that there is an essentialist optimal, whatever, is a myth.

We are all deviating from the optima

because the optima is just an emergent, imaginary thing.

The other final point is, something that you guys are

going to be much better at than any previous generation,

which, if you grow up thinking, when

I want to find out if a movie is good or not,

I do bottom-up stuff, you are growing up with a mindset

that you don't need blueprints.

You don't need top-down blueprints.

And implicit in that, when you look

at how you can get complex, adaptive, optimized systems

without blueprints is the fact that, if you

don't need blueprints, you don't need somebody

who makes the blueprints.

And it will be a lot easier to comprehend that

as being the case.

You don't have to have a source of top-down instruction

if you don't need a blueprint.

OK, so I don't know.

I'm talking about something--

For more, please visit us at stanford.edu.