Alan Kay talk at BSO MultiMedia Group

[Music]

in an vena even bouton Los Angeles Bond Alan Curtis gay benefactors are out a

Varrick by Apple and were called the Archfiend upon the personal computer who

noon a fool dr. tend to see it in the communicate see to some demands and the

machine you need all that optimal openness I'm a person who's always been

interested in design and the whole

interaction of how our minds work with our bodies through our visual senses and

through our ability to capture things symbolically that's what I'm interested

in all my life Alan Kay the Nieto did lament on a sabbatical year Elaine

softens done on Schmitt assault vehicle ten Eva Cronin on the computer talking

about a certain article at artfully from the Maris and proceeded upon the windows

marked a distance the computer of fulfillments and to conclude just as far

fewer people right well then are able to speak it's a little writing is a little

bit more artificial than reading I think

you are forced to do it less often the same thing with computer you can use

tools made by other people for a very long time but then at some point you

find something I'd like I wish it could do this or I wish I could do that and

it's allowing people to be able to do that without having to become a

profession interests L&K needle a peck of computers

Maroko Oracle's tears up in the pan circuit character from a keyboard from

the computer nerd clavier Fernet Oracle is will have my recliner stop many many

computer scientists are musicians come

in a because of the parallels I think between the two just as Einstein played

violin one of the things he drew from it with were aesthetic feelings that he

wanted to apply to mathematics and physics I I mean I think most of us computer scientists have the same kind

of feeling that we're looking for a kind

of structure that is like the computer architectures we deal with it jaeseok

I've seen dogs on commencin who in fact enough of stone Deluca holding ten

opposite the vanadium not a violent businessman and educators have to look

after their own interests they have to understand that the computer is a new

piece of media so it has the impact of

the printing press and those who were slow to adapt to what the printing press

was all about we're slow to get past the Renaissance and into Western

civilization comparative a lot L&K long as it's turned on velvet on a Turkish

taught school in all arrests nearly a day in up to doom sometimes ideas pop

into your head and also the the relaxed

feeling of the beach is more the way I would like human computer communication

to be then tense and goal-oriented

okay ended on TV day and hey Topher down sort of a new hood that was in here and

here is Alan Kay

okay thank you I'm here to talk about

computers and communication two words that haven't often gone together in the

past is sort of like Airlines and food or consulting in business but the way we

try and communicate with computers these days the most successful ways have come

about from under trying to understand better what a computer is not as a

device for doing word processing imitating paper in various ways but to

try and understand how it lies at the root of the human experience

what can we compare it to well one one

thing I like to compare it to and I'd like to go to the first slide here is

the book we often think of a computer as a calculator but in fact the book I

think is a closer way of thinking of it a book is a piece of technology that

contains marks that represent our ideas and the marks can be about things that

we think are true they can be about things we'd like to think are true for

instance this is a chapter about transforming a dead moon like our own

into an earth-like planet and there are equations and there's an argument in

this book that says this could be done if we could make such and such an effect

and that's books contain ideas that are

outside of the physical world and sometimes have to do with it now

if we look at a computer one way of thinking about as a computer is a book

that can read and write itself and so if

we wanted to transform a moon into an earth-like planet and we were thinking

about representing it on a computer we wouldn't just represent what we wanted

to do we wouldn't just represent equations we'd represent it as a dynamic

simulation that actually tries to carry it out and because of that we get to

understand our ideas and yearnings much better

let's take a look at this as it's actually done on on a computer let's

look at the first video so here we have

our airless moon and [Music]

[Applause] [Music]

[Applause]

[Music] [Applause] [Music]

[Applause]

[Music]

so the computer is big very very big

computer art is not showing a picture but showing a process

so those geological formations that you saw were actually being carried out

using our best knowledge of how mountains and valleys are created on the

earth so that was not a cartoon animation but actually a representation

and illustration of the kind of simulation that computers are actually

capable of and it's in that light of

looking at computers I want to give the talk today a second idea that I'd like

to share with you is that most of our

existence as human beings over the last several hundred thousand years we've had

two major ways of extending ourselves and the first way immediately comes to

mind go to the next slide please I I need to get my tie signal down so we

immediately think of tools like a screwdriver and a wheel but we also have

some mental tools like language and

mathematics and all of these tools I

think of as extensions of the gesture they help us manipulate things they make

them into little bits and pieces that we

can then move around and make new combinations that may have new meanings

that are useful for us the second way that we've extended ourselves over the

years is a little more subtle but it illustrates perfectly by this by this

gathering and that's that we extend our goal structures into other people so we

can another person can act for us as an

agent so we can manipulate and carry out

goals by doing it ourselves or by

getting another human being to do do it for us and even better an agent of ours

can get other agents to do things so if we want to work on something hard we may

have dozens of agents that are working with other agents and for most of

history actually until just recently this way of extending ourselves of

making machines was the most dominant way and the the M word for that is

management so we have manipulation for tools management with with agents now

the problem with the mainframe computer when it came along is it didn't resemble

either a tool or an agent and so it was some complicated actually resembled an

altar of a God cared for by priests and

white coats and so forth and it didn't

feel like you could step up to it and use it either directly or by getting it

to take on on some of your goals now a

very pivotal year in the history of

communicating with computers was 1968

when there are a number of systems that were very different one of them was done

by Douglas Engelbart in California and

let's take a look at what his system was like in 1968 let's run that next so here

he is in 1968 this is a picture that could be taken today he's sitting in his office he has a black on white screen he

has a mouse to point at things and the

system was more than just a text editor as you see right now let's have some

sound with this please for the produce they get big I'd like to say one branch

only and let me look just that low and I

see it oh I can say I'd like to see one line only I can see it but there's a

little thing I can do there's a root place that I have here

so here I'm afraid I'll need a different picture the view so here's what I do

with a picture drawing capability here's a slight and lamp if I start from work and here's the route I seem to have to

go to to pick up all the materials and that's my plan for getting home tonight

but if I want to I can say the library what am I supposed to pick up there I

can just point to that you know oh I see overdue books and all while there was a

statement there with that name on it go back what if I once am I supposed to

pick up the drugstore hmm I see everything all right market can do

things if I want to just say I'd like to interchange the parties and camp

materials bingo they're all numbered right if I

care to look interchanging them very quickly cans are going to get

interchanged with Prados they do it and all gets me numbered so that vision

could have been shot yesterday even so

there are very few computers today that can do the kind of hyper media hypertext

stuff that you saw there this is a complete vision of what I call the

personal computer desktop machine with a

pointing device a use of the machine other than just imitating flat paper

completely modern in every way and for me back then it led to a machine design

and what it looked like this this is one of the very first tabletop computers for

very special tables this machine weighed about 200 kilograms and it had some so

many components in it that it had a fan that was likened to a 747 taking off but

for special tables made out of steel this was a tabletop computer and the

problem with it was that it had really good function but it wasn't easy to use

in fact we never even thought about the idea of computers being easy to use at

least I never did until I built this machine this is what it looked like on its own display by the way so you can

see what people were thinking about back in 1967 and 68

well at very same year a couple of new

systems also appeared that were very different one of them was done at RAND

Corporation for people who were not computer scientists in fact they didn't

even like to type they said we we hate typing we're used to putting our ideas

on paper and moving them around and everything can't you do something about that that let the invention of the first

really good high-resolution tablet and it looked like this first we erase a

flow arrow then move the connector out of the way so that we may draw a box in

its place so I drew a circle I mean a square it's all I wanted a rectangle of

made one now it's recognizing his handwriting is being used as commentary only in this case the box is slightly

too large so we may change its size here's where modern-day window control

came from then draw a flow from the connector to the box attach a decision

element to the box and draw a flow from it to scan we then erased the floor

arrows attached to the process post new area and move the box to a new position

this allows us to draw a new box so

forth no menus no function keys every command given as an analogy to the thing

you actually wanted to appear there well I had a chance to use this in 1968 and

felt just like I was pushing my hands right through the glass of the display

and touching the structures directly underneath felt completely different

from angle Bart's system and I had to call it intimate I felt like I was

dealing directly with the information I was trying to work with there are less

barriers there and the other thing that

happened that year was the first flat-screen display a little one-inch square piece of glass that could show a

few dots on it and that led to this image cardboard model I made in 1968

which I called the Dynabook dynamic book so portable you can carry other things

to something that you would always have

with you connected into the world of information continuously using digital

cellular and so forth a dream machine but of a very different type an intimate

type and much of what has happened in the last 21 years has had to do with

playing out these two ways of using the of using computers in the first way I

showed you we were using the mouse for pointing at things the machine was

showing us what it was doing and it was reacting to us was reacting to things

that we did we had it we had to do things so we were watching it tools are

something that you watch and manipulate the second case the system was watching

us it was trying to see what we were trying to do as we were fumbling around

trying to make drawings and then it would recognize those drawings as goals

we were trying to achieve and achieve those goals for us so in the second case the system watched us and the difference

between tools and agents is and tools we watch them and an agents they watch us

they try and fit closer to us so what I'd like to do is to three ways in 1968

existed for thinking about communication with computers and one of the ways of

characterizing it is just where is the computer that you use yesterday it was

in somebody else's computer room was like a railroad somebody else made the

schedules decided what you could do with it today that for most of us the computer is on our desk and tomorrow the

computer is going to be wherever we are in California we'll weave it into our t-shirts the computer of tomorrow won't

have an on/off switch it'll be something really useful like a clock that we've

never think of turning off because it even when we're not directly working

with it is going to be working for us all the time because it can take on our goal structures another way of thinking

about how we measure performance on computers on time sharing systems it was

how long does it take to get a response back from the main computer today on a desktop computer we

think of how many millions of instructions can it carry out for us and

tomorrow none of those things will be important the computer of 1995 will have

25 to 50 times the computing power of the machines that we have today all

we'll worry about tomorrow is through the global networks that are now being

set up will we be able to access to find the information and tools that we need

to advance our goals this change is

going to be as profound as the change from the mainframe to the desktop

computer what kinds of data do we work with well just letters and numbers on

the mainframe for the most part on a Macintosh its fonts and 2d graphics

tomorrow it's going to be all media and we'll see an example of that from the

last speaker today field of interaction feels like editing on a 30 to 70 screen

on a Macintosh it feels like layout because we're moving things around but

tomorrow it's going to feel like orchestration or conducting because we

aren't going to be dealing with passive objects anymore but active ones active

ones that are trying to find what it is that we want to do how do we issue

commands on the institutional computer we had to remember and type and still do

for people who buy them Macintosh at C in point it always shows us what it's

willing to do next but tomorrow we'll go

from personal to intimate intimate communication is through gesture asking

telling and so forth and a lot of what I want to say today has to do with that

transition now another idea is that

functionality which people think they're paying money for on computers is

actually not very worthwhile if I gave you a screwdriver like this you'd be

very angry at me yeah this is the functional part of it functionality

needs a user interface and the result is

a tool a tool is a function plus the user interface and if you don't have

either one you don't have a tool now

with agents we use the word functionality here's one

that knows how to use a screwdriver in a drill and again we need a user interface

language in context as the first three speakers have talked about quite a bit

to have the idea of that the screw is a

good place to use the screwdriver and the result is an agent so functionality

plus context and language gives us an agent so in both cases we have the

problem of how do we find a reasonable user interface now like the earlier

speakers I believe that language

basically has to do with trying to agree on some context that we both share so

here are two people each with a thought cloud they overlap on to be or not to be

in the screwdriver but the one on the

left is a mathematician and a physicist and the one on the right is a musician

and they have a would have a very hard time communicating because communication

just as I'm trying to do with you today is not the result of just telling you in

English or any other language because language itself does not carry much

meaning what language is is an extension of the gesture what I'm really trying to

do is to point to something in my thought cloud that I hope is in your

thought clouds and if that is then we are communicating and of course the

difficulty was communicating with computers is they don't have much in their thought clouds now in there are

two ways around that you know I'll show them to you in order to to work with

this we have to have some sort of understanding of the human mind and the

problem is that nobody really understands how it works it's extremely

difficult most of the common sense notions about how humans work like a

common sense notion for example is I am a single eye I'm a unity I am a unitary

mentality just me there but in fact it

doesn't quite work that way consider this now take a look at this and please

don't be shy tell me what you see is wrong with this picture who can see it mouth is upside

down and what else anybody see it

the let's pretend we're in school so I'm

going to tell you in English a fact about this and so description of ink in

English of this as we've taken a photograph of a young girl extracted her

eyes and mouth turned them upside down put them back into the picture and then

turned the entire picture upside down now you know everything about this

picture because I just told you so you shouldn't be at all surprised when I show you what it looks like right side

up now I'm going to move this back

because I've discovered nobody will listen to me when it's on the other way

these are the kinds of things that perceptual psychologists really love the

feminists don't sorry the I'll give you

equal time later [Applause]

the the important idea here is I

presented the same information twice once through language went through your

ears into a part of your brain that processes language and the thought about

it said okay that's a fairly reasonable statement that he made it's like and the

other way when I showed it to you it went through the visual part of your

nervous system and was processed in a completely different way produced a

startling reaction because if you see

something like the face that I showed you your nervous system built in just as

the first speaker is talking about this is a built-in reaction see something

like this might mean you're going to die in the next ten seconds it works for

most mammals so it's built in and so if

you try this experiment on a dog I suggest you pick a small one now what

does this mean what it means is that our mentalities

are actually composed of separate

modules the modules are often tied to

the senses but they can be tied to other things the modules came about four

different evolutionary reasons they think about the world differently they

take in information differently and they occasionally oppose each other so I I

can't see out in the audience too well but could I see the hands of all the

tennis players that are here okay how

many people learn to play a set of tennis with forehand backhand and

overhand serve in a week okay how many

did it in a day how many did it in 20 minutes now why is

tennis hard or is it seems to be the

pros like us to pay them for many tennis lessons you can take the lights down now

thanks in fact a challenge was issued a

few years ago to a fellow who wrote a book that says tennis is actually quite

easy anybody can learn to play a decent game of tennis in a single afternoon if

they don't try too hard now of course he was from California we don't try on

anything and a newscaster got very upset

at this and decided to try and expose him so he sent in a television team out

to show the world that this was ridiculous and the television people

rounded up 35 people who had never played tennis before they signed legal

affidavits and they took them all out in the court found the worst one who is a

55 year old lady 40 pounds overweight 5 foot 2 in a muumuu

and she had the muumuu because she had

not even exercised for 20 years I did

not even have sports clothes they had to buy her sneakers to get her out in the

court and then in front of the rest of the people and live television they said

to this tennis teacher all right you have 20 minutes we can't

give you a completely afternoon this is television after all you have 20 minutes

to teach her how to play tennis on live television I want to show you what

happened here and in the context of which we're talking about see if we can

let's see if we can understand what's actually going on here so let's roll that next clip

not with anyone you know I need someone I haven't done

anything physical in like 20 years I could just picture myself as like just

chasing balls all the time you know never hitting it just chasing it and I'd

heard about the inner game but I really didn't think it would work with me as a matter of fact I had called yesterday to

cancel it because I thought it'd be hopeless but I was talked into kind I'm

gonna have Tim hit some balls to me and as the ball flies through the air I

want you just to watch it the very instant the ball hits the court I want

you to say bounce and the very instant the ball hits the racket I want you to

say hit to just be saying bounce hit all you need to do is watch the ball okay

the key of all the exercises in the inner game is to focus their minds

attention somewhere where it will not interfere with the body's ability to hit

the ball automatically you stand here and Tim will throw you a couple and you

just continue doing the same thing I try to hit you not at first at first just

say bounce when the ball bounces and hit when you would hit it but don't hit then

after you feel like I didn't go ahead and hit it okay bounce yeah

bounce it bow yeah okay when the players

first learning the game of tennis and when he says bounce he's also out of the corner of his eye

watching me hit the ball so even though he's not trying to see how I hit it the

image of me hitting it is going right into his memory you enjoying yourself

yeah it's really fun you like that okay the cardinal principle in any book

written on tennis about your tennis is to watch the ball no tennis player

really sees the ball enough and the reason is they get bored so I tried to

give them something to see in the ball that's interesting so we watched the

trajectory of the ball the the gracefulness of the line made by the

balls that flies through the air I might start listening to the ball more listen to how it sounds in your racket

here laughs yeah here down right what's

that sound like very dull it was like would that would sound people this man

you know he'll hear the sound that you like pretty good just listen first

okay now there's another side okay

now feel where that racket is part of your arm is just feel it who tries too

hard let it happen that's it she was comfortable night every time I did start

to think things went wrong they just stopped thinking then the body

seems to know what to do

[Music] I'll tell you what my we're gonna do one

other stroke now and this one's called the serve and the server is just like a

dance be just like a dance we'll come back here and I'll show you how to do a

dance call the syrup okay and I've done

this dance a little bit so I'll just do it okay in fact you can start humming if

you want the rhythm like this that's

[Music] allowed

okay now anyway you do don't you stand

there shut your eyes that's your eyes and imagine yourself

doing that same dance might start

humming a little bit as you're doing it yeah okay now what I want to do there's

no thinking just go do it quick just no thinking just go serve yeah sir don't

worry about missing keep serving keeps don't think go doo-doo-doo quick that's

it down for it again don't worry about hearing it out loud

let's hear it out loud there you go

I get into home the rhythm of my serve

so they're not thinking oh this is what are you doing with his elbow this is what are you doing his hands they're not

trying to remember all that they start humming and they see themselves doing

very similar things muscular movements to what I did but they're not thinking

about it so it just comes out naturally and fluidly that didn't take long

that's called serving

[Music] all right out

[Music]

Sam s it come down come down okay okay

that's your first point in tennis scores 15-love okay I'm on no need for

you to miss all of a sudden everything became effortless just I was just sort

of like floating along you know and doing what came naturally with me now and I said I'm playing tennis yeah it's

really really beautiful I have a clear

idea of what you want then let it happen don't try too hard I'm gonna try to make

you miss Molly because it's going on too long so all just get a little bit harder

you just keep watching the ball

keep watching it die rootin for you

won't don't be distracted keep on that wall that's fine that's called tennis

[Music]

[Music]

well that happened many years ago and it happened to be it was the key to the

work we've been doing at the Xerox Research Center on trying to develop a

new way for people to communicate with computers we why shouldn't learning a

computer be like that why shouldn't and what what did we learn from that well we

saw from the two faces that we have a visual mentality we have a symbolic

mentality the processes language we saw here that we have a kinesthetic

mentality when we appeal directly to these mentalities then we can get

tremendous results and yet none of these mentalities covers the whole world one

of the big problems in most human relations particularly in business is

people try and do everything with language language of the symbolic kind

and yet there are other ways to solve problems come up with new ideas and so

forth so let me show you how we turned that into something that is very

successful today let's show the next clip please

that was about 1974 and here's the

forerunner of both the Macintosh and our

workstations today a little bit bigger

but very powerful about three times the power of a Macintosh or a 386 computer

it's also playing the music and we did all this stuff by working with children

because children force us to think of new ideas and the way we combine the

three mentalities this is a tool built by a child is to combine doing with

images concretely to make symbols

okay let's roll the next clip here's the result and this is a 22 month old young

girl she's two months before her second birthday both of her parents own a

Macintosh their parents work at home and when I discovered she was interested in

computers I gave her an Apple to which she rejected she wanted a Mac just like

her folks this little girl has never

lived in a world not densely populated by computers like the Macintosh so she

doesn't know the computers are difficult this is part of her environment and

because of the way it's set up it appeals directly to her innate ability

to deal with her environment now even though we designed this stuff

for a very young child I was amazed at

how was youngest was about five here she is not yet to feeling very confidently

with the situation's here but still it's

not too remarkable of course she can point at visible icons and make things

happen what happened next though really amazed me she wants a fresh sheet of paper so

she goes up to the clothes box on the window saves her old drawing using the

pop-up and then using the pulldown gets herself a new sheet of paper and she's

off and running again now what does all

this mean it doesn't mean that all of

the fundamental problems of communication have been solved in

something like the Macintosh but it shows what a fantastic advance can be

made simply by going back to the human being as a creature who is much more

complicated than we'd like to believe but which were starting to understand that

we are created with a kinesthetic way of knowing things a visual way of knowing

things a symbolic way of knowing things we can combine those together then we'll

be able to work our way into the future

no matter how scary it seems to be and as always when we're going and exploring

into a place we haven't been before the best weapon we have to do it is the one

between our ears providing it's loaded thank you very much

[Applause]