Directions in Object-Oriented Programming From Actions to Agents

From Viewpoints Intelligent Archive
Revision as of 20:24, 5 December 2017 by Ohshima (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search
in the timeframe of 15 or 20 years ago Alan Kay conceived of the Dynabook which
most of you probably know was a
conceptual forerunner of the PCs and and and probably some future machines that
we've not not even seen commercially yet Alan Kay obviously as the man who's
always been ahead of his time he was the
inventor of the first small talk some of you may know that there were actually
several small talks the one that you now know a small talk ad was the fourth or
fifth in a sequence of small talks Alan Kay did that first that very first small talk and currently he is a an Apple
fellow and prior to that he was a chief scientist at Atari with credentials like
that I found it interesting to note in his biography that the only professional
Society of which he is a member is the International Society of organ builders
without taking any more of Alan's time let me now bring up Alan Kay
I have a piece
so I found it necessary when giving talks in the East Coast to put the slide
up so that people who might be offended can leave early I thought this might be
an especially appropriate place well I
think a good way to start off is to
recall a statement by William James made
about a century ago about the three stages in the life cycle of any idea and
the first stage is that the new idea is condemned as being the work of madmen
then a few years later it's remarked on as being commonplace and obvious all the
time and the final stage is when the original condemned errs claim to have invented it
I think we're at that stage now with
objects in window oriented and pointing user interfaces which to me means that
they must be hopelessly obsolete so
we're actually we're actually having a symposium here about a patient who's
already dead but in the grand tradition
of computer science will live on for at least a century this is gonna be a
different talk than norms talk norm is a norms talk as a practical talk by a
person who got interested in
object-oriented programming as a way of implementing large-scale complex systems
in the context of a university where you
don't have a lot of people to do the
work and a lot of the work is transient and this this need to deal with
complexity in a way is one of the main selling points in object-oriented
systems and people have this notion that
because object-oriented systems are usually worked on by small groups that
it's something mainly for small groups the main reason small groups use it I
think is because it's one of the few vehicles that a small group can do truly
interesting things in a short time but if object-oriented systems are
implemented in a in a thoughtful way than by their very nature and we'll talk
about this in a bit if the protection mechanisms that are inherent in the
notion of object-oriented programming are also implemented which they often aren't then one should be able to
organize very large-scale projects as well because the whole joy in the
object-oriented design is the notion that when you put an object in the
system it should be able to make use of what's there but its presence in the
system shouldn't disturb any other objects that's in the system it should
be something that is self-contained it should not be able to be destroyed by
the system and so forth and so the
in an object-oriented system that's well designed you should be able to add new
knowledge in an incremental fashion almost linear fashion how we can a
little bit later we'll talk about a couple of the places where that isn't really true and maybe some of the
solutions for it but I thought a good way of starting off would be to just
take a look at a couple of object-oriented systems of the past or
almost object-oriented systems the first one and I understand they're only a few
people here last night to look at it I have an excerpt maybe this is even a
different film of Ivan Sutherlands sketchpad which was one of the earliest
object-oriented systems although Ivan didn't really quite think of it that way
there's an interesting history behind it so let's show that first tape this is
implemented by making this room that's -
the last computer in the u.s. to have its own roof this is what you saw there
was the very first implementation of the rubberband technique and here's something you don't often see even in
graphic systems today he pointed he did a rough sketch he pointed to those edges
and told him he wanted them all mutually perpendicular and sketchpad figured out
how to do it we just saw that sketchpad was also the first system to have a
window this is all done in 1962 there he
pointed to those two lines and told them that he wanted them to be parallel and
now the constraint is to be collinear so
these little dashes will be precisely lined up over the lines underneath this
is the first time when people discovered that the light pin was not a good input device
and in spite of that it's been reinvented at least 90 times in the last
25 years so he's made a hole in the
flange and now he wants a rivet and
again the whole idea behind sketchpad is you should be able to quickly sketch in
an approximation of what you wanted who's going to use that as the center
for the arc and then get the system to complete the drawing by implementing
various constraints or rules and here he's saying make those four lines
mutually perpendicular the doing of that
pulls the diagonal line which pulls the center which pulls the arc which makes a
perfect rivet so this is also the first
non procedural programming system
showing that no matter what you do and of course he could have added further
constraints that would constrain the ratios of those sides so that no matter
how you distorted it always would come back to something that was the same
relative size so that was a master what
we call a class today and here's an instance of that rivet the instances can
be scaled and rotated in real time you might wonder why is a display jumping
around like that well I didn't display
is back then the tx2 here is putting up every screen individually in fact memory
for each separate dot so this is a software oriented display
and when you put up a lot of dots that starts flashing because it's tracking
the pen here are a couple more instances
listen you might imagine led to the development of better displays now he
doesn't like the cross piece on the rivet so he goes back to the master and
says make those invisible they're still there but they're invisible now and lo
and behold we see that the instances feel that change dynamically and
remarkably sketchpad even had a form of inheritance who's actually one of the
first object-oriented software systems so now he's made this constructed thing
into a master and now he can get instances of it I once asked Ivan
Sutherland many years ago how is it possible for you to in a single year of
1962 to do the first computer graphics system the first non procedural
programming system and the first object-oriented software system and he said well I didn't know it was hard ok
stopped them made a tremendous
impression I had to learn it that's 22
now when I first went to graduate school
the desk at the University of Utah Dave Evans had one of these theses in and
hand out and you had to prove that you understood it before you're allowed to sit down and think I might
this isn't worthwhile looking at in any detail maybe I'll just use this from the
main line here this this is essentially
the pathway that I took getting interested in this stuff I was in the
Air Force and there was a file system on the Burroughs 220 that was what we would
call today object oriented back in those days I didn't have operating systems of
course and these days did not have operating systems either back then there
was more excuse for it
and the Meniere's had the problem of how
can I hand data tapes around Air Training Command
they all have Burroughs 220s but there is no standard operating system and so
somebody some genius in the late 50s came up with the following idea he said
well what if we had to take format like this with each long tape record is going
to be made up of three parts the first part is going to be a relative transfer
vector into the second part which is going to be a bunch of Burroughs 220
code that 220 code will know how to
interpret the third part which are going to be our unit records so all you have
to do when you get one of these tapes is you simply read the transfer back to the
front part of it into a special place in core and then you can do anything you
want with these tapes because we'll have a standard protocol in this transfer
vector that's the way that file system worked back then heck of a good idea I
was just mentioning to somebody this morning that the first time I ever did
what we would call object-oriented programming today was done on the 1401
in Auto coder because we used this technique on the 1401 s there as well
and made up macros to allow us this generic access to things
terrific idea why didn't it last well COBOL actually drove it out the adoption
of COBOL as a as a way of getting ahead
actually gave us a weaker metaphor that we're still saddled with today
the borough's be 5,000 actually implemented these ideas and hardware it
it antedate sketchpad and probably sent seven out of the top ten ideas ever
thought up in software are actually implementing the hardware of this machine Burroughs made the enormous
mistake of trying to explain this machine to their customers back then
they actually hired college graduates and taught them how the machine works so they could go out and tell the customers
how good it really was and I was I think it was the third machine that I learned
back then and the these college graduates scared the heck out of all of
these old data processing managers who are just struggling out of punch cards
and the machine was late as well so it just never caught on sketchpad and then
Simula was the first programming
language that I would call trying to be
an object-oriented language and there are two versions of it in between I
designed a language called flex which is
an attempt to simplify Simula and in some sense some real sense the small
talks were an attempt to send to simplify Simula as well but there are
all these other things that fed into it over here which the most important ones
for I think object-oriented programming are the Lisp influences and over on this
side influences that are basically either philosophical or coming out of
cell biology now reason I bring these
these up is because I think it's it's
critical when thinking about object-oriented programming not to look at the the systems that are out there
today and think this is object-oriented programming because in my opinion it
isn't to me small talk eighty is not an object-oriented language because it
lacks one of the things that I thought was most important in object-oriented
design and that is differential protection which means that you should
be it when an object is put out into the world that should be able to protect
itself against various cendars so for instance if you have a
file object but you would like the file object to be able to do is to
discriminate about who's sending it the message to change bite number eight in
the file and it's possible to do that in
small talk but it's not set up to actually do that you can act in small
talk you have to instrument every method that you want to protect in order to in
order for it to do something the earliest small talk it was combined
with what's known as capability protection there was objects were seen
as a generalization of capability protection and that protection was built
in so that no object could command another one to me the biggest difference
between whether you really have an object-oriented system or not is whether you are really sending messages or not
and small talk and Objective C and C++
and all this you aren't really sending messages you're actually just doing subroutine calls which means that in
some sense the sender guy on the outside is commanding the object to do things
and the whole notion is not for doing
commands but for doing something else let me give you a sort of a feeling for
when I started thinking about this stuff
for real after I done a done a few
failures I came to realize that the only
model that I knew of either done by man or in nature that really could handle
complexity in any appreciable fashion was a biological one the biological one
ninety percent of a cell's effort is going into its own protection only ten
percent of the cells effort is going into contribute or less as going into contributing to what the organism is
about and the cells communicate both in the embryo and in the adult organism by
sending messages around that they have very elaborate mechanisms to keep most
messages out but many of the diseases that we fall prey to including AIDS have
to do with cells not being able to keep certain messages out so that translated
to me into take the biological cell
metaphor and try and translate it into machine terms it's like you have lots of
separate little computers running on a network able to send messages back and
forth to each other when you think of this as the ground base of
object-oriented systems then it's very difficult to think of this machine
commanding this other machine because real computers have mechanisms for
holding off commands they don't have to listen to you if they don't want and so
the idea that each one of these objects is responsible for its own protection as
the strongest thing you should ever be able to do is to be able to beseech
please do this for me but you shouldn't be able to say do this for me any more
than you should be able to do the equivalent of an assignment statement so
one of the major things I worked on when I was designing the early small talk was
how can we get rid of the assignment statement as an imperative and replace
it with something that was much more of a suggestion and the amount of state that got changed
if any was completely up to the discretion of the underlying object to
me any language that doesn't implement that fully is not an object-oriented
language because you've what you have is something more much more like abstract
data types where you have a bad old idea and you're trying to put Bearnaise sauce
on it in the hopes that will be better but Bearnaise sauce on a hot dog doesn't
help so the input basic two implications
here is that you have an outside and an
inside the outside is completely behavioral messages can't be commands
and assignment has to be replaced by advise if you have those two things then
you have object or whether you have inheritance or not is not nearly as critical not nearly as critical because
I'm a good object-oriented language you can actually build it and you may very well want to in some form of delegation
mechanism so on these mysteries started
becoming clear when I worked on this machine this is the old flex machine
that Idi Cheadle and I did 20 years ago 67 to 69 this is what it looked like on
its own display and it was influenced very much by sketchpad by angle Bart's
first hypertext system by simulant had a
programming language I designed called flex it had a user interface that was so
well designed it most people reacted to
it the way you would if you opened up the lid of a garbage can and saw something moist and fuzzy down at the
bottom or pretty much the way people react to
APL today I'm really proud of that user
interface of all the things I've
designed it produced probably the most startling reaction but didn't have a few
ideas on it it had a really good windowing system one of the first and I
had multiple windows it was a calligraphic display so the windows didn't overlap back then but a lot of
the things that we're familiar with now in modern user interface were developed
on this machine although kind of ass backwards most of the time so after
wandering about what we should do I realized that McLuhan made this great
statement that I read one day said I don't know who discovered water but it
wasn't a fish and I realized that as a
graduate student I was the world's worst user interface designer because the
whole thing about being a graduate student is to learn a really stupid system with 500 obscure commands just to
prove that you can when you turn a person like that loose on a user
interface you wind up with something like ms-dos
so for a while I was thinking well I need some other context there must be
some other context and I looked at some of the language work with chimpanzees
was rather hard to get hold of chimpanzees the Xerox corporate
executives who would never stay around long enough
so finally for various reasons decided to work with children who are also an
alien species but Palo Alto had had bred
hundreds of them just for our experiments and once we started thinking
about children as the end users lots of things changed because children had to
have something so portable they could carry other things too
in other words 0.5 herniation spur block is not portability so about 20 years ago
I came up with this idea which is called the we call the Dynabook back then in it
my friend Dave Liddell who is here today
when he was at the University of Illinois and worked for Owens Corning
worked on the very first flat panel display which I saw a one-inch square
version of back in 1968 and realized it was only going to be a number of years before we could put all of the silicon
that was in the Flex machine on the back of the display and that this would be
one of the inescapable form factors of the computers of the of the future so
once starting to think about that and thinking about the children as the end
users it meant that things had to be much much simpler than we'd ever done
before and one of the hints I got was from a great system done at RAND
Corporation a Rand probably has contributed more to user interface
design than almost any other single place nobody knows that probably nobody
in this room or maybe only one or two people in this room remember a language
called Joss Joss was designed in the very early 60s on a machine called the
johniac and Joss was probably the best
teletype or typewriter oriented programming system ever done it was the
one that invented the interaction scheme that we now know of in APL of having two
in direct commands that came from Jos basic adopted at many years later and so
forth and Jos had the misfortune of being implemented on the wrong machine or else we would have been able to start
much further along than then a language like basic so this language have been
developed for non expert users who are mostly economists who had to do a fair
amount of computation and the these
users loved Jos fact I get a chance to use it in the 60s as well and it was
just there's one of the few systems that you've ever worked on in which there was
a pleasant little tingle when you knew you were going to get a chance to work
on it for a few hours in the afternoon how many systems can you say that about it was just encrypt there was nothing
left to chance everything was done beautifully aesthetically and so forth
and the only complaint these economists had was they said hey none of us can
type we love this system but we can't type can't you do something for us so
the RAM people went actually went through their waste paper baskets to see
what they did right I guess not too many divorces were caused by that these guys used to write
on legal pads and so forth and they made
diagrams and so the RAM people in 1964 invented the first tablet was called Ram
tablet I remember when we got one at Utah a few years later 1967 one of those
Rand tablets thousand by a thousand stylus pointing device first good one
ever done 1967 that single device cost
$18,000 and you had to hook it to a machine that was worth a half a million
to a million to make any use of it a year after 1965 a guy by the name of
Gabe groaner did the first really good gesture recognizer and a few years later they
put this all together into a system called Grail for graphical input
language let's run the second cut on tape one here's what this looked like 20
years ago
this list takes us back where we may
start to edit the flow diagram the no keyboard first we erase a flow arrow
then move the connector toddled away so that we may draw a box in its place
it recognizes he wants a box and just makes one the printed nine used as
commentary only in this case the box is
large so we make as where modern-day
window control came from literally Android flow from the connector to the
box attached a decision element to the
box and joy flow from it to scan we then
erased the floor arrows attached to the process post new area and move the box
to a new position this allows us to draw
a new box then chop off its corner and
label it sub scan with a residual error
I notice it misses the end here he sees it he has to thing but it's motiveless
so just continues what he was doing it makes the change correct the label add a
decision on sub scan so that control may flow to the pleat the diagram from scan
to post a chance for an hour really
changed used interaction or synching your hands right display the structures
underneath so it wasn't just personal
wasn't just going from the mainframe to the desktop this was intimate because
you were right there there were fewer barriers notice that the command scheme
is not even menu-driven here the commands are actually analogues to the
results you want to have so it's not just icon it's an illogical very powerful set of
ideas again done on a machine almost the
size of this this room but slightly less powerful than a 512 K Mac but this is a
technology that actually got buried we we did many versions of this kind of
stuff at Parc I used to show this movie every couple of months to my group so we
wouldn't forget what it was that we were trying to do and the we're ready now in
the next few years to see a rise of gesture recognition because the micros
are now powerful enough to do this and still have some cycles left over for
doing work for the for the user and you
may not know it or not but actually one of the best groups in the country in doing a handwriting recognition is at
IBM big surprise but you actually have
one of the better technologies here and
so if you're you're interested in this I suggest you look in the IBM phone
dictionary for the group that is doing it now John Sculley was once wearing how
many scientists that were at IBM and I said yeah John there are many more
scientists at IBM there are employees at Apple but don't worry IBM doesn't listen
to them just like it Xerox you know you
want them to let you alone when you're working but then you want them to come around when you've got something
it's hard to get both to happen so I
want to intertwine some stuff in here because in although this isn't really a
historical thing it's very to me it's very important that the stuff that gave
rise to what this symposium is all about was not at all disconnected from the
fact that there were going to be users working on the on a system and so the
the whole thing that brought forth real object-oriented programming and small
talk and the user interface that went along with it was that we had these children and we wanted these children to
not just be able to have access to things but we wanted the children to be
able to build things so our if this what we said was is if
this is if this were print literacy we don't want to teach the kids just to
read print literacy isn't just reading it's also writing we want the kids to
somehow be able to write and the equivalent of writing on the computer is
not necessarily programming in the in the COBOL sense but it certainly is
building structures of your own and so very early we got into this whole area
of wondering well how does the mind work
if at all and the first thing we
discovered that whatever common sense notions we had about the mind working
they certainly weren't at all in accord
with what had been discovered in the last couple of decades
for instance common sense idea that's been in history for thousands of years
is that because we think of ourselves as an individual we think there's a single
eyeness and even in the work of Piaget we see in thinking of the progression of
an infant mind to an adult mind as something like the transformation of a
caterpillar into a butterfly that you go from something less form to something
more form but it's all made out of the same stuff whereas the the truth seems
to be quite different and quite startling that our mind is really made up of a kind of a patchwork quilt
modules or objects if you will and these
objects indicate imperfectly because they came about at different biological times for
different evolutionary reasons and it give you an idea of the modularity of
the mind let's take a look at this and let's see who in the audience can spot
what's wrong with this picture besides the fact you've all seen ups and
upside down faces before who can see
smile is what upside down and what else
these teeth are and what else the eyes okay so now if we were in school and I
were a teacher I would say to you this is the way things are taught in school
by language I would say that what we've
done here is we've taken a picture of a young girl extracted the eyes and mouth
turned them upside down put them back in with an airbrush and then turned the
entire picture upside down okay so having said that you should be perfectly
prepared for what it looks like right side up
now I'm gonna turn this back as I've discovered discovered nobody will listen
to me when it's on the other way so
that's interesting that I gave you the exactly the same amount of information
two times one through language and one visually and you had a completely
different reaction to the exact same information that's the kind of thing
that perceptual cycle and cognitive psychologists really love because means
that different information going into different channels is going to different
parts of the brain and you're having different reactions therefore there are modules in this particular one the
module that's causing this effect the two modules are actually known
physiologically they're located on the under right hand side of the brain where
you do face recognition and about an inch apart from each other one of them
is a expert in recognizing face like things as the thing allows you to see
faces in the clouds when children get scale scared at twilight with menacing
shadows and so forth that module is going off saying I see a face like thing
right now it's reporting I see an upside down face like thing about an inch away
is another module that's only interested in eyes and mouth it doesn't care where they are and it's
reporting that's a good looking mouth and those are okay eyes and so the two
descriptions basically have very little interaction they're quite independent
reporting's from two different modules now when I go to this version the face
guy is saying that's a right-side up face like thing and the eyes and mouth
guy is saying something is very wrong here
now you can imagine why nature might want you to be startled see a face like
this up close to might mean you're going to die on the next 10 seconds and in
fact it's this is not something that you learn it's actually built into our
nervous systems from birth into most mammals in fact so if you try this
experiment out on a dog I suggest you pick a small one
[Applause] so important idea here is that there are
modules the just what the modules are is
still open to debate some of the modules
are even metaphoric I'm not all of them are physiologically cited because of the
way the the nervous system is organized they can act we can have modules that
are distributed all over the the brains of going ideas like left brain and right
brain are a particularly weak way of trying to characterize it but most
important thing is once you realize that there's more than one module and that
the modules have different ways of thinking about things and are somewhat
weak recalcitrant and unable to talk with each other then you have the basis
for a much stronger user interface design or curriculum now the ideas that
led to this user interface that norm was talking about it was developed at the
same time as small talk came from a Harvard psychologist by the name of
Jerome Bruner and Bruner had redone a
number of the experiments of Piaget but
with some twists so for instance Bruner took a child he was in this middle
Piaget stage the image stage and is
where you take a squat glass of water and you pour it into a tall thin glass and the child says yep this was more
water in the tall thin glass and what Brenner thought to do is to then occlude
the tall thin glass with a piece of cardboard and ask the kid again and the kid said oh wait a minute there must be
the same amount of water because where would it go and Brendan would take it
away and the kids would say no look there's more there's more Brenda would cover it up and they could say well hey
wait a minute so if you have any six-year-olds you'd like to tour man
and burner did booty did many experiments in the 50s and early 60s and
the conclusion was that what we didn't have was a progression of stages of the
caterpillar going to the butterfly but instead what we had were multiple mentalities and a change in dominance
because brunner could show the existence of the previous and mentalities to come
by various tricks and Brunner said of
well there are lots of different mentalities and he picked three that he
could show the existence of and concentrated his curriculum ideas on
those and those three were a kinesthetic mentality identical with the PJs first
stage where a whole is to diggit an object is to grab it the iconic
mentality that we just talked about and then around 11 or 12 a symbolic
mentality where facts and logic are used or at least in Swiss French children
around 11 and 12 American children have
a very difficult time ever getting to the facts and logic stage because we
have an enormous iconic barrier between them which is called television now the
existence of these three mentalities they each they each have striking
characteristics and in a nutshell as I
can only go into here the kinesthetic mentality is the one that places you in
space not your visual mentality as you
might think because you can blind a person person can be blinded and still
have a perfect sense of where they are in space but if you remove the for
instance a person's limb from their body map as sometimes happens in an accident
then they may have a perfectly healthy leg attached to them but they cannot
feel that it is there and they they relate waking up in the middle of the
night with somebody else's leg in bed with them and in fact they have a very hard time looking at their leg to see
that it's actually attached to their body there's kind of a blur there
because of course we see not what our eye sees but what our brain knows we see
a simulation that is pieced together and our brain has had its knowledge removed
that this limb is part of us and is very hard to actually what it what it actually yes so the
kinesthetic sense is the one that places us in space it is the one that when we
when removed is the most alienating contact one is the deepest one the
oldest evolutionary one the tactile
sense the image one has two main characteristics that are worth noting
one is the ability to remember images after only a brief exposure is
astounding hopper who is a psychologist in the 60s
in several experiments where he showed 2,500 photographs spaced 10 seconds of
parts just a bunch of slide trays for hours until I seen 2,500 images all
sorts of random pictures and he discovered after 30 days that all of the
experimental subjects had a better than 90% recall of those random slides and
could even identify occluded parts on slides and so forth this is an
experiment you can do yourself get a friend to manipulate a cable TV movie
channel and have them dial into a movie
you haven't seen for 18 years and see how many frames of the movie you have to
see before you can identify it most people can do it in under 90 frames
three seconds of exposure is quite
enough for them to say yep I've seen this movie before and I know it's gonna here's what's gonna happen next I think
about that means 18 years ago you saw this movie you're being put into the
center of it without any preparation and stuff and for some reason your visual system can organize all of those pixels
or what the pixels have turned into to be able to perform that recognition so
it's striking recognition is also fast
so you put a hundred pictures on a bulletin board you can find a specified
picture four times as fast as you can find the word in a list of four on the
list of 100 words so it's much more efficient and this recognition in
comparison and stuff is done by a completely different part of the brain
visual system is also the mentality that is least blocked because it's very
modus operandi is to flit around and take in different parts not to become
fixated in one thing in the visual scene so it's there are many things you can do
that in which an access to visual scene
augments the short-term memory and boosts creativity then finally the the
facts and logic system which is what we concentrate on in school unfortunately
in its strong in in many ways because of
its abstractions but it's actually the weakest mechanism that we have it only
works when there's a strong context so when we're thinking about these ideas
but we had this basis from Brenner and it took us three or four years of
experimentation to come up with this
model that what we what we really should
do this is something has been misunderstood about this user interface because it's always talked of as the
graphical user interface but in fact it was an attempt to combine three
mentalities into some kind of synergy so
the doing with images make symbols says there's some kind of progression which
Brenner identified as well that if you learn where you are learn things tact
alee that gives you a much better set of
hooks for your image system to put things on and then at the very end you
have something to abstract that you want to princess if you want to learn physics
the best way to learn it is body physics and visual physics before you start
looking at calculus then you have intuition and Bruner claimed that and I
believed him that what we call intuition is simply these mentalities that don't
talk sort of vaguely whispering to us
somewhat variant thoughts than the than the one that the one we call eye
actually does so if you take those ideas you the way we picked and we didn't even
like the mouse but as I said tablets were expensive and so we took angle
Bart's Mouse and made a better one and let that be our pointing device if you
believe what I'm saying about the user interface then would suggest number one that we should
have better pointing devices and number two that the pointing devices should
start to give us some force feedback back into our hand because then the
kinesthetic contact would be much greater and the windows and icons are
for the purposes I've just mentioned and we came up with small talk to be the
underlying symbolic framework for all of this stuff this is why the on the
Macintosh and the Macintosh imitators that the pointing scheme is not
functional you don't give the command first and then be stuck in a mode and go
out and then try and pick parameters for it what you do is you do it exactly the
same order that the you do object-oriented programming and that is
you designate the objects of interest first and then you send the messages so
translate it to user interface it says selection first messages later and that
gives you an approach to doing modeless interaction and here's an interesting
idea that we implemented at Parc but in
fact has never made its I just cannot believe that this idea has never made
its way out into the out of the world because it is a direct implication of
object-oriented design the implication of object-oriented design is not cutting
and pasting that is absolutely at odds
with an object-oriented user interface the this is the implication of
object-oriented design this is a document that has a variety of different
paragraph types in it one of them holds texts and other one holds pictures and
these are as norm was pointing out these are the applications themselves so you
don't go out to another app you don't go out in this system you didn't go out to
a painting system to make this picture the painting was actually developed right in the context of the document
because you just started a new paragraph type there and what you see here is a
this is a page from the Lord of the Rings and the cursor has just gone into
this picture and what has happened is it's sprouted a menu because it thinks
you might want to edit it further so as you go into each each paragraph type it activates the sub
activation if you the sub if you will so
that all of the things that you're working on are in the context of your
result it's the thing we call results mode interaction which i think is one of
the most important aspects of WYSIWYG that hasn't really made it into the
outside world always keep the user working in the context of their results
that means they shouldn't go out other applications to get values and bringing
them back they should be able to develop those the values that they want from
within the final thing that they're working on
now this next in order to talk about
this next next part of the talk a little
more context if we think about the two ways that humans have extended
themselves over the last several hundred thousand years one of them is by making
amplifying tools physical tools like the lever and the wheel figurative tools
like mathematics like language itself and so forth all these I think of as
being extensions of manipulation but even mathematics which we think is
something fairly abstract is actually a
way of taking the hopelessly abstract making it into concrete things that we
can then move around so all of these things I think of as being manipulation
the N word is manipulation and the other one and of course the Macintosh user
interface is that style it's basically make everything into an
object and was norm mentioned if you take things that are verbs in English
and make them into objects you have a much better time of it because of course the universe doesn't seem to have any
verbs right English has invented to talk
about horses and cows and plowed fields and so forth and it didn't anticipate
modern physics which as far as we can tell there are no abstract verbs out in
the universe instead what we have are things that are much more like objects
in process so when I hear a sentence like the clock runs down
or the boy runs down the only way I can come up for any meaning of what runs
down actually is all about is by thinking of it as an attribute of the
subject of the sentence okay and there
is enough metaphorical similarity between the two items so that we can use the same word but I always think of what
we think of as a verb as really an attribute of some subject otherwise the
sentence wouldn't make any any real sense and that is an accord with with
modern physics so the other way we've
extended ourselves over the years what I call goal cloning goal cloning means
that we simply are able and willing to go out and convince other people to take
on our own goal structures that's how we get most things done so what's going on
here we have a group that has for the moment has fewer goals than the sum of
its parts and companies are that way
countries cultures and so forth sometimes the methods accrued as in the
way the Egyptians got the slaves to build the pyramids sometimes they're slicker like in the
modern television advertisements mumford called this building a mega machine he
pointed out that over most of recorded history most machines built by human
beings had human beings as parts and the M word for this to me is management that
you manage that kind of process and we manipulate objects and we're going to
see a transformation from actions having to do with objects into agents which we
are going to manage now let me give you the examples the first is sort of an
example of what you can do with a tool
and what a tool really means so let's show the I want to show tape number two
this 2012 young girl who never lived
world not insulated by Akash her mother's my aunt both are there and
father work at home if another father as an act to her this is an acknowledge
that one around your warrant we do think and some paper technology is what Israel
never see anything I fast use it yes in computers I Karen told she injected and
I don't mean you sir today my did one with her anis governing athlete for here
has been stolen disks even though the
belt is you inter favorite children I never even it is about child anywhere here than I was quite intrigued yeah
yeah here says doing visible you man acne and of course you don't have to get
them out I thought well I'll go along
this is arrested but wasn't me impress until I saw what havin a she wants a fresh sheet paper so she used a closed
box in the window and user Ultron's the pop up and goes to the plan F rating
okay implications here it's kind of
interesting it do the user interface at all an accord with what's actually going
on inside of people's head then you get something that can actually extend all
the way down into childhood and be to them as fundamental as the marking
instruments that we grew up with and I think this is very important because I
believe that the the children that can form themselves by being around this
media as media are going to be thinking
and doing things very differently than the way we do this is another example of
in the long line of how we act smarter
by changing our representations we go from Roman numerals to Arabic we go from
Ptolemaic the Copernican every time we change the representation system our
weak systems of logic are made that much
stronger and I think of user interface main aim is not to present but to
represent and it's figuring out what those representation schemes are going
to mean in terms of leverage they give the user this is the most important point that point has been missed in most
people who are doing user interfaces today including a lot of the people at
Apple and certainly the the Apple imitators because they think that the
whole job of the user interface is to present old-style applications with
multiple windows and pull-downs and so forth nothing can be further from the truth
most important aspect of the multiple windows stuff is not to have different
applications up there because as I pointed out that's not the way you want to go after different parts of the
things the most important thing for having multiple windows is to be able to
present different points of view on the same thing marvin minsky says you don't
understand something unless you understand it in more than one way and
almost no one who uses these user interfaces thinks to sit down even
though Apple showed them in the finder that you want to have multiple views of different kinds of things thinks of what
the leverage means is to be able to look at the same kind of thing for multiple points of view and what it means
pedagogically is once you've seen that you realize that if I only know one
point of view about something I don't know it in fact I'm likely to be misled by it so now let's take a look at the
other way that's just starting to come
in and think of why it's going to why are we going to have management of
semi-intelligent computer processes now
I think it's because we're gonna we're starting to be networked up and there
are a whole bunch of things that networks give you as opportunities that
we simply don't have the time and the scope to deal with give you an example
when I was a kid I used to get questions answered by going into the into a
library and once I was visiting some friends in Washington DC and I had a question so I demanded to be taken to
the library I'd heard there's a library in the town and we got to the Library of
Congress and I discovered that the card catalog which they didn't even allow you
to go in was much larger than my home town library and instead of
having you do the browsing they had people who would meet you at the door
their job was to absorb what your goals were and then apply their expertise of
the facilities to that so they became your agent and they even would energize
other agents so my my problem turned out
to be fairly interesting for one brought in by a nine-year-old and so he had five
or six adults scurrying around trying to find the answer to it and it left a
lasting impression and this notion of
agents is as old as computer interfaces itself it goes back to McCarthy's
original advice taker paper it basically says that when you have an information
utility there's going to be an embarrassment of riches organized in a
way so that no human will have the time to go after the stuff no human will be
able to remember all the things they might be interested in so something else has to go and do it now the problem with
agents in the user interface sense is that AI has not progressed much since
1970 it was extremely difficult to build an agent I have built several over the
last eight years or so just to see what they would would be like and they're
excruciating to build one of them was a system called News peak at MIT that
stays up all night trying to find for you the newspaper you'd most like to
read at breakfast so it what it does is
to log in to a dozen or so information
sources New York Times Associated Press Nexus Reuters looking for articles it
thinks you might be interested in it reads the articles to the best of its
ability if it sees a famous name like Mitterrand it has a video disc of 45,000
pictures of famous people it's like a newspaper more get picks a picture of
MIT Iran did have there's a map disk with 37,000 maps and so forth uses an
expert system to format the page and you wind up the next morning with something
that is very much like USA Today
and in often with about the same amount of content
but there are some differences because the headline might say more fighting in
Afghanistan or it might just as well say your three o'clock meeting is cancelled
today because one of the 12 news sources it logs into is your own electronic mail
it reads it while you sleep and if it spots to anything that looks important
it puts that in the newspaper the next morning a sidebar might say your
children slept well last night because remember what we call news is entirely
dependent on the communications system that we have in the late 1700s the news
was all about depth the Federalist Papers were published in the newspapers
of New York City something would be unthinkable today actual essays but
people had time to read them when the telegraph got invented all that went
away because they couldn't possibly have you reading anything long that had to be
thought about and understood because you'd still be working on it when the next news came along so currency and as
McLuhan predicted as soon as you start hooking up news to electronic media you
start getting fragmentation so that's
the idea of an agent that newspaper is something that people would like to have
but nobody is going to sit up doing it even with the best sequel type database and so forth
because it's very hard to even remember all the things that you're interested in so the agents also are going to be
somewhat anthropomorphic this is a real problem because they aren't going to be
very smart when we start off we have to in the user interface design somehow
subliminal ii project exactly how smart
they are and still make the user now we
want something like a puppy dog that goes out and gets your newspaper most of
the time it only occasionally makes a mess on the floor so let's let's take a
look at this is a video we made for John
Scully and we start off at the 1988 Winter Olympics in HyperCard HyperCard
stack that ABC television used to keep track of things and then we project four
years into the future and see what things might look like in 1992 with the
user interface that is still mostly manipulative but has a little bit of the
of the management aspect to it let's roll tape number three please so much
this Bob's math Calgary if you click on
any Olympic site you see a list of the events scheduled for that site you can
get more information about any event by clicking on the name let's say ski
jumping background maybe you can brag to
the last or the main map and then
continue exploring here at the University of Calgary I see the figure
skating events are running now that's a sport my family's interested in so I'd
like to learn more about it the previous Winter Olympics took place at Sarajevo
in 1984 when Macintosh was barely a month old
imagine what Macintosh together with hyper card will allow us to do in
another four years by the time of the next Olympics
[Music] we will require powerful techniques to
help us find and manipulate volumes of images text music video and applications
we might by that time see the beginnings of intelligent agents or perhaps by then
they'll be only semi-intelligent Phil
today is March 3rd 1992 you have several data service articles ready and you have
ma'am this is my agent I call him Phil Phil
open the 1992 Olympic stack
the 1992 Winter Games are being held in Alba V friends let's get a closer look
at the town let's look at ski jumping
again we can play the video
[Music] we could learn more about this sport or
go back to the math let's look at figure skating
I'd like to see who's on the USA team and I'd like to see Debbie Thomas I'm
looking for a spin
like that let's go back alan kay was
explaining why skaters go faster as they tuck their arms in he showed me a
simulation to illustrate the underlying principles let me call that up Phil add
the physics draw toolbox which tool box
physics draw let's see I'll need a
counter we can make the window
translucent let's step through the spin
and count the frames okay that's one
complete rotation now I'll draw a model
representing a skater with her arms extended select the center axis and
rotate how fast to rotate this speed and
here's an overhead view of the figure let's hold on to that earlier Allan and
I created a similar model where the skater has tucked her arms in and
appears to be spinning faster Phil display skater a model 1 let's compare
these two models
the small one the one with the shorter radius seems to run faster but let's
check that I'll set them both to run for three tenths of a second a small one
turns farther around the circle but let's see how much distance is actually
covered I'll straighten out the curved paths to measure them both paths are the
same length even though spin increases the distance
traveled is the same the luge event is about to begin show it live or record it
no I'm not interested in luge I don't understand I like skating
I don't understand don't show luge don't
record it I understand
I'd like to say this simulation Phil start a link Phil show the stack map
too far okay show recent link to this
background now anytime I'm on a skating
card I can link to my physics simulation by the time the Winter Olympics come to
a birdie in 1992 CD media will be well
established each of us will be able to choose our own path through Worlds of
entertainment and information and tools like those that we've been seeing will
allow us to integrate the full range of sound video and text information this
demo ends here John should I show the
pre-recorded speech or will you give it live humane things to think about it
here are that although we are of course
going to go to hypermedia and it's going to be object-oriented it's really not
going to be complete until the user is able to do active thought experiments
experiments most hypermedia up till now is assumes that the user is going to
navigate around stuff that may be active
but the user isn't going to be able to do a lot of creation themselves this is
a critical thing to to realize and that
the agents of the future are going to be helpful mainly for their ability to work away from us not from the amount of
comprehensiveness that they have the best object we have for exploring this
new area is the one between our ears providing its active thank you
tell some questions yes yes
well you can do that but just consider
the problem of a system file that some people have access to and other people don't have access to all right there's a
perfect example of when somebody's what you really like to have is when somebody sends a message not a command but a
message to that system file you'd like to have some way number one of checking
the access privileges before you do anything and number two and you'd like
to be able to do that checking quickly okay but even being able to do the first
would be good whereas in the current things that are called object oriented
languages you are already committed you've already formed an activation record because it's a procedure call you
are already inside the method and thus each method that is sensitive has to
have the responsibility access you can
put that in you can put all through a procedure and you have to remember the next time you do that it has this
problem that isn't all I mean languages
also allow your things that allow you access this way I'll talk allows you to
turn a thing called a block that block is a particular kind of object that it's
a block context actually that by virtue
of what it is it has access to some of the internal variables of the object and
so it's you can create Trojan horses and
so forth and but basically what happened is when they were implementing the newer
versions of small talk some of the older people who worked on it didn't work on the newer ones number one and number two
most of the newer people had never done any operating system type stuff and so
the operating system type stuff that was in the original one didn't get put in as
the later the later small talk went back to be looking a lot more like similar
than the original one did the original one when you sent a message you weren't
calling a procedure at all all you were doing is knocking on it and saying hey
you know it's the equivalent of the postman leaving a thing that says there's a message waiting for you down
at the post office if you would care to get it it was that arms off relationship that
small talk 72 had that allows you to implement all the operating system
protection ideas that are important and I think that the the ability to
implement large systems in object-oriented languages as well as
being a being able to implement them conveniently is going to have a lot to
do about how much protection integrity there actually is it's easy to do you
just have to know that it's important and put it in the in the system yes
yeah well I think again I had a there's
a little technical part of the talk that I left out that but I think that one of
the things that's going to happen is that the way we're thinking about objects now just even as people are
struggling to use them one of the when
you look at object based systems today they're mainly modular in terms of state
right the state is sealed off and sometimes it's not even state because
you can have procedural interpretations for things and the thing that's not
modular in object-oriented systems is control in fact sometimes gets worse
because you can have what's called a buck passing theory of control where you
constantly pass and control all over the place and eventually it gets to somebody
you can handle it and they handle it and it's both through inheritance and also
through containment that that happens when you when you start looking in the
system that's been implemented that way you start realizing whoa all of a sudden you have to start understanding things
again and the whole idea of object-oriented programming is be able to program things without having to
understand how everything works it's like Prolog it's like magic for the
Baths the first three hours and then all of a sudden you have to understand ten times more than you ever did so it's
it's kind of a cruel trick and if you
look into biological systems again the first small talk was designed
more this way it was less procedural in
many ways and it was hoping to use a control structure invented by Dave
Fisher who did a great thesis at Carnegie in 1970 called the control
definition language that was one of the that was one of the inputs to the design
of small talk and there he was very interested in what you might call a
gorrik computing and the the current
stuff I've been fooling around with our systems that don't send messages but only receive in other words one of the
ways of getting around the control problem is to if you're a module what
you do is let the system know what you're interested in it's up to the system to somehow get that in for
to you you don't care how what you don't want to have is a system which every
time a new object goes into it some other object has to know it should send
it a message to write because then you're right back to something analogous
to the data structure problem having random procedures understand the formats
of data structures and here we're talking about having objects have to
haven't understand essentially control formats so I've been working on a design
of a system that only receives but it
receives by indicating things that's interesting interested in and then the the system constructs a structure that
will guarantee to deliver that information to it in a timely fashion and again I think if you just get away
from software systems and think about networks again it will tell you a lot
about what to do with objects because you really you can't take objects
seriously unless you think of a transmission line between them it's too easy to just think of sending a
procedure call to them and you don't have any of the interesting things that have happened with concurrency and
synchronization and protection and all of those things that are really going to
be a major part of just standard
programs that we're doing in the world so I think you just think of it as being a whole bunch of net networked computers
crying to cooperate and realize and what the problems there and how you want to
write the code then I think it tells you what to do with objects that was the way
the original small talk was designed anyway yes
right good question the I don't actually
have any of those things in mind but when when Smalltalk 72 was done 16 years
ago the idea back then was what
capabilities could do if abilities are a
short way of dealing with access rights
okay and the access rights there our instance are at the instance level in
capability systems and I wouldn't think that trying to protect at the class
level is C it's easy to protect the class if you can protect at the instance
right because the class is an instance of class class that's trivial to do it
so you want to protect at the instance level its itself and protecting at the
instance level in capability systems is tantamount to protecting each message
individually because that's what a what a capability map actually does for you
so it's a very nice way of doing it yeah
sure no I don't see any if you can deliver it
for that price I think he will sell a few but as far as I can see there's no
in fact the interesting thing is the user interface a lot of the things in
user interface are more like small talk than than than the Mac he has
hierarchical browsers in it and the proportional scroll bars and other kinds
of things that small talk and although
he has a very strange scroll nose we saw
at window resizing method apparently I haven't actually used the machine yeah
but somebody told me you can only resize one dimension of the window at a time and he would would have known that Apple
didn't own that if he'd seen the the Grail movie because
well again in the longer in a longer talk what when I started thinking about
it I realized I didn't have not only didn't have any good ideas about AI that
I'd never had I had had a good idea in AI and I figured that was either because
I was completely stupid or else I was thinking about it in a way that wasn't
terribly fruitful or the stuff that
people have been doing in AI didn't have any good extrapolation into what would
be powerful I still don't know which of those three is the is the case but I
have a couple of degrees in biology so I started thinking about it from biological stand the first thing that I
realized was that trying to essentially do a Beethoven which has been one of the
main aims of AI who knows to do some superhuman
intelligence right off the bath to impress the Pentagon sponsors is a real
loser because we can't even do a decent ant and so I got basically I got
interested in thinking about what would it be like if you forgot about forget about Beethoven for a while what would
it be like to just try and understand a I as an architectural problem which I I
do this is I think of life as an architecture right the material moves
through us we recycle the atoms in our bodies every seven years or so but the pattern remains some kind of a weird
feeling to think of matter passing through this force field or so that
organizes it and the same same thing is
likely to be true for intelligence it's it's an architecture and when you're doing doing architecture you have to
start building things and seeing which cathedrals fall down and you're really
in bad shape in architecture if you build dog houses because you can build a doghouse out of almost anything and it
won't fall down and doesn't scale you know the big problem with our whole field is big good ideas don't often
scale and we always get screwed when somebody has a good idea that works in
the small and then they try and scale it off by a factor of thousand and all of a
sudden anymore so the to me it seemed like the
an interesting thing to do would be to try and build some biologically relevant
models of animal intelligence and pick
at a bunch of different levels so that you wouldn't get into the scaling bug so
that you actually thought you were developing some something that might
give you a few hints as to what the what a good architecture for intelligence is
I joined up forces with Marvin Minsky at MIT set up a project called the vivarium
which it involves children because the again it's it's hard to design for
adults it gets boring after a while so the project for children wound up being
saying what would it be like for children to actually build these animals
what would it be like for a child to sit down and build up a both a physical
animal and a mental animal that you could put into a big environment if we have one more second I can show you a
little clip of some of the work that we've been doing this is will be taped
six and what it is is before you show it
what it is is some real stuff and some fake stuff the real stuff is all of the
3d graphics that are in it which we did in real time on a an Evans & Sutherland
flight simulator this is the machine that has something like 3.5 gigaflops of
computation it's about six to eight Cray
XMP 48 worth of stuff so it can actually
generate on a database that has some sub planetary in size
you know because you have to have to fly over hundreds of miles of it at Mach 3
and it's able to perfect anti rastering
perfect texture mapping on the thing and it has it has the ability to have as
many as thirty two independent cameras at once so you can see look into the
same database from multiple points of view so all the stuff that you see in in
this user interface at that level is real and then what we did is a little
mock-up to see what the user interface some of the elements of the user
interface might be for children we're using elementary school children so
they're both visual and very tactile so we got very interested in what would a
tactile kind of user interface be particularly one without any menus in it
so when you so I think you'll be able to tell the difference between the two so
let's roll that tape number six so here
just turn out the lights so here we have actually four desktops in operation
under projects and we go into the vivarium one and here's the flight
simulator we have two different points of view two camera controls as you see
there so this is 60 frames a second
real-time computer we wanted to do it all in real time to keep honest about it
and one nice thing about real-time graphics is you want three minutes of
real-time graphics you it only takes
three minutes to generate so here's the
little kids notebook going rockfish you
see the real real crab over there and the simulated crab and the crab model
that the kids working on here's the sea
snail weird color it has is actually the
color that's how we do scrolling
and here's an avoidance reaction this is what a real one looks like really an
amazing amazing color and down in the
sales simulation here it's just chucking along on its basic movement cycle these
these snails these particular ones have only about 2,000 neurons so here we open
it up a little bit kids love that
various brain models there's three or four different points of view here's the
one that they would use to program it with all the sensors and now if we put
them all to work get to see what the whole environment actually looks like
this about two years into a six-year project where we basically built all
this stuff by bite simulator brains of
the animals we built with a bunch of thesis projects at MIT that Marvin
Minsky and I are supervising the next phase of the thing is to do a system now
that children can actually build the the animals and put the brains and so we're
now the the idea is about two years from now that is to start testing that aspect
of it by the end of the six years is to exhibit sort of an animal building kit
with brain models that ethologists would not turn up their noses at thank you
very much [Applause]
we're gonna take a break and it'll be
ten minutes please return in ten minutes thanks