X-Message-Number: 8657
Date: Sat, 04 Oct 1997 11:25:52 -0400
From: "John P. Pietrzak" <>
Subject: Re: CryoNet #8652 - #8656
References: <>

Hi, sorry I've been delayed in responding lately, I'm trying to cut
down my time reading e-mail at work...

Thomas Donaldson wrote:
> Some comments for John Pietrzak:
> 
> 1. First of all, neural nets do NOT learn categories.  [ The net
>    reacts to input, the human describes the category. ]

I believe I may have a more liberal definiton of categorization than
you do...  Basically, if someone imposes a partitioning on a set of
objects, such that some objects fall into one group and another set of
objects fall into another, I call those groupings categories, whether
or not there are names for each category or any meaning to why the
partitioning was chosen.

>    This is actually what we ourselves do in the early stages of
>    learning a language. We don't so much get a verbal definition of
>    a word as learn to recognize its instances, which is not the same
>    thing. Later on, we may have many synonyms and so play around with
>    verbal definitions.

Indeed, modern neural nets are also quite good at learning words; I
believe most of the best speech recognition systems I've seen are based
on the technology.  I'm not arguing with that; but again, I consider
that a categorization task.  What does each word _mean_?  (If, by
the above "recognize its instances", you mean associating visual cues
with the word, I argue that you've just pushed the task down a level:
i.e., when you hear the word "ball" and see a ball, you associate the
word with the object.  Now, how did you come to understand that the
"ball" was an object in the first place, when you saw it?)

> 2. I do not intend to defend everything Searle has said.

Sorry about my Searle outburst.  I may take issue with the Turing Test
at times, but I really hate Searle's stuff.  His carefully-constructed
unanswerable questions have wasted a great deal of peoples time, IMHO.

>    That problem is simply that recognition is fundamental to
>    everything we do, and cannot be explained purely by algorithms, no
>    matter how complex.

(I would argue that a neural net running on a computer, performing
recognition tasks, actually is an algorithm, but I suppose that's not
what you're driving at here.)

>    First we learn words without being able to define them other than
>    by pointing at instances. THEN we learn how to define them with
>    other words. Searle's Chinese Room is an attempt to show this
>    problem: sure, I know lots of symbols, and can even answer one
>    train of symbols with another. But as for knowing what they MEAN,
>    in the sense that I could apply them to say something about the
>    real world, I'm helpless.

I agree with you.  What you've just stated is exactly what I've been
trying to say as well.  You can't operate in the world as a human by
just learning categories, by just letting other people tell you what
the world is.  You have to start out with something already defined
within yourself.

For example, consider a human who has never learned any words before:
for him, what is an instance?  If I say the word "rabbit" and point to
one of the rabbits digging holes in my lawn, how does he know I'm saying
the word for "a fuzzy long-eared mammal that hops around", rather than
"an ugly hole in an otherwise fine lawn"?  For that matter, how does he
know I'm not using the word for "fuzzy", "long-eared", "mammal", "to
hop", or any number of the other innumerable objects or concepts which
are all instantiated on my lawn at that moment?  He needs some built-in
bias to assume that a particular region of your visual field is the most
likely thing I'm talking about, or we'd never be able to communicate
within our lifespan, because there's just too many different things we
could be talking about.

>    I will add, though, that I differ strongly from you as to whether
>    we will SOON have "simpler devices able to do the same things that
>    neural nets do now".

Hey, now, that's NOT what I said. :)  (From Cryonet #8627: "Personally,
I believe that programs with much less sophisticated systems than a
true NN will be beating the TT regularly in the near future.")  For
doing what neural nets do, it's generally best to use a neural net, I
agree.  I just don't believe that beating the TT requires a neural net.

> One major point. Several times in this conversation I have said that
> I'm perfectly happy with the notion that we might build a DEVICE that
> could not only pass the Turing Test but even respond like a human
> being in the real world.

To be honest, this seems trivial to me: there are at least, what, five
billion devices in existence in the world today which can not only pass
the Turing Test but even respond like human beings in the real world.

[ On prediction and logic ]
> If we know what we are doing, such systems have proven very useful,
> but they should never be identified with the world itself. They are
> constructions we have made so that we can predict and try to
> understand, and exist only in our heads.

Fine.  Doesn't help me one bit, though.  Whether in the world or in
our minds, there are a vast number of structures out there which can
be combined or grouped in an infinite number of ways.  In order to make
any predictions or communicate with others in any way, we've got to have
some way to make the task easier to begin with: and I argue, that's by
already having a built-in context or bias in our minds, to be built
already understanding some concepts in some degree.

[ Next note, on flavors of Turing machine ]
> I forgot to answer another one of your claims, to wit, that working
> with a finite Turing machine would be worthless since the sizes and
> capabilities of computers are changing very fast.
> 
> I really don't understand how that could be a problem. You use these
> algebraic things called VARIABLES.

Ok, fine.  Let's use variables then.  (NT for tape space, VT for time
taken moving over tape, as you stated.)

> Then you can do such things as study whether it can perform a given
> algorithm more efficiently with large NT and high VT or whether only
> a high VT is needed.

What is a "large" NT?  What is a "high" VT?  With a Turing Machine,
you can encode most of the algorithm directly into the FSA for many
problems (those which are solvable by an FSA), and this requires no
tape space, no tape movement, no reading or writing to the tape (beyond
reading input and writing output); thus, these have 0 NT and 0 VT.
You can also encode the FSA in better or worse ways to solve a
particular algorithm: for example, a universal TM will always use more
tape space and spend more time going over the tape, in that the input
to it is both the input for the problem and a description of another TM
which can actually solve the problem.  Thus, the TM you choose will have
a significant effect on NT and VT for the same algorithm.

(BTW, I've been ignoring here something that Peter Merel brought up
earlier, that there already is an "analysis of algorithms" providing
some measure of the cost of an algorithm in space and time.  They do, in
fact, use variables to deal with this, but nothing is said about a
"large" amount of space or a "high" amount of time, it is rather the
relative rate of use which they compare.  For example, an algorithm
which takes an amount of time which is exponential in the "amount" of
input given is considered more expensive than an algorithm which takes
a linear amount of time for increasing amounts of input.  This analysis
still uses the good old generic Turing Machine, because although we
talk here about space and time, we never do end up talking about a
_particular_ amount of space or time.)

> [...] But by using variables rather than constants, you've got a way
> to explore the range of possible machines, too. You may not have heard
> of this, but a breath of it happens in parallel computing, where we
> have classes of machines: SIMD machines, MIMD machines,
> multiprocessors with the same addressing space, multiprocesors with
> distributed addressing spaces, and so on.

But, you see, when you have a _real_ computing device, you can then
get excellent results on how it works with given algorithms by defining
a mathematical representation of the actual device and using that for
your proofs, rather than attempting to limit the extremely generalized
Turing Machine in various ways to make it a closer bound to your
product.


John

Rate This Message: http://www.cryonet.org/cgi-bin/rate.cgi?msg=8657