To Afford or Not to Afford: A New Formalization of Affordances

Transkript

Adaptive Behavior
http://adb.sagepub.com
To Afford or Not to Afford: A New Formalization of Affordances Toward Affordance-Based Robot
Control
Erol Sahin, Maya Çakmak, Mehmet R. Dogar, Emre Ugur and Göktürk Üçoluk
Adaptive Behavior 2007; 15; 447
DOI: 10.1177/1059712307084689
The online version of this article can be found at:
http://adb.sagepub.com/cgi/content/abstract/15/4/447
Published by:
http://www.sagepublications.com
On behalf of:
International Society of Adaptive Behavior
Additional services and information for Adaptive Behavior can be found at:
Email Alerts: http://adb.sagepub.com/cgi/alerts
Subscriptions: http://adb.sagepub.com/subscriptions
Reprints: http://www.sagepub.com/journalsReprints.nav
Permissions: http://www.sagepub.com/journalsPermissions.nav
Citations (this article cites 39 articles hosted on the
SAGE Journals Online and HighWire Press platforms):
http://adb.sagepub.com/cgi/content/refs/15/4/447
Downloaded from http://adb.sagepub.com at Middle East Technical Univ on November 9, 2007
© 2007 International Society of Adaptive Behavior. All rights reserved. Not for commercial use or unauthorized distribution.
To Afford or Not to Afford: A New Formalization of
Affordances Toward Affordance-Based Robot
Control
Erol Şahin, Maya Çakmak, Mehmet R. Dogar, Emre Ug
ur, Göktürk Üçoluk
KOVAN Research Laboratory, Department of Computer Engineering, Middle East Technical
University, İnönü Bulvari, Ankara, 06531, Turkey
The concept of affordances was introduced by J. J. Gibson to explain how inherent “values” and
“meanings” of things in the environment can be directly perceived and how this information can be
linked to the action possibilities offered to the organism by the environment. Although introduced in
psychology, the concept influenced studies in other fields ranging from human–computer interaction to
autonomous robotics. In this article, we first introduce the concept of affordances as conceived by J. J.
Gibson and review the use of the term in different fields, with particular emphasis on its use in autonomous robotics. Then, we summarize four of the major formalization proposals for the affordance
term. We point out that there are three, not one, perspectives from which to view affordances and that
much of the confusion regarding discussions on the concept has arisen from this. We propose a new
formalism for affordances and discuss its implications for autonomous robot control. We report preliminary results obtained with robots and link them with these implications.
Keywords
1
affordance · autonomous robots · control architecture · formalization · perception
Introduction
The concept of affordances was introduced by J. J. Gibson to explain how inherent “values” and “meanings” of
things in the environment can be directly perceived, and
how this information can be linked to the action possibilities offered to the organism1 by the environment.
Although J. J. Gibson introduced the term to clarify his
ideas in psychology, it turned out to be one of the most
elusive concepts that influenced studies ranging from
human–computer interaction to autonomous robotics.
The affordance concept, from its beginnings, has
been a hazy one. Despite the existence of a large body of
literature on the concept, upon reviewing the literature,
one encounters different façades of this term, sometimes
contradictory, rather like the description of an elephant
by the six blind men in the famous Indian tale.
In the MACS (Multi-Sensory Autonomous Cognitive Systems interacting with dynamic environments
for perceiving and using affordances) project,2 we, as
roboticists, are interested in how the concept of
affordances can change our views about the control of
an autonomous robot and so we set forth to develop an
affordance-based robot control architecture. In our
quest, we reached an understanding of this elusive
concept, such that it can be formalized and used as a
Correspondence to: Asst. Prof. Dr. Erol Şahin, Department of
Computer Engineering, Middle East Technical University, İnönü
Bulvari , Ankara, 06531, Turkey. E-mail: [email protected]
Tel.: +90-312-210 5539, Fax: +90-312-210 5544
Figures 1, 3–5 appear in color online: http://adb.sagepub.com
Copyright © 2007 International Society for Adaptive Behavior
(2007), Vol 15(4): 447–472.
DOI: 10.1177/1059712307084689
447
448
Adaptive Behavior 15(4)
base for autonomous robot control. The formalization
presented in this article summarizes our work on this
quest which was developed within the MACS project,
but included additional aspects of the affordance concept that went beyond the core work.
In the next section, we review the concept of
affordances and affordance-related literature in different fields. We then summarize different formalizations
of the affordance concept in a common framework. We
point out three different perspectives through which
affordances can be viewed and propose a new formalism that could form a base for an affordance-based
control architecture.
2
The Concept of Affordances and
Affordance-Related Research
In this section, we first describe the concept of
affordance, as originally proposed by J. J. Gibson,
and then review affordance-related studies in different
fields.
2.1 J. J. Gibson’s Affordance Concept
J. J. Gibson (1904–1979) was one of the most influential psychologists of the 20th century, who aimed to
develop a “theory of information pick-up” as a new
theory of perception. He argued that an organism and
its environment complement each other, and that studies on the organism should be conducted in its natural
environment rather than in isolation, ideas that later
formed the basic elements of ecological psychology.
The concept of affordance was conceived within this
context.
In his early studies on visual perception, J. J. Gibson tried to understand how the “meanings” of the
environment were specified in perception for certain
behaviors. To this end, he identified optical variables
in the perceptual data that are meaningful. He gave
one such example for a pilot landing a plane. The
meaningful optical variable in that example was the
optical center of expansion of the pilot’s visual field.
This center of expansion, according to J. J. Gibson,
was meaningful for a pilot trying to land a plane, indicating the direction of the glide and helping him to
adjust landing behavior.
In his book J. J. Gibson (1986) also stated that he
was influenced by the Gestalt psychologists’ view
which pointed out that the meanings of things are perceived just as immediately as other seemingly meaningless properties such as color. In that book, J. J.
Gibson quotes from Koffka:
Each thing says what it is … a fruit says “Eat me”; water
says “Drink me”; thunder says “Fear me”; and woman says
“Love me”. (Koffka, 1935)
Hence, the value of the things in the environment are
apparent to the perceiver just like other properties.
Based on these studies of meaningful optical variables and the Gestaltist conception of the immediate
perception of meanings of the things, J. J. Gibson built
his own theory of perception and coined the term
affordance to refer to the action possibilities that
objects offer to an organism in an environment. The
term affordances first appeared in his 1966 book (J. J.
Gibson, 1966), and is further refined in a later book (J.
J. Gibson, 1986). In the later book, affordances were
discussed in a complete chapter, which laid out the
fundamental aspects of affordances.
Probably his most frequently quoted definition of
affordances is:
The affordances of the environment are what it offers the
animal, what it provides or furnishes, either for good or ill.
The verb to afford is found in the dictionary, but the noun
affordance is not. I have made it up. I mean by it something that refers to both the environment and the animal in
a way that no existing term does. It implies the complementarity of the animal and the environment. (J. J. Gibson, 1979/1986, p. 127)
For instance, a horizontal and rigid surface affords
walk-ability, a small object below a certain weight
affords throw-ability, and so forth. The environment is
full of things that have different affordances for the
organism acting in it. Although one may be inclined to
talk about affordances as if they were simply properties of the environment, they are not:
… an affordance is neither an objective property nor a subjective property; or both if you like. An affordance cuts across
the dichotomy of subjective-objective and helps us to understand its inadequacy. It is equally a fact of the environment
and a fact of behavior. It is both physical and psychical, yet
neither. An affordance points both ways, to the environment and to the observer. (J. J. Gibson, 1979/1986, p. 129)
Şahin et al.
J. J. Gibson believed that affordances are directly
perceivable (a.k.a. direct perception) by the organism,
thus the meaning of the objects in the environment are
directly apparent to the agent acting in it. This was
different from the contemporary view of the time that
the meaning of objects were created internally with
further “mental calculation” of the otherwise meaningless perceptual data.
Formalizing Affordances for Robot Control
•
•
The perceiving of an affordance is not a process of perceiving a value-free physical object to which meaning is somehow added in a way that no one has been able to agree
upon; it is a process of perceiving a value-rich ecological
object. (J. J. Gibson, 1979/1986, p. 140)
Discussions on the perception of object affordances
naturally had some philosophical consequences on the
much debated concept of “object.”
•
However, to date, there has been much confusion
regarding the concept of affordances. We believe that
there are a number of reasons for this confusion, and
that an explicit statement of these reasons is essential
for a healthy discussion of the concept:
J. J. Gibson’s own understanding of affordances
evolved over time. As pointed out by Jones
(2003), J. J. Gibson always considered his ideas
on the concept as “subject to revision”:
What is meant by an affordance? A definition is in order,
especially since the word is not to be found in any dictionary. Subject to revision, I suggest that the affordance of
anything is a specific combination of the properties of its
substance and its surfaces taken with reference to an animal. (J. J. Gibson, 1977, p. 67)
As a consequence of this evolution, different quotations of J. J. Gibson can be seen to support contradictory views of the concept. An excellent
review of the evolution of the concept, dating
back to even before the conception of the term,
was written by Jones (2003, p. 112).
J. J. Gibson’s own ideas on the concept were not
finalized during his lifetime, as Jones (2003) concludes. We believe that the evolution of the term
should continue, and that discussions should be
led toward the point he indicated, rather than
return to the point he had already reached. This is
the view that we have taken in this article.
J. J. Gibson’s idea of affordance can be fully
understood only in contrast to the background of
contemporary ideas on perception, rather than in
isolation. One can read J. J. Gibson’s writing to
understand the background where the concept of
affordances was born, and how the concept of
affordances radically challenged existing views:
Orthodox psychology asserts that we perceive objects
insofar as we discriminate their properties and qualities.
… But I now suggest that what we perceive when we look
at objects are their affordances, not their qualities. We can
discriminate the dimensions of difference if required to do
so in an experiment, but what the object affords us is what
we normally pay attention to. (J. J. Gibson, 1979/1986, p.
134)
The theory of affordances rescues us from the philosophical muddle of assuming fixed classes of objects, each
defined by its common features and then given a name. You
do not have to classify and label things in order to perceive
what they afford. (J. J. Gibson, 1979/1986, p. 134)
•
449
•
J. J. Gibson defined affordances as a concept that
relates the perception of an organism to its action,
whereas his main research interest lay in the perception aspect. Although he explicitly pointed to
other aspects of affordances, such as action and
learning, he did not conduct any research along
these lines.
J. J. Gibson’s own discussions on affordances
were often blended with his work on visual perception. As a result of this blending, early studies
of affordances in ecological psychology, as will
be reviewed below, concentrated on visual perception of the world, with particular emphasis on
optical flow. Therefore, when reading J. J. Gibson’s ideas on affordances, it is important to keep
in mind that the concept provides a general theory
rather than a specific theory of visual perception.
After J. J. Gibson, discussions on the concept of
affordances, and on its place in ecological psychology
have continued. Also, a number of attempts to formalize the concept have been made, because its description as J. J. Gibson left it was ambiguous. These
studies will be reviewed in Section 3. But first we will
review affordance-related research in different fields,
450
with particular emphasis on its application and its
relation to existing concepts in autonomous robot control.
2.2 Affordance-Related Research
2.2.1 Ecological Psychology J. J. Gibson’s view of
studying organism and environment together as a system (including the concept of affordances) has been
one of founding pillars of ecological psychology. Following the formulation of the theory of affordances,
the ecological psychology community started to conduct experiments in order to verify that people are
able to perceive the affordances of the environment,
and to understand the mechanisms underlying this
perception. These experiments (Chemero, 2000; E. J.
Gibson et al., 1987; Kinsella-Shaw, Shaw, & Turvey,
1992; Mark, 1987; Warren, 1984; Warren & Whang,
1987) aimed to show that organisms (mostly human)
can perceive whether a specific action is do-able or
not-do-able in an environment. This implies that what
we perceive is not necessarily objects (e.g., stairs,
doors, chairs), but the action possibilities (e.g., climbable, passable, sittable) offered by the environment.
Although the number of these experiments is quite
high, their diversity is rather narrow. They constitute a
class of experiments characterized by two main
points: taking the ratio of an environmental measure
and a bodily measure of the human subject; and, based
on the value of this ratio, making a binary judgment
about whether a specific action is do-able or not.
The first point indicates how the experimenters
interpreted affordances. Since affordances were roughly
defined as the properties of the environment taken relative to the organism acting in it, the goal was to show
that the ratio between an environmental measure and a
bodily measure of the organism have consequences
for behavior. This ratio must also be perceivable, so
that the organism is aware of this measure which, in a
way, determines the success of its behavior. Thus, this
relativeness of environmental properties was incorporated into the experiments simply as a division operation between two metrics, one of the environment and
one of the organism. From a conceptual point of view,
this is a crude simplification of the relation between
the properties of the organism and the environment
that comprise an affordance, but for the particular
actions and setups used in the experiments, it seemed
sufficient.
Warren’s (1984) stair-climbing experiments have
generally been accepted as a seminal work on the
analysis of affordances, constituting a baseline for
later experiments which seek to understand affordancebased perception. In these studies, Warren showed
that organisms perceive their environment in terms of
intrinsic or body-scaled metrics, not in absolute or
global dimensions. He was able to calculate the constant π proportions that depend on specific properties
of the organism-environment system. For instance, a
human’s judgment of whether he can climb a stair step
is not determined by the height of the stair step, but by
its ratio to his leg-length. The particular value of these
ratios that signaled the existence of an affordance
were called the critical points, whereas the ratios
which determined whether an action can be performed
with minimum energy consumption and maximum
ease were called the optimal points.
Warren and Whang (1987) showed how the perception of geometrical dimensions such as size and
distance is scaled relative to the “perceived eye height”3
of the perceiver, in an environment where the subjects
were to judge the affordance of walking through an
aperture. Mark’s (1987) surface sitting and climbing
experiments also incorporated a similar approach.
Some of these studies (E. J. Gibson et al., 1987; Kinsella-Shaw et al., 1992) criticized former studies for limiting themselves to only one perceptual source, namely
visual information. Instead, these studies reported
experiments related to haptic perception in infant traversability of surfaces and critical slant judgment for
walking on sloped surfaces. While in these experiments
human subjects were asked to judge whether a certain
affordance exists or not in a static environment,
Chemero (2000) conducted other experiments in order
to prove that changes in the layout of affordances are
perceivable in dynamic environments, and found that
the results are compatible with critical ratio values.
Another important work is the study by Oudejans,
Michaels, VanDort, and Frissen (1996) of street-crossing behavior and perception of a critical time-gap for
safe crossing. This work is novel, since it shows that
not only the static properties of the organism, but also
its dynamic state is important when deciding on its
actions.
All these experiments were performed in a one
shot manner, and the subject is either stationary or
moving (Warren & Whang, 1987), has either monocular or binocular vision (Cornus, Montagne, & Laurent,
Şahin et al.
1999), uses either haptic or visual information (E. J.
Gibson et al., 1987), determines either the critical or
optimal points (Warren, 1984), or examines either
searching for affordance or change in the layout of an
affordance (Chemero, Klein, & Cordeiro, 2003).
An overview of the experiments mentioned shows
that they are mostly focused on the perception aspect
of affordances. Other cognitive processes such as
learning, high level reasoning and inference mechanisms are not the subjects of these experiments, and
the link between affordances and these higher level
processes is not discussed. In the following, we will
try to close this gap, by presenting some existing studies on the learning of affordances, and the relation of
affordances to high-level perception.
2.2.2 Cognitive Science E. J. Gibson studied the
mechanisms of the learning of affordances and used
the ecological approach to study child development.
She stated that J. J. Gibson was not particularly interested in development and that “his concern was with
perception” only (Szokolszky, 2003, p. 271). As a
result, he did not discuss the concept of affordances
from a developmental point of view, and only mentioned that affordances are learned in children (J. J.
Gibson, 1986).
E. J. Gibson defined learning as a perceptual
process and named her theory of learning “perceptual
learning.” She argued that learning is neither the construction of representations from smaller pieces, nor
the association of a response to a stimulus. Instead,
she claimed, learning is “discovering distinctive features and invariant properties of things and events”
(E. J. Gibson, 2000, p. 295) or “discovering the information that specifies an affordance” (E. J. Gibson,
2003, p. 283). Learning is not “enriching the input,”
but discovering the critical perceptual information in
that input. She named this process of discovery differentiation, and defined it as “narrowing down from a
vast manifold of (perceptual) information to the minimal, optimal information that specifies the affordance
of an event, object, or layout” (E. J. Gibson, 2003,
p. 284). E. J. Gibson suggested that babies use exploratory activities, such as mouthing, listening, reaching
and shaking, to gain this perceptual data, and that
these activities provide “information about changes in
the world that the action produces” (E. J. Gibson,
2000, p. 296). As development proceeds, exploratory
451
activities become performatory and controlled, executed with a goal.
Studies on affordance, reviewed so far, have not
provided any ideas regarding its relation to other
higher-level cognitive processes. The process of recognition is an example: A subject may indeed seek for
sittability when all he needs is to sit, but what would
he do when he needs to recognize his chair, and how
far can affordances help him in this context? In his
“Cognition and Reality” book, Neisser (1976) tried to
place affordances and direct perception into a complete cognitive system model and tried to link them
with other cognitive processes such as recognition.
According to him, J. J. Gibson was right in stating that
the meanings of the environment are directly available. Invariance attuned detectors are used for this purpose. However, he claimed, the Gibsonian view of
affordances of perception is inadequate, since “it says
so little about perceiver’s contribution to the perception act” (p. 9). Instead, he suggests a perceptual system where a cyclic activity, continuous over time and
space, occurs. This cycle “prepares the perceiver to
accept certain kinds of information … At each moment
the perceiver is constructing anticipations of certain
kinds of information, that enable him to accept it (information) as it becomes available” (p. 20). Since every
natural object has an infinite number of affordances,
this cycle could also be employed to prepare the perceiver to search for particular affordances at each
moment, and attune specific detectors to perceive these
affordances.
Neisser tried to integrate both constructive and
direct theories of perception. As a result, in a later
paper Neisser (1994) constructed a three-layered perceptual system, whose first and third layers correspond to direct perception and recognition, respectively.4 While the direct perception system is
identified by the perception of the local environment,
recognition refers to identification of familiar objects
and situations.
2.2.3 Neurophysiology and Neuropsychology
J. Norman (2002, p. 25), in a similar vein to Neisser,
“attempted to reconcile the constructivist and ecological approaches” into one bigger system, using studies
from neurophysiological and neuropsychological studies. Based on evidence from human dorsal and ventral
systems, he suggested a perceptual system where two
452
different and interacting visual systems work. While
the dorsal system is mainly responsible for the pickup
of information from light which is used to modulate
actions, the ventral system is concerned with high level
perceptual tasks, such as recognition and identification.
Thus, according to J. Norman (2001), it is straightforward to conclude that “the pickup of affordances can
be seen as the prime activity of the dorsal system.” To
support his two-perceptual-systems idea he presents
examples from a patient who lacks a ventral system (J.
Norman, 2002). The patient is able to successfully avoid
obstacles, or insert mail into slots in correct orientation
using her dorsal system. However, while performing
actions successfully, she does not recognize the objects
she is interacting with, and thus cannot report them.
Another set of findings of neurophysiological and
neuropsychological research that is also associated
with the idea of affordances came from studies on
mirror and canonical neurons which were discovered
in the premotor cortex of the monkey brain. During
experiments with monkeys, mirror neurons fired both
when the monkey was grasping an object and when
the monkey was watching somebody else do the
grasping (Rizzolatti, Fadiga, Gallese, & Fogassi 1996;
later similar findings were also found for human subjects by Fadiga, Fogassi, Pavesi, & Rizzolatti, 1995).
These findings implied that the same neurons were
used both ways: for the execution of an action as output of the system, and also for perceiving that action
as an input to the system (Gallese, Fadiga, Fogassi, &
Rizzolatti, 1996). Their discovery supports the view
that says action and perception are closely related.
These neurons, which are located in the premotor cortex of the monkey brain, are thought to be responsible
for the motor activation of prehension actions such as
grasping and holding.
Rizzolatti and Gentilucci (1988) discovered that
canonical neurons, normally considered to be motor
neurons for grasping actions, would fire when the subject does not execute a grasping action, but only sees a
graspable object. Their activity on such a purely perceptive task that included an object that affords the
particular action the motor neurons were responsible
for, indicated that they may be related to the concept
of affordance. The resulting conclusions are interestingly similar to those of the ecological approach:
This process, in neurophysiological terms, implies that the
same neuron must be able not only to code motor acts, but
also to respond to the visual features triggering them. …
3D objects are identified and differentiated not in relation
to their mere physical appearance, but in relation to the
effect of the interaction with an acting agent. (Gallese, 2000)
Humphreys (2001) showed that, when presented
with a tool, some patients, who lacked the ability to
name the tool, had no problem in gesturing the appropriate movement for using it. According to Humphreys, this suggested a direct link from the visual
input to the motor actions that is independent of more
abstract representations of the object, for example, its
name. In another study that Humphreys presented,
two groups were shown object pictures, non-object
pictures, and words. One of the groups was asked to
determine if some actions were applicable to what had
been presented. The other control group was asked to
make size judgments. The brain activities in both
groups were compared using functional brain imaging. It was observed that a specific region of the brain
was activated more in the first group who were to
make action judgments. It was also seen that this specific region was activated more when the subjects
were presented with pictures of the objects rather than
with the name. This showed that action related regions
of the brain were activated more when the visual input
was supplied, rather than just naming it. All these
findings suggest that there is a strong link between
perception and action in terms of neuropsychological
activity.
2.2.4 Human–Computer Interaction The concept of
affordance has influenced other, seemingly unrelated,
disciplines as well. One of these is the human–computer interaction (HCI) domain. The concept was introduced to the HCI community in D. A. Norman’s (1988)
popular book, Psychology of Everyday Things (POET).
In his book, D. A. Norman discussed the perceptual
information that can make the user aware of an object’s
affordances. In this context, he defined affordances as
follows:
…affordance refers to the perceived and actual properties
of the thing, primarily those fundamental properties that
determine just how the thing could possibly be used. (p. 9)
Unlike J. J. Gibson however, D. A. Norman was interested in how “everyday things” can be designed such
Şahin et al.
that the user can easily infer what they afford. He analyzed the design of existing everyday tools and interfaces, identifying design principles. In this respect, his
discussion of affordances deviated from the Gibsonian
definition of the term (McGrenere & Ho, 2000). D. A.
Norman (1999) writes:
The designer cares more about what actions the user perceives to be possible than what is true. (p. 39)
Since POET, the term affordance has been used in
many ways in the HCI community, some in the sense
that D. A. Norman introduced, some being more loyal
to J. J. Gibson’s definition, and others deviating from
both of these and using the term in a totally new way
(McGrenere & Ho, 2000).
In a later article D. A. Norman (1999), uncomfortable with the misuse of the term in the HCI community, distinguished between “real affordances,”
indicating the potentials in the environment independent from the user’s perception, and what he called
“perceived affordances” stating:
When I get around to revising POET, I will make a global
change, replacing all instances of the word “affordance”
with the phrase “perceived affordance.” (p. 39)
The concept of affordances is highly applicable to autonomous robot control and it has influenced studies in this field. We
believe that, for a proper discussion of the relationship
of the affordance concept to robot control, the similarity of the arguments of J. J. Gibson’s theory and reactive/behavior-based robotics should first be noted. An
early discussion of this relationship was made by
Arkin (1998, p. 244) and our discussion partially builds
on his.
The concept of affordances and behavior-based
robotics emerged in very similar ways as opposing
suggestions to the then dominant paradigms in their
fields. J. J. Gibson constructed his theory based on
criticism of the then dominant theory of perception
and cognition, which favored modeling and inference.
Likewise, behavior-based robotics was motivated by
criticism of the then dominant robotic architectures,
which favored modeling and inference. This parallelism between the two fields suggests that they are
applications of the same line of thinking to different
2.2.5 Autonomous Robotics
453
domains (1998, p. 244; Duchon, Warren, & Kaelbling, 1998).
Opposing modeling and inference, J. J. Gibson
defended a more direct relationship between the organism and the environment and suggested that a model of
the environment and costly inferential processes were
not needed. In a similar vein, behavior-based robotics
advocated a tight coupling between perception and
action. Brooks, claiming that “the world is its own best
model,” suggested an approach that eliminated all modeling and internal representation (Brooks, 1990, p. 13).
J. J. Gibson suggested that only the relevant information is picked up from the environment, saying
“perception is economical” (p. 135). In robotics a
behavior is a sensory-motor mapping which can often
be simplified to a function from certain sensors to certain actuators. In this sense, the perceptual part of a
behavior can be said to implement direct perception
by extracting only the relevant information from the
environment for action, without relying on modeling or
inference. Such a minimality is also in agreement with
the economical perception concept of the affordance
theory.
As discussed above, most of the concepts within
affordance theory are inherently included in reactive
robotics. The behaviors should be minimally designed
for the task, taking into account the niche of the
robot’s working environment and the task itself. This
is in agreement with the arguments of ecological psychology. Some roboticists have already been explicitly using ideas on affordances in designing behaviorbased robots. For example, Murphy (1999) suggested
that robotic design can benefit from ideas in the theory
of affordances such that complex perceptual modeling
can be eliminated without loss in capabilities. She
studied three case studies and drew attention to the
importance of the ecological niche in the design of
behaviors. Likewise, Duchon et al. (1998) benefited
from J. J. Gibson’s ideas on direct perception and optic
flow in the design of behaviors and coined the term
Ecological Robotics for the practice of applying ecological principles to the design of mobile robots.
The use of affordances within autonomous robotics is mostly confined to behavior-based control of the
robots, and its use in deliberation remains a rather
unexplored area. This is not a coincidence, but a consequence of the shortfalls in J. J. Gibson’s theory. The
reactive approach could not scale up to complex
tasks in robotics, in the same way that the theory of
454
affordances in its original form was unable to explain
some aspects of perception and cognition.
In cognitive science, some cognitive models associated affordances only with low-level processes (J.
Norman, 2002), others viewed affordances as a part of
a complete cognitive model (E. J. Gibson, 2000; MacDorman, 2000; Neisser, 1994; Susi & Ziemke, 2005).
Similarly, in robotics, some hybrid architectures inherit
properties related to affordances only at their reactive
layer (Arkin & Balch, 1997; Connell, 1992), while
others study how the use of affordances may associated
with high-level processes such as learning (Cooper &
Glasspool, 2001; Cos-Aguilera, Canamero, & Hayes,
2004; Fitzpatrick, Metta, Natale, Rao, & Sandini, 2003;
MacDorman, 2000; Stoytchev, 2005b), decision-making (Cos-Aguilera, Canamero, & Hayes, 2003), and
planning (Stoytchev, 2005a).
Recently a number of robotic studies focused on
the learning of affordances in robots. These studies
mainly tackled two major aspects. In one aspect,
affordance learning is referred to as the learning of the
consequences of a certain action in a given situation
(Fitzpatrick et al., 2003; Stoytchev, 2005a, 2005b). In
the other, studies focus on the learning of the invariant
properties of environments that afford a certain behavior (Cos-Aguilera et al. 2003, 2004; MacDorman,
2000). Studies in this latter group also relate these
properties to the consequences of applying a behavior,
but these consequences are in terms of the internal
values of the agent, rather than changes in the physical
environment.
Cooper and Glasspool (2001) referred to the
learning of action affordances as the acquisition of
environment–action pairs that result in successful execution of the action. Their paper associated the
affordance to the whole perceived situation of the
environment and asserted the consequences of actions,
rather than learning them, by judging the outcome of
actions as to reinforce successful ones.
Cos-Aguilera et al. (2003) used affordances in
action selection by learning the relation between perceived features of objects and the consequence of
performing an action on the object, where the consequence is judged by the robot in terms of the change
in homeostatic variables in its motivational system. In
a later study (Cos-Aguilera et al., 2004) they gave
more emphasis to learning the “regularities” of objects
and relating them to the outcome of performing an
action.
Similarly, MacDorman (2000), extracted invariant features of different affordance categories. In his
study, the invariant features are defined as image signatures that do not vary among the same affordance
category but vary among different affordance categories. However, his affordance categories were
defined in terms of internal indicators, such as tasty
or poisonous, and were not directly related to the
actions.
Stoytchev (2005a, 2005b) studied learning for the
so-called “binding affordances” and “tool affordances,”
where learning binding affordances corresponds to
discovering the behavior sequences that result in the
robot arm binding to different kinds of objects whereas
learning tool affordances corresponds to discovering
tool–behavior pairs that give the desired effects. In
this study the representation of objects is said to be
grounded in the behavioral repertoire of the robot, in
the sense that the robot knows what it can do with an
object using each behavior. However, in this study,
object identification was done by assigning unique
colors to each object, hence leaving no way of building associations between the distinctive features of the
objects and their affordances. Therefore, a generalization which would make the robot respond properly to
novel objects was not possible.
Fitzpatrick et al. (2003) studied the learning of
object affordances in a robotic domain. They proposed
that a robot can learn what it can do with an object
only by acting on it, “playing” with it, and observing
the effects in the environment. For this aim, they used
four different actions of a robot arm on four different
objects. After applying each of the actions on each of
the objects several times, the robot learned about the
roll-ability5 affordance of these objects, by observing
the changes in the environment during the application
of the actions. Then, when it needs to roll an object, it
uses this knowledge. However, as in Stoytchev’s
study, Fitzpatrick et al. did not establish any association between the visual features of the objects and
their affordances, giving no room for generalization of
the affordance knowledge to novel objects.
Finally we would like to note that affordance theory has mostly been used as a source of inspiration in
robotics. Most of the studies reviewed above preferred
to refer to J. J. Gibson’s original ideas as formulated
in his books, ignoring modern discussions on the concept. As a result, only certain aspects of the theory
have been used, and no attempts to consider the impli-
Şahin et al.
cations of the whole theory toward autonomous robot
control have been made.
3
Prior Formalizations of Affordances
Following J. J. Gibson’s work, there have been a
number of studies which attempted to clarify the
meaning of the term affordances and to create a common understanding on which discussions can be based
(Chemero, 2003; Greeno, 1994; Michaels, 2003; Sanders, 1997; Steedman, 2002b; Stoffregen, 2003; Turvey, 1992; Wells, 2002). We will now review four of
the proposed formalisms.
3.1 Turvey’s Formalization
One of the earliest attempts to formalize affordances
came from Turvey (1992). In his formalism, Turvey
defined an affordance as a disposition. Here, a disposition is a property of a thing that is a potential, a possibility. These potentials become actualized if they
combine with their complements (e.g., “solubility” of
the salt is its disposition, and if it combines with its complement, which is water’s property of “being able to
dissolve,” then they get actualized, resulting in the salt
getting “dissolved”). Therefore, dispositions are defined
in pairs, and when two complement dispositions meet
in space and time, they get actualized. Basing his
views on this account of dispositions, Turvey defined
affordances as dispositions of the environment, and
defined their complement dispositions as the “effectivities” of the organism. He provided this definition:
An affordance is a particular kind of disposition, one whose
complement is a dispositional property of an organism.
(p. 179)
Later in his discussion, Turvey formalized this
definition as follows:
Let Wpq (e.g., a person-climbing-stairs system) = j(Xp, Zq)
be composed of different things Z (person) and X (stairs).
Let p be a property of X and q be a property of Z. Then p is
said to be an affordance of X and q the effectivity of Z
(i.e., the complement of p), if and only if there is a third
property r such that:
•
Wpq = j(Xp, Zq) possesses r [where j(·) is the juxtaposition function that joins Xp and Zq].
•
•
455
Wpq = j(Xp, Zq) possesses neither p nor q.
Neither Z nor X possesses r. (p. 180)
Here, when the physical structure that renders the
stairs climbable (Xp), and the effectivity of the agent
(Wq) that makes it able to climb come together (j(·)),
new dynamics – the action of climbing – (r) arise.
In this formalism, although the actualization of
affordances requires an interaction of an agent on the
environment to produce a new dynamics, Turvey
explicitly attached affordances to the environment that
the organism is acting in.
3.2 Stoffregen’s Formalization
A criticism of Turvey’s formalism came from Stoffregen (2003). According to Stoffregen, there are two
main views about affordances. The first view places
affordances in the environment alone, while the second view places affordances in the organism–environment system as a whole. Stoffregen adopts the latter
view and argues that affordances cannot be defined
only as properties of the environment, as Turvey did.
From this point of view, Stoffregen (2003) described
affordances as:
Affordances are properties of the animal–environment
system, that is, that they are emergent properties that do
not inhere in either the environment or the animal. (p. 115)
He claimed that attaching affordances to the environment was problematic for their specification to the
organism. The reason was that if affordances belong
to the environment only, and if what the organism perceives are affordances, then the organism perceives
things that are only about the environment but not
about itself. If this is the case, then the agent has to do
further perceptual processing to infer what is available
for him. However, this goes against the basic notion of
direct perception.
Based on these criticisms, Stoffregen modified
Turvey’s definition to propose a new one to resolve
these problems. He presented it in the following way:
Let Wpq (e.g., a person-climbing-stairs system) = (Xp, Zq)
be composed of different things Z (e.g., person) and X
(e.g., stairs). Let p be a property of X and q be a property
of Z. The relation between p and q, p/q, defines a higher
order property (i.e., a property of the animal–environment
456
system), h. Then h is said to be an affordance of Wpq if and
only if
•
Wpq = (Xp, Zq) possesses h.
•
Neither Z nor X possesses h. (p. 123)
Here, affordances are defined as “properties of the
animal-environment system,” rather than as properties
of the environment only.
3.3 Chemero’s Formalization
Chemero (2003) also criticized Turvey’s view which
placed affordances in the environment regarding them
as environmental properties. Partially in agreement
with Stoffregen’s proposal, Chemero suggested that:
Affordances, are relations between the abilities of organisms and features of the environment. (p. 181)
This definition refines Stoffregen’s proposal in a
number of ways. First, it states that affordances are
“relations within the animal-environment system,”
rather than “properties of the animal-environment system.” Second, it also notes that this relation exists
between the “abilities of the organism” and the “features of the environment,” as compared with a property (of the system) being generated through the
interaction between the “property of the organism”
and the “property of the environment.”
Formally Chemero proposed that an affordance is
a relation that can be represented in the form of:
Affords-φ (feature, ability), where φ is the afforded behavior.
Here the term “ability” stands for the functional properties of the organisms that are shaped through the
evolutionary history of the species or the developmental history of the individual. In that respect, they are
different from simple body-scale measures (e.g., the
leg-length), but correspond to more general capabilities of the organism. One of the main differences
between the two similar formalisms of Stoffregen and
Chemero, which both define affordances at the organism–environment scale, is that while Stoffregen’s definition of affordance does not include the behavior
exploiting the affordance, Chemero’s definition does
include it.
3.4 Steedman’s Formalization
Independent of discussions in the ecological psychology literature, there have also been other attempts at
formalization of affordances. One of these came from
Steedman (2002b) who used linear dynamic event calculus to reach a formalization of affordances. Steedman’s formalization skips the perceptual aspect of
affordances (e.g., the invariants of the environment
that help the agent perceive the affordances, and the
nature of these invariants and the relation of them to
the bodily properties of the agent etc.), but instead it
focuses on developing a representation where object
schemas are defined in relation to the events and
actions that they are involved in. For instance, Steedman suggests that a door is linked with the actions of
“pushing” and “going-through,” and the preconditions
and consequences of applying these actions to the
door. The different actions that are associated with a
particular kind of object constitute the affordance-set
of that object schema, and this set can be populated via
learning. More formally, in Steedman’s formalization, an object schema is a function mapping objects
of that kind into second-order functions from their
affordances to their results.6 Thus, an object instance
specifies what actions can be applied to it, under which
conditions, and what consequences it yields. This makes
the formalization also suitable for planning, for which
Steedman argues that reactive/forward-chaining planning is the best candidate. Steedman’s formalization is,
as far as we know, the first attempt to develop a formalization of affordances that allows logical/computational
manipulation and planning. Steedman also believes this
structure of affordances to have implications for the
linguistic capability of humans.
To summarize, it can be said that Stoffregen’s and
Chemero’s formalizations, by defining affordances as
a relation on the scale of organism–environment system, differ from Turvey’s formalization which defines
affordances as environmental properties. But there are
also differences between Chemero’s and Stoffregen’s
definitions, one of them being the inclusion of behaviors in the definition of affordances in Chemero’s formalization. Steedman’s formalization differs from the
other three formalizations by providing an explicit
link to action possibilities offered by the environment,
and by proposing the use of the concept in planning.
We believe that none of the reviewed formalisms
can be used as a base to develop an affordance-based
Şahin et al.
robot control architecture. In the next section, we will
introduce three perspectives through which affordances
can be discussed, to explain the source of confusion in
the discussions.
4
457
ment. We propose that the affordances in this ecology
can be seen from three different perspectives:
•
•
•
agent perspective;
environmental perspective; and
observer perspective.
Three Perspectives of Affordances
One major axis of discussions on affordances concerns
where to place them. In some discussions, affordances
are placed in the environment as extended properties
that are perceivable by the agent, whereas in others,
affordances are said to be properties of the organism–
environment system. We believe that the source of the
confusion is due to the existence of three – not one! –
perspectives to view affordances. We argue that in
most discussions, authors, including J. J. Gibson
himself, often pose their arguments from different
perspectives, neglecting to explicitly mention the perspective that they are using. This has been one of
major sources that have made the arguments confusing, and seemingly contradictory at times.
The three different perspectives of affordances
can be described using the scene sketched in Figure 1,
which consists of a (robot) dog, a human(oid) and a
ball. In this scene, a dog is interacting with the ball,
and this interaction is being observed by a human,
who is invisible to the dog and is not part of the dog–
ball system. In this scene, the dog is said to have the
agent role, whereas the human is said to have the
observer role. We will denote the ball as the environ-
We will now describe how affordances can be
viewed from these three different perspectives.
4.1 Agent Perspective
In this perspective, the agent interacts with the environment and discovers the affordances in its ecology. In
this view, the affordance relationships7 reside within
the agent interacting in the environment through his
own behaviors. In Figure 1, the dog would “say”: “I
have push-ability affordance,” upon seeing the ball.
This view is the most essential one to be explored
for using affordances in autonomous robot control,
and will be the central focus of our formalization to be
developed in the next section.
4.2 Environmental Perspective
The view of affordances through this perspective
attaches affordances over the environment as extended
properties that can be perceivable by the agents. In our
scene, the ball would “say”: “I offer hide-ability
affordance” to an approaching dog. When interrogated
to list all of its affordances, the same ball may say: “I
offer, push-ability (to a dog), throw-ability (to a
human),..., affordances.”
In most of the discussions of affordances, including some of J. J. Gibson’s own, this view is often
implicitly used, causing much of the existing confusion.
4.3 Observer Perspective
Figure 1 Three perspectives to view affordances. In this
hypothetical scene (adapted from Erich Rome’s slide depicting a similar scene), the (robot) dog is interacting with
a ball, and this interaction is being observed by a human(oid) who is invisible to the dog. (Drawing by Egemen
Can Şenkardeş.)
The third view of affordances, which we call the
observer perspective, is used when the interaction of
an agent with the environment is observed by a third
party. In our scene, we assume that the human is
observing the interaction of the dog with the ball. In
this case, the human would say: “There is push-ability
affordance” in the dog–ball system.
In writings of J. J. Gibson, support for the observer
perspective can also be seen. While describing the
458
nature of the optical information for perceiving
affordances, J. J. Gibson (1986) mentions that it is
required for a child to perceive the affordances of
things in the environment for others as well as itself:
The child begins, no doubt, by perceiving the affordances
of things for her, for her own personal behavior. But she
must learn to perceive the affordances of things for other
observers as well as herself. (J. J. Gibson 1979/1986,
p. 141)
That is, one must also have the capability of taking
the observer perspective when perceiving affordances,
at least for the agents of the same species as the
observer.
5
An Extended Affordance
Formalization
In this section, we develop a formalism to describe
our understanding of affordances. Our motivation in
attempting this task differs from the prior formalizations studies that we have reviewed, because it stems
from our interest in incorporating the affordance concept into autonomous robot control.
In agreement with Chemero, we view affordances
as relations within an ecology of acting, observing
agents and the environment. Our starting point for formalizing affordances is:
Definition 1. An affordance is a relation between the
agent8 and its environment as acquired from the interaction of the two.9
Based on this definition, an affordance is said to be
a relation that can be represented as
(environment, agent).
However, this formalism is too generic to be useful, and needs to be refined. As Chemero also asked in
his formalization, “which aspect of the environment is
related to which aspect of the organism (agent), and in
what way?” Therefore in this relationship, the environment and the agent should be replaced with “environmental relata” and “agent’s (organismal) relata” (as in
Chemero’s terminology), to indicate the relevant
aspects of the two.
First, we use the term, entity, to denote the environmental relata of the affordance instead of features (as
used by Chemero) or object (as generally used). In our
formalism, entity represents the proprioceptive state of
the environment (including the perceptual state of the
agent) as perceived by the agent. The term entity is chosen since it has a generic meaning that is less restricting
than the term object. Although for some affordances the
term object perfectly encapsulates the environmental
relata, for others, the relata may not be confined to an
object and may be more complex.
Second, the agent’s relata should represent the part
of the agent that is generating the interaction with the
environment that produced the affordance. Ideally, the
agent’s relata should consist of the agent’s embodiment
that generates the perception–action loop that can realize the affordance. We chose the term behavior to
denote this. In autonomous robotics, a behavior is
defined as a fundamental perception–action control unit
to create a physical interaction with the environment.
We argue that this term implicitly represents the physical embodiment of the interaction and can be used to
represent the agent’s relata.
Third, the interaction between the agent and the
environment should produce a certain effect. More specifically, a certain behavior applied on a certain entity
should produce a certain effect, for example, a certain
perceivable change in the environment, or in the state
of the agent. For instance, the lift-ability affordance
implicitly assumes that, when the lift behavior is
applied to a stone, it produces the effect lifted, meaning that the stone’s position, as perceived by the agent,
is elevated (Figure 2a).
Based on these discussions, we refine our first
definition as:
Definition 2. An affordance is an acquired relation
between a certain effect and an (entity, behavior)
tuple, such that when the agent applies the behavior
on the entity, the effect is generated.
We refine our formalization as
(effect, (entity, behavior)).
This formalization explicitly states that an affordance
is a relation which consists of an (entity, behavior)
pair and an effect such that there exists a potential to
generate a certain effect when the behavior is applied
Şahin et al.
459
Figure 2 (a) An affordance is a relation between an entity in the environment and a behavior of an agent, saying that
there exists a potential for generating a certain effect through the application of that behavior on that entity. In this example, the application of lift behavior on a can generated the effect of being lifted, and this relation is called lift-ability. Liftability is shown as a “cloud” to indicate that it is just a label for the relation used to make the discussions more clear. (b)
Entity equivalence: Many different entities (red-can and a blue can) can be used to generate the same effect (being lifted) upon the application of a certain behavior (lift). (c) Behavioral equivalence: More than one behavior (lift-with-rightarm and lift-with-left-arm) can be applied to a certain entity (blue can) to generate a certain effect (lifted). (d) Affordance
equivalence: Different (entity, behavior) tuples [(river, swim) and (ground, walk)] can generate the same effect (traversed).
on the entity by the agent. In this formalism, we
assume that this relation resides within the interacting agent. This means that all three components are
assumed to be sensed by the agent. The behavior
denotes the executed perception-action routine that
generated the interaction as sensed by the agent. The
entity refers not to an abstract concept of an entity
(such as a stone) but to its perceptual representation
by the agent. Similarly, the effect refers to the change
inflicted in the environment (including changes in the
state of the agent) as a result of the behavior acting on
the entity as perceived by the agent.
The proposed formalization, with its explicit inclusion of effect, can be seen as a deviation from J. J.
Gibson’s view at its outset. It is not. In J. J. Gibson’s
writings, the issue of effect had always remained
implicit. For instance in the definition of the lift-ability affordance, the expected effect of lifted is implicitly present. Similarly, this has been implicitly
included in Chemero’s formalism where he named the
relation as Affords-φ to exclude the instances that did
not produce the affordance. On the other hand, in both
Turvey’s and Stoffregen’s formalizations, the desired
effect is represented as h and r respectively. The proposed formalization differs from these by not only
making it explicit, but also putting it on a par with the
entity and the behavior.
The idea of explicit inclusion of a third component into the affordance representation in addition to
behavior, and entity was first set out in Dorffner,
Irran, Kintzler, and Poelz (2005) and Irran, Kintzler,
and Pölz (2006) within the MACS project. In these
studies, the learning of affordances was proposed as
the learning of bilateral relations between three components, namely, entity, action and outcome (corresponding to behavior and effect respectively). The
proposed formalization builds on this idea but differs
from it in two aspects. First, instead of using outcome,
which was assumed to be derived from the “time
series episode starting after the begin of the application of an action and ending with the end of the action
application,” we used effect as the third component,
460
which can be defined as the change inflicted on the
environment. We believe that it is essential for an
affordance to have an effect in the environment, and
that the issue of change has to be emphasized. Second,
entity and behavior components are grouped into a
tuple before being linked to the effect. As will become
apparent in our discussions later in the article, such a
grouping has important benefits.
One question that may be posed is whether this
formalism has equated affordance with effect. This is
not the case. The formalism uses effect as the index to
(entity, behavior) tuples. In this sense, given a desired
effect to be achieved, the agent can directly access
which (entity, behavior) can be used to that purpose.
An important aspect of affordances, which is also
explicitly stated in our definition, is that they are
acquired through the interaction of the agent with the
entity. Therefore it is essential to consider the acquisition aspect in order to understand the nature of the three
components of our formalism. Note that, whether this
acquisition is done through learning, evolution or trialand-error based design is irrelevant for our discussion.
In the rest of the discussion we will use a hypothetical humanoid robot trying to discover affordances
in his operating environment as our guiding scenario.
We assume that the robot will experiment with the
entities in its environment using its repertoire of
behaviors and record the effects as relation instances
in the proposed formalism. For instance, imagine that
the robot applied its lift-with-right-hand behavior on a
black-can and observed the can being lifted as its
effect. This knowledge can be stored as
(lifted, (black-can, lift-with-right-hand)).
experiment and does not have any predictive ability
over future experiments, hence is not a relation. As
the robot explores its environment, it will populate its
knowledge database using such relation instances:
(lifted, (black-can, lift-with-right-hand))
(lifted, (blue-can, lift-with-right-hand))
(not-lifted, (blue-box, lift-with-left-hand))
(lifted, (black-can, lift-with-right-hand)).
However, such a database can hardly be called
affordances. Affordances should be relations with predictive abilities, rather than a set of unconnected relation instances. In the rest of the section, we will propose
four aspects through which relation instances can be
bound together toward discovering affordances.
5.1 Entity Equivalence
The class of entities which support the generation of
the same effect upon the application of a certain behavior is called an entity equivalence class. For instance,
our robot can achieve the effect lifted, by applying the
lift-with-right-hand behavior on a black-can, or a bluecan (Figure 2b). These relation instances can then be
joined together as:

  blue-can 

 lifted,  
, lift-with-right-hand  .

  black-can 

(1)
Here, note that the term black-can is used just as a
short-hand label to denote the perceptual representation of the black can by the interacting agent. Similarly, lifted and lift-with-right-hand are labels to the
related perceptual and proprioceptive representations.
For instance the representation of black can be a raw
feature vector derived from all the sensors of the robot
looking at the black can before it attempts to apply its
lift behavior. The naming of such a representation
with a label like black-can, from the viewpoint of an
external observer is merely to make our discussions
easier to read.
We call Expression 1, a relation instance, to indicate that it contains knowledge obtained from a single
This relation can then be compacted by a mechanism
that operates on the class to produce the (perceptual)
invariants of the entity equivalence class as:
(lifted, (<*-can>, lift-with-right-hand))
where <*-can> denotes the derived invariants of the
entity equivalence class.
In this particular example, <*-can> means “cans
of any color” that can be lifted upon the application of
lift-with-right-hand behavior. Such invariants create a
general relationship and enable the robot to predict the
effect of the lift-with-right-hand behavior applied on a
novel object, such as a green-can. Such a capability
offers great flexibility to a robot. When in need, the
Şahin et al.
robot can search and find objects that would provide
support for a desired affordance.
We would like to note that the concept of entity
equivalence is related to the concept invariance,
defined as “persistence under change” in broad terms
by J. J. Gibson. He mentioned the concept in many
contexts throughout his book and devoted one section
of the Appendices to it. These invariants correspond
to the properties which remain constant under various
transformations, that is, invariants of optical structure
under changing illumination or under change of the
point of observation. Although J. J. Gibson did not
explicitly define these invariances, he gave some
clues about the perception and usage of them.
… There must be invariants for perceiving the surfaces,
their relative layout, and their relative reflectances. They
are not yet known, but they certainly involve ratios of
intensity and color among parts of the array. (J.J. Gibson,
1979/1986, p. 310)
Entity equivalence can also be related to matched
filters10 (Wehner, 1987) which suggests that certain
sensor states are equivalent if they induce the same
motor response, and there are typically some key features that discriminate the relevant situations for certain motor actions. In this sense, matched filters can
also be considered as classifiers of entity equivalence
classes.
We argue that the discovery of invariants in entity
equivalence classes can also produce abstractions over
existing entities. For instance, the invariant <*-can>
denotes a can without color, in an environment where
all cans have color. In this sense, if one restricts entity
to only the perceptual representation of the external
world, the component <entity> can be referred to as
an affordance cue (Fritz et al., 2006), which hints at
the existence of a potential affordance. We would also
like to note that when the term entity also includes the
perceptual state of the agent itself, the term <entity>
can be considered to be equivalent to the term precondition in deliberative planning. Finally, note that the
question of how these invariants can be discovered
and represented is a challenge that needs to be tackled.
5.2 Behavior Equivalence
The concept of affordance starts with equi-distance to
perception (through the entity in the environment) and
461
action (through behavior of the agent). Yet the role of
action is often less pronounced than the role of perception, and most of the discussions concentrate on
the perception aspect of affordances. We argue that, if
we wish to maintain a fair treatment of the action
aspect of affordances, then the same equivalence concept should be generalized to that aspect as well.
For instance, our robot can lift a can using its liftwith-right-hand behavior. However, if the same effect
can be achieved with its lift-with-left-hand behavior, then
these two behaviors are said to be behaviorally equivalent. This can be represented in our current formalism as:


 lift-with-right-hand  
 lifted,  <*-can>, 
 


 lift-with-left-hand  
as also shown in Figure 2c. One can join these into
(lifted, (<*-can>, <lift-with-*-hand>))
where <lift-with-*-hand> denotes the invariants of
the behavior equivalence class.11
We would like to note that, as with the entity
equivalence, the use of behavioral equivalence will
bring a similar flexibility for the agent. Through discovery of the perceptual invariants of an entity equivalence class, the agent gains the competence to use a
different entity to generate a desired effect, even if the
entities that had generated the effect in the past are not
present in its environment. Such a “change of plan” is
directly supported by the entity equivalence classes. A
similar competence is gained through behavioral
equivalence classes. For instance, a humanoid robot
which lifted a can with one of its arms, loses its ability
to lift another can. However, through behavioral
equivalence it can immediately have a “change of
plan” and accomplish lifting using its other hand.
5.3 Affordance Equivalence
Taking the discussion one step further, we come to the
concept of affordance equivalence. Affordances such
as traversability are obtainable by “walking across a
road” or “swimming across a river” (Figure 2d) as

 ( <road>, <walk> ) 
 traversed, 
 .

 ( <river>, <swim> ) 
462
That is, a desired effect can be accomplished through
different (entity, behavior) relations. As a result of
this, at a first glance, one is tempted to revise the formalization as:
(effect, {(<entity>, <behavior>)}).
However, we claim that a better and more general formalization that is consistent with the discussions made
up to now would be:
(effect, <(entity, behavior)>).
This formalization suggests that the entity (the sensory
information) is to be concatenated with behavior (the
motor information) and that the invariances are detected
on this combined representation. We would like to note
that this formalization is consistent with ideas of effect
and behavioral equivalence and that such equivalence
classes would emerge as well. An interesting support
for this formalization can be drawn from studies of
mirror neurons, which are observed to be activated
during pure perception as well as during action.
5.4 Effect Equivalence
The concepts of entity, behavior and affordance
equivalence classes implicitly relied on the assumption that the agent, somehow, has effect equivalence.
For instance, applying the lift-with-right-hand behavior on a blue-can would generate the effect of “a blue
blob rising in view.” If the robot applies the same
behavior to a red-can, then the generated effect will be
“a red blob rising in view.” If the robot wants to join
the two relation instances learned from these two
experiments, then it has to know whether the two
effects are equivalent or not. In this sense, all the three
equivalences rely on the existence of effect equivalence classes.
At its outset, the need for effect equivalence turns
the problem into a chicken-and-egg problem. The
challenge of discovering effect equivalence classes
concurrently with entity and behavioral equivalence
classes will be an interesting problem for the learning
of affordances by autonomous robots. On the other
side, the inclusion of effect equivalence highlights that
the invariant detection operation would apply to all
three components of the representation and that effect
is no exception.
5.5 Agent’s Affordances
Finally, we propose that an affordance can be formalized as:
(<effect>, <(entity, behavior)>).
This formalism represents affordance from an
agent’s perspective. We will make this perspective
explicit, and revise our definition as:
Definition 3. Affordance (agent perspective): An
affordance is an acquired relation between a certain
<effect> and a certain <(entity, behavior)> tuple such
that when the agent applies an (entity, behavior) within
<(entity, behavior)>, an effect within <effect> is generated.
This definition differs from the previous version
because it explicitly states that affordance is a relation
between equivalence classes, rather than a relation
instance between an effect and an (entity, behavior).
5.6 Observer’s Affordances, and Agent
Equivalence
We can now extend the affordance formalization to
accommodate affordances from the observer perspective as:
(<effect>, (<agent>, <(entity, behavior)>)).
where agent denotes the perceptual characteristics of
the agent that is being observed and <agent> represents the agent equivalence class. Such an equivalence class can be the basis for the learning of species
concepts. That is, after observing what affordances
different mice would have in the presence of a stone,
the human observer can develop a “mouse” concept.
However, we should also note that the affordance
would also allow the formation of a “small creatures”
class, which would allow the human to predict the
behavior of a rat. One would even speculate whether
the <agent> class for the agent’s own affordances can
be linked to the concept of self or not. However, this is
a controversial issue, and we will not elaborate on it.
We also would like to note that this representation
will be different for the human observing a mouse
than the human observing his own self. Although not
Şahin et al.
explicitly stated in our formalism, the behavior representation included motor information when representing
one’s own affordances. However, when representing
others’ affordances, the behavior is the behavior of the
other agent as perceived by the observer.
We will make this perspective explicit, and revise
our definition as:
we will discuss the major aspects of affordances as
proposed within the formalism, and the corresponding
implications toward robot control:
•
Definition 4. Affordance (observer perspective): An
affordance is an acquired relation between a certain
<effect> and a certain (<agent>, <(entity, behavior)>)
tuple such that when the observed agent within <agent>,
applies an (entity, behavior) within <(entity, behavior)>,
an effect within <effect> is generated.
•
5.7 Environmental Affordances
As we have discussed above, this perspective of
affordance exists merely in discussions over the concept, and it is not relevant for affordance-based robot
control. However, this perspective can also be formalized. For this, we will assume that the entity being
interacted with can also acquire an affordance relation
based on its interaction with the agents in its ecology.
Under this assumption, an affordance can be formalized as:
(<effect>, <(<agent>, < behavior>)>).
Note that, the <entity> component drops, since
we are dealing with a single entity, and that the relation is assumed to reside inside the entity. A definition
can be provided:
Definition 5. Affordance (environmental perspective): An affordance is an acquired relation between a
certain <effect> and a set of (<agent>, <behavior>)
tuples such that when the agent within <agent>,
applies a behavior within <behavior> on the entity
(both taken from the same tuple), an effect within
<effect> is generated.
6
Discussions of the Formalism and Its
Implications for Robot Control
We believe that the proposed formalism has laid out a
good framework over which the concept of affordance
can be utilized for autonomous robot control. Below,
463
•
Affordances can be viewed from three perspectives, not one; namely, agent, observer, and environment. In our formalism, we defined affordance
from these perspectives with the hope that these
different, but related, definitions will be of help in
clarifying the discussions of the concept. We consider only the agent and observer perspectives to
be relevant and provide the environment perspective only as a means to tie the proposed formalism
to some philosophical discussions of the concept.
Affordances (agent and observer perspective) are
relations that reside inside the agent. At first glance,
this claim can be seen to go against the common
view of affordances in ecological psychology which
places affordances in the agent–environment system, rather than in the agent or in the environment
alone. However, we argue that representing these
relationships explicitly inside the agent does not
contradict the existence of these relations within
the agent–environment system. As discussed in
the previous bullet, we are interested in how the
relations within the agent–environment system are
viewed from the robot’s perspective. We argue
that these agent–environment relations can be
internalized by the robot as explicit (though not
necessarily symbolic) relations and can enable
robots to perceive, learn, and act within their environment using affordances.
Affordances are acquired relations. The acquisition aspect is an essential property of the formalization, yet the method of acquisition is irrelevant.
Here, acquisition is used as an umbrella term to
denote different processes that lead to the development of affordances in agents including, but not
limited to, evolution, learning, and trial-and-error
based design. In some discussions, affordances
have also been classified based on the process of
acquisition leading to: innate affordances (J. Norman, 2001) that are acquired by the species that the
organism belongs to through evolution; learned
affordances (E. J. Gibson, 2000), that are acquired
by the interaction of the organism with its environment during its life-time; and designed affordances
(Murphy, 1999) that are “acquired” by the robot
through a trial-and-error design phase.
464
•
•
The formalism implies that in order to have robots
acquire affordances within their environment, first,
relation instances that pertain to the interaction of
the robot with its environment need to be populated, and then these relation instances should be
merged into relations through the formation of
equivalence classes. The issues of how relation
instances can be generated, and how relation
instances can be merged into affordance relations
are open problems that beg to be studied. However, we would like to claim that the acquisition
process, regardless of the method being used,
would lead to two major gains. First, it should
lead to perceptual speed-up: a reduction in perceptual processing requirement after acquisition. This gain has already been mentioned as a
major motivation for affordances and E. J. Gibson’s studies on the mechanisms of learning of
affordances already provide clues to how such a
speed-up can be achieved. Second, we argue that
acquired relations would naturally be in the socalled body-scaled metrics, in agreement with the
affordance studies in ecological psychology.
Affordances encode “general relations” pertaining
to the agent, environment interaction, such as: balls
are rollable. Naturally, exceptions to these general
relations, such as “the-red-ball-on-my-table is not
rollable (since it is glued to the table)” do exist.
However, unlike affordance relations, these “specific relations” possess little, if any, predictive
help over other cases, such as whether the-blueball-on-my-table is rollable or not. The proposed
formalization, differs from the existing formalizations, by explicitly stating that an affordance is a
relation that exists between equivalence classes,
rather than a relation instance, and embodies
power to generalize into novel situations.
The implication for autonomous robot control is the
existence of two control systems; an affordancebased one that acquires and uses general relations,
and a complementary add-on system that complements the affordance-based system by learning its
exceptions. It is interesting to note that this implication is also in agreement with Neisser’s cognitive model (Neisser, 1976) which suggested
an object-recognition system that complements
affordances.
Affordances provide a framework for symbol formation. Symbolic representation and processing
•
are important issues in both cognitive science and
robotics. However, the problem of how symbols
are related to the raw sensory-motor data of the
agent, also known as the symbol grounding problem (Harnad, 1990), still attracts considerable
research focus. In the proposed formalism, the
categorization of raw sensory-motor perceptions
into equivalence classes can be considered as a
symbol formation process. We would like to point
out that the formation of equivalence classes is
intertwined with the formation of relations. In this
sense, the formation of symbols is not an isolated
process from the formation of affordance relations. Instead, as also argued by Sun (2000), these
symbols would be “formed in relation to the experience of agents, through their perceptual/motor
apparatuses, in their world and linked to their
goals and actions.” Finally, it will be an interesting
challenge to link the different equivalence classes
(entity, behavior, affordance, effect, and agent)
with the lexical and semantic types in natural languages.
Affordances provide support for planning. Planning is described as “an abstract, explicit deliberation process that chooses and organizes actions by
anticipating their expected outcomes” to achieve
“some prestated objectives” (Ghallab, Nau, &
Traverso, 2004). The link between affordances and
planning was first noted by Amant (1999) within
the human computer interaction domain. Later,
Steedman (2002a, 2002b) formalized affordances
such that they could be used for planning, as
reviewed earlier. Steedman pointed out that
planning is closely related to the discussion on
affordances, even when they are not directly attainable to the agent. For example, we can perceive the
graspability of a mug, even when it is not within
our reach and not immediately graspable. Even for
a seemingly simple task such as this, a plan (such
as stand-up, walk, and bend toward) is needed to
make the graspability of the mug evident to us.
In classical planning, also commonly known as
STRIPS planning (Fikes & Nilsson, 1971), systems work with operators which consist of three
main components: precondition, action, and effect
denoting the initial requirements for the action to
be applied, the atomic action to be taken, and the
expected changes to be inflicted in the environment, respectively. The planner uses the operators,
Şahin et al.
planner. These are important limitations, which
can be addressed by the operator structure implied
by the formalism. Not surprisingly, however, these
limitations were discussed and addressed by some
of the relatively more recent work in robotics,
such as Firby’s reactive action packages (RAPs;
Firby, 1989). Firby defined RAPs as a representation that groups together and describes all known
ways to carry out a task in different situations. A
RAP is composed of a success test and a number
of methods, with each method consisting of a context and a task network. The task network denotes
a partial plan which may use one or more RAPs,
whereas the context specifies the situation that the
method is applicable, similar to the precondition
component of STRIPS. The success test typically
contains an algorithm which judges whether the
application of a method was successful or not.
Note that all methods are subject to the same success test. The traversability relation can be represented as a RAP as follows:
which are assumed to be pre-coded, to generate a
sequence of operators, such that its application
would take the system from a given initial state to
a desired goal state.
We argue that the proposed formalism creates
relations that can also be used as operators for
planning. An affordance relation is indexed by its
effect and include tuples which store how that particular effect can be achieved. For instance, the
<entity> and <behavior> components in the proposed formalism, can be considered to correspond
to the precondition and action components in the
STRIPS representation. A major difference between
the STRIPS representation and the affordance representation is the way the operators are indexed.
In STRIPS, operators are indexed by their actions,
whereas affordances (as our operators) are indexed
by their effects. For instance, the proposed formalism implies that the traversability affordance
can be represented as a planning operator:
(index: traversed
effect: traversed
(entity: river, behavior: swim)
(entity: road, behavior: walk)
)
(index: traverse
(success-test: traversed?)
(context: river,
task-network: swim)
(context: road,
task-network: walk)
)
whereas, the same relations could represented
using two different operators in STRIPS as:
(index: swim
action: swim
precondition: river,
effect: traversed
)
(index: walk
action: walk
precondition: road,
effect: traversed
)
The different representations of operators have
important implications for planning. In STRIPS,
the whole environment is assumed to be perceived before the planner can start planning, a
plan effectively consists of a sequence of actions
(since operators are indexed by their actions), and
that any change in the environment during execution may require the plan to be revised by the
465
We believe that the similarity between the RAP
representation and the affordance representation
is interesting because they were developed in different contexts and that this needs to be further
investigated.
Most of the discussions regarding the proposed formalism and their implications toward autonomous robot
control highlight the need for further studies on physical
robot systems. In the next section, we will briefly report
some preliminary results obtained from autonomous
robots and link them to the discussions above.
7
Toward Affordance-Based Robot
Control: Preliminary Results
We have conducted a number of preliminary experiments with robots to implement and evaluate certain
466
aspects of the proposed formalism. Specifically, we
studied how a mobile robot can learn to perceive the
traversability affordance in a room filled with spheres,
boxes, and cylinders. Ugur, Dogar, Çakmak, and Sahin,
(2007; an extended version appears as Ugur, Dogar,
Çakmak, & Sahin, 2006), defined traversability as the
ability “to pass or move over, along, or through.”
Hence, the environment is said to be traversable in a
certain direction if the robot moving in that direction
is not forced to stop as a result of contact with an
obstacle. Thus, if the robot can push an object by rolling it away, that environment is said to be traversable
even if the object is on robot’s path and a collision
occurs. In this view, which is different from simple
obstacle avoidance, boxes and cylinders in upright
positions become non-traversable and spheres become
traversable.
The experiments were first conducted in a physics-based robot simulator, and then verified on the real
robot. The robot and its simulated model used a 3-D
range scanner as the main sensor. The environment
typically contained one or more objects, with arbitrary
size, orientation and placement, in the frontal area of
the robot. The robot used its 3-D range scanner to create a range image. The image was split by a 30 × 30
grid. Thirty-nine low-level feature detectors were
applied to each of the grids generating a raw perceptual vector of size 35,100. The robot then executed
one of the seven pre-coded movement behaviors,
which ranged from turn-sharp-right to turn-sharp-left,
and recorded whether it was able to successfully
traverse or not, through its odometry. Hence, the robot
was able to generate relation instances; entity being
the raw perceptual vector, behavior being the index
(range 1–7) of the movement behavior executed, and
effect being 1 or 0 indicating success or failure. Each
experiment consisted of an exploration phase, during
which the robot accumulated a number of relation
instances, a training phase in which entity equivalence
classes were learned from these relation instances, and
an evaluation phase for testing. The training phase
was carried out in two steps. First, the relevant perceptual features were extracted, and then a classifier was
trained using these relevant features to learn the mapping from feature space to the effects.
We will report three experiments and discuss the
results with respect to the proposed formalism. In the
first experiment, the robot explored traversability in
setups where it was faced against a random collection
of objects dispersed in the environment. The robot
generated relation instances from 2,000 different random setups, and used them to learn the traversability
of each movement behavior. After training, the robot
was able to predict whether the environment affords
traversability for a given behavior with around 95%
success in 1,000 random setups generated for evaluation. A sample course of the simulated robot in a room
full of different objects is shown in Figure 3.
We would like to discuss a number of points to
link these results back to the formalism. First, entity
equivalence classes were discovered by the trained
classifiers. As a preliminary study, we would like to
note that only the formation of entity equivalence
classes were studied, while assuming that behavior
and effect equivalences are pre-coded. Second, the
relations between these three equivalence classes were
explicitly represented inside the robot. Third, the
acquisition process used, that is learning, generated
perceptual economy for the robot. Our analysis
showed that only 1% of the raw feature vector was relevant for perceiving traversability and that these relevant features were grouped on the range image with
respect to the direction of the movement as shown in
Figure 4.
In the second experiment, the trained robot was
tested in setups that were inspired by Warren and
Whang’s (1987) study on walking through apertures.
Warren and Whang studied the perception of passthrough-ability affordance, where participants, presented with apertures of varying width, were asked
whether the apertures afford walking through or not.
The results showed that the aperture-to-shoulder-width
ratio is a body-scaled constant for this affordance, and
that a critical point existed for the subject’s decision.
In a similar vein to these experiments, we placed two
box-shaped objects in front of the robot and tested the
robot’s predictions of traversability affordance for
apertures with different widths. As shown in Figure 5,
the robot is able to correctly perceive the affordances
of pass-through-able apertures, where critical passable width is clearly related to the robot’s width.
We would like to point out that these results can
be viewed from two different perspectives: observer
and agent. An observer of this experiment would
indeed conclude that the traversability affordance of
the robot depends upon the ratio of the aperture width
to the robot’s width. Although this conclusion would
be correct, it bears little relevance to the nature of the
Şahin et al.
467
Figure 3 On the left: The course of the trained robot in a virtual room cluttered with 40 objects. The robot tries to go
forward while making as few and small turns as possible (Ugur et al., 2007) (©2007 IEEE). On the right: Instances from
the trajectory of the robot. In (a) a turn to the left was afforded, and the robot drove toward the spherical object. In (b), although the robot made contact with the box on the right, it selected forward move. In (c), the only behavior that was afforded was turning left sharply. In (d), none of the behaviors were afforded because the robot got too close to the wall
and all seven behaviors in the robot’s repertoire would have caused a collision. Note the slight difference between (c)
and (d), where the robot was able to find the small open-space toward its left in (c).
Figure 4 The relevant grids in the range image for each action. A grid is marked as relevant if any of the features extracted from it were learned to be relevant. Note how the relevancy region is correlated with the direction of the movement behavior (Ugur et al., 2007) (©2007 IEEE).
Figure 5 Three experiments for evaluating pass-through-ability for the robot. For each experiment, the view from the
observer’s and the robot’s perspectives are shown. The views from the robot’s perspective consisted of range images of
the environment as generated by the 3-D range scanner of the robot. In (a) the width of the aperture is too narrow
whereas in (b) it is wide enough to support the pass-through-ability. (c) This shows the case where the aperture is slightly toward the right of the robot. In this case, it is important to note that the aperture seen from the robot’s point of view is
actually narrower than the one in (b). Yet, the robot successfully took this factor into account in its decision.
468
sensory-motor processing done in the robot. As
described above, the robot does not possess the concept of object, aperture or width at any perceptual
level and the affordance relation that exists within the
robot is different from the affordance relation perceived by the observer. Also, we argue that the existence of a body-scaled relation from an observer’s
perspective merely indicates that the relation was
acquired through the physical interaction of the robot
with its environment.
In the third experiment, the robot explored traversability when it was faced with a lying cylinder,
which may or may not afford traversability to the
robot depending on its relative orientation. After training, the robot was tested with spheres, boxes, and
upright cylinders, objects that it had not interacted
with before. Yet the robot was able to predict that
boxes and upright cylinders were non-traversible
(both 100% success), and that spheres were traversible (83% success). We claim that, in this study, the
robot learned “general relations” that pertained to its
physical interaction with the environment and that
these relations were useful for making successful predictions about the traversability of novel objects.
The results reported here were obtained from our
preliminary studies, and provide only limited proof
for some of the implications discussed in the previous
section. A fully-fledged evaluation of all the implications put forward, requires a long-term research effort.
Our on-going work has focused on an extended scenario, where the movement behaviors span a continuous range of movements, instead of a discrete set, and
where the effects are no longer grouped into success
and failure. Also, the use of equivalence classes as a
symbols for “affordance-based planning” remains a
challenge for future studies.
8
Conclusions
The concept of affordances has been both inspirational and hazy (which may have contributed positively to its influence over a wide-range of fields). In
this article, we have reviewed the discussions around
the concept and explored how the concept can be formalized to be utilized in autonomous robot control.
Toward this end, we have taken the view that our
thinking should be led toward the point J. J. Gibson
indicated, rather than return to the point that he had
already reached. As a consequence, the proposed formalism extended the Gibsonian notion of affordances
in two major aspects. First, although the proposed formalism agrees with the Gibsonian view that affordances
are relations within the agent–environment system, it
differs by arguing that these relationships can also be
projected onto the agent. Hence, unlike the prior
formalizations, the proposed formalism stops short
of providing any “perspective-free” definitions for
affordances, since it is not considered to be relevant
for using the concept in robot control. The philosophical issue of whether an affordance can be defined
without reference to any perspectives is possible or
not, and how much such a definition would contribute
to the development of a “theory of information
pickup” in agents, which constituted J. J. Gibson’s
main motivation, will remain as topics for further discussion.
Second, the proposed formalism differs from the
Gibsonian view because it argues that affordances (for
instance, as viewed from the agent perspective) can be
internalized and explicitly represented within the
agent. The Gibsonian view may reject this extension
by arguing that J. J. Gibson had developed the concepts of affordance and direct perception to object the
existence of “representations” in the organism. We do
not agree with such an argument. In our understanding, J. J. Gibson objected to the view that perception
has to create a generic world model, which has been
often referred as “representation,” over which the
organism infers whether an affordance exists or not.
He argued that affordances are directly perceivable,
that is, without using a world representation and without making inferences. The proposed formalism represents relations, not world models, within the robot
and therefore we claim that it does not conflict with
the J. J. Gibson’s line of thinking.
Extending an already controversial term such as
affordance is bound to be subject to criticism. One of
the previous commentators on our project warned us
of the dangers of being drawn into the heated debate
over the term, and suggested that a related-sounding
but different term, such as “affoodance,” might relieve
us from such debates.12 This difficult dilemma is
expressed in our title which begins with “to afford or
not to afford.” We believe that conceiving new terms
without properly relating them to already existing
terms does more harm than good. Instead, in this article, we have presented our formalization and defini-
Şahin et al.
tion of the concept according to our understanding of
it and leave the final judgment to the readers.
Finally, we would like to note that the implications
of the proposed formalism on the development and
implementation of an affordance-based robot control
architecture is our current and on-going work in the
MACS project. Although we believe that there are many
challenges ahead toward this goal, the ideas proposed in
this article will be of help to guide us on this quest.
Notes
1
In this article, the terms organism, animal and agent will
be used interchangeably. The use of organism and animal
will be mostly confined to discussions related to psychology, and the use of agent to discussion related to robotics.
2
More information is available at http://macs-eu.org
3 Warren and Whang (1987) defined eye height as the
height at which a person’s eyes would pass through the
wall while walking and looking straight in a natural and
comfortable position.
4 The second layer is about inter-personal perception and is
not discussed here.
5 What the robot actually learns about objects is the most
probable rolling direction of the objects with respect to
their principal axis. Hence, after the learning phase, the
robot knows that the bottle rolls perpendicular to its principal axis, and the toy car rolls parallel to its principal
axis.
6
Steedman’s actual formalization requires at least a basic
presentation of linear dynamic event calculus and lambda
calculus. Since we do not have the space for these here,
we restrict ourselves to the prose definition. For a complete account of this formalization, see Steedman (2002b).
7 The formalization of an affordance as a relationship will
be developed in the next section.
8 In the rest of our discussions, we will use the term agent
instead of organism or animal.
9 Discussions of affordances also spread into concepts such
as species, evolution and design. This definition can be rephrased to take such discussions into account, as: An
affordance is a relation between the organism (or the species) and its environment as acquired from the interaction
of the two, through either learning, evolution or trial-anderror based design.
10 The relationship between affordances and matched filters
was questioned/pointed out by Barbara Webb during discussions at the Dagstuhl Seminar “Towards AffordanceBased Robot Control.”
11 In robotics, behaviors are often considered to be atomic
units, and the invariants of a group of behaviors can sound
469
meaningless. However, if one implements behaviors as a
set of parameters whose values determine the interaction,
then invariants of behaviors can be discovered on these
parameters, similarly to the discovery of invariants in
entity equivalence classes.
12 We would like to acknowledge R. Arkin of Georgia Institute of Technology, GA, USA, for this.
Acknowledgments
This work was partially funded by the European Commission
under the MACS project (FP6-IST-2-004381).
We would like to acknowledge that the ideas presented in
this article were conceived and partially shaped through our
discussions and exchange of knowledge within the MACS
project and would like to give credit all project partners,
namely: Erich Rome and his colleagues at Fraunhofer IAIS,
Lucas Paletta and his colleagues at Joanneum Research, Patrick
Doherty and his colleagues at Linköpings Universitet and
Georg Dorffner and his colleagues at Österreichische Studiengesellschaft für Kybernetik. Our special thanks go to Erich
Rome, who read the first version of the article and provided
many constructive comments for its improvement.
We would like to thank Barbara Webb and Joel Norman
for commenting on the first version of the article. Last but not
least, we would like to thank Tony Chemero and our anonymous reviewer, for providing constructive comments that led to
the clarification of the stance of our formalism against the Gibsonian view.
References
Amant, R. S. (1999). User interface affordances in a planning
representation. Human Computer Interaction, 14, 317–354.
Arkin, R. C. (1998). Behavior-based robotics. Cambridge, MA:
MIT Press.
Arkin, R. C., & Balch, T. (1997). AuRA: Principles and practice in review. Journal of Experimental and Theoretical
Artificial Intelligence, 9, 175–189.
Brooks, R. A. (1990). Elephants Don’t Play Chess. Robotics
and Autonomous Systems, 6 (1/2), 3–15.
Chemero, A. (2000). What events are. Ecological Psychology,
12, 37–42.
Chemero, A. (2003). An outline of a theory of affordances.
Ecological Psychology, 15, 181–195.
Chemero, A., Klein, C., & Cordeiro, W. (2003). Events as
changes in the layout of affordances. Ecological Psychology, 15, 19–28.
Connell, J. H. (1992). SSS: A hybrid architecture applied to
robot navigation. In Proceedings of the IEEE Interna-
470
tional Conference on Robotics and Automation (pp. 2719–
2724). Los Alamitos, California.
Cooper, R., & Glasspool, D. W. (2001). Learning action
affordances and action schemas. In R. M. French & J. P.
Sougne (Eds.), Connectionist models of learning, development and evolution (pp. 133–142). London: Springer-Verlag.
Cornus, S., Montagne, G., & Laurent, M. (1999). Perception of
a stepping-across affordance. Ecological Psychology, 11,
249–267.
Cos-Aguilera, I., Canamero, L., & Hayes, G. M. (2003). Motivation-driven learning of object affordances: First experiments using a simulated Khepera robot. In F. Detjer, D.
Dörner and H. Schaub (Eds.), Proceedings of the 9th
International Conference in Cognitive Modelling (ICCM).
Bamberg, Germany.
Cos-Aguilera, I., Canamero, L., & Hayes, G. M. (2004). Using
a SOFM to learn object affordances. In Proceedings of the
5th Workshop of Physical Agents. Girona, Catalonia, Spain.
Dorffner, G., Irran, J., Kintzler, F., & Poelz, P. (2005). Robotic
learning architecture that can be taught by manually
putting the robot to action sequences (Tech. Rep.). The
Austrian Research Institute for Artificial Intelligence
(OFAI). (MACS Project Deliverable 5.3.1.)
Duchon, A. P., Warren, W. H., & Kaelbling, L. P. (1998). Ecological robotics. Adaptive Behavior, 6, 473–507.
Fadiga, L., Fogassi, L., Pavesi, G., & Rizzolatti, G. (1995).
Motor facilitation during action observation: a magnetic
stimulation study. Journal of Neurophysiology, 73, 2608–
2611.
Fikes, R., & Nilsson, N. J. (1971). Strips: A new approach to
the application of theorem proving to problem solving.
Artificial Intelligence, 2, 189–208.
Firby, J. (1989). Adaptive execution in complex dynamic
worlds. Ph.D. thesis, yaleu/csd/rr #672, Yale University.
Fitzpatrick, P., Metta, G., Natale, L., Rao, A., & Sandini, G.
(2003). Learning about objects through action – initial
steps towards artificial cognition. In Proceedings of the
2003 IEEE International Conference on Robotics and
Automation (ICRA) (pp. 3140–3145).
Fogassi, L., Gallese, V., Buccino, G., Craighero, L., Fadiga, L.,
& Rizzolatti, G. (2001). Cortical mechanism for the visual
guidance of hand grasping movements in the monkey – a
reversible inactivation study. Brain, 124, 571–586.
Fritz, G., Paletta, L., Kumar, M., Dorffner, G., Breithaupt, R.,
& Rome, E. (2006). Visual learning of affordance based
cues. In S. Nolfi et al. (Eds.), From animals to animats 9:
Proceedings of the 9th International Conference on Simulation of Adaptive Behaviour (SAB) (pp. 52–64). Roma,
Italy. Berlin: Springer-Verlag.
Gallese, V. (2000, May). Agency and motor representations:
new perspectives on inter-subjectivity. SC working papers
2000-6. Workshop on Autism and the Theory of Mind.
Lyon.
Gallese, V., Fadiga, L., Fogassi, L., & Rizzolatti, G. (1996).
Action recognition in the premotor cortex. Brain, 119,
593–609.
Ghallab, M., Nau, D., & Traverso, P. (2004). Automated planning theory and practise. San Francisco: Elsevier.
Gibson, E. J. (2000). Perceptual learning in development:
Some basic concepts. Ecological Psychology, 12, 295–
302.
Gibson, E. J. (2003). The world is so full of a number of things:
On specification and perceptual learning. Ecological Psychology, 15, 283–288.
Gibson, E. J., Riccio, G., Schmuckler, M. A., Stoffregen, T. A.,
Rosenberg, D., & Taromina, J. (1987). Detection of the
traversability of surfaces by crawling and walking infants.
Journal of Experimental Psychology, 13, 533–544.
Gibson, J. J. (1966). The senses considered as perceptual systems. Boston: Houghton Mifflin.
Gibson, J. J. (1986). The ecological approach to visual perception. Original work published 1979. New Jersey: Lawrence Erlbaum Associates.
Greeno, J. (1994). Gibson’s affordances. Psychological Review,
101, 336–342.
Harnad, S. (1990). The symbol grounding problem. Physica D,
42, 335–346.
Humphreys, G. (2001). Objects, affordances … action!!! The
Psychologist, 14, 5.
Irran, J., Kintzler, F., & Pölz, P. (2006). Grounding
affordances. In R. Trappl (Ed.), Cybernetics and systems.
Vienna: Austrian Society for Cybernetic Studies.
Jones, K. S. (2003). What is an affordance? Ecological Psychology, 15, 107–114.
Kinsella-Shaw, J. M., Shaw, B., & Turvey, M. T. (1992). Perceiving walk-on-able slopes. Ecological Psychology, 4,
223–239.
Koffka, K. (1935). Principles of gestalt psychology, New York:
Harcourt, Brace.
MacDorman, K. F. (2000). Responding to affordances: Learning and projecting a sensorimotor mapping. In Proceedings of 2000 IEEE International Conference on Robotics
and Automation (ICRA) (pp. 3253–3259). San Fransisco,
California, USA.
Mark, L. S. (1987). Eyeheight-scaled information about
affordances: A study of sitting and stair climbing. Journal
of Experimental Psychology: Human Perception and Performance, 13, 361–370.
McGrenere, J., & Ho, W. (2000). Affordances: Clarifying and
evolving a concept. In S. Fels, P. Poulin (Eds.), Proceedings of the Graphics Interface (pp. 179–186). Toronto.
Michaels, C. F. (2003). Affordances: Four points of debate.
Ecological Psychology, 15, 135–148.
Murphy, R. R. (1999). Case studies of applying Gibson’s ecological approach to mobile robots. IEEE Transactions on
Systems, Man, and Cybernetics, 29, 105–111.
Şahin et al.
Neisser, U. (1976). Cognition and reality: Principles and
implications of cognitive psychology. San Francisco:
W. H. Freeman and Co.
Neisser, U. (1994). Multiple systems: A new approach to cognitive theory. The European Journal of Cognitive Psychology, 6, 225–241.
Norman, D. A. (1988). The psychology of everyday things.
New York: Basic Books.
Norman, D. A. (1999). Affordance, conventions, and design.
Interactions, 6, 38–42.
Norman, J. (2001). Ecological psychology and the two visual
systems: Not to worry! Ecological Psychology, 13, 135–
145.
Norman, J. (2002). Two visual systems and two theories.
Behavioral and Brain Sciences, 25, 73–144.
Oudejans, R., Michaels, C., VanDort, B., & Frissen, E. (1996).
To cross or not to cross: The effect of locomotion on
street-crossing behavior. Ecological Psychology, 8, 259–
267.
Rizzolatti, G., Fadiga, L., Gallese, V., & Fogassi, L. (1996).
Premotor cortex and the recognition of motor actions.
Cognitive Brain Research, 3, 131–141.
Sanders, J. T. (1997). An ontology of affordances. Ecological
Psychology, 9, 97–112.
Steedman, M. (2002a). Formalizing affordance. In: W. Gray
and C. Schunn, Proceedings of the 24th Annual Meeting
of the Cognitive Science Society, LEA, Mahwah, NJ.
Washington D.C.: Lawrence Erlbaum.
Steedman, M. (2002b). Plans, affordances, and combinatory
grammar. Linguistics and Philosophy, 25.
Stoffregen, T. A. (2003). Affordances as properties of the animal environment system. Ecological Psychology, 15,
115–134.
Stoytchev, A. (2005a). Behavior-grounded representation of
tool affordances. In Proceedings of 2005 IEEE International Conference on Robotics and Automation (ICRA)
(pp. 18–22). Barcelona, Spain.
471
Stoytchev, A. (2005b). Toward learning the binding affordances
of objects: A behavior-grounded approach. In Proceedings of AAAI Symposium on Developmental Robotics (pp.
21–23).
Sun, R. (2000). Symbol grounding: A new look at an old idea.
Philosophical Psychology, 13, 149–172.
Susi, T., & Ziemke, T. (2005). On the subject of objects: Four
views on object perception and tool use. TripleC: Cognition, Communication, Co-operation, 3, 6–19.
Szokolszky, A. (2003). An interview with Eleanor Gibson.
Ecological Psychology, 15, 271-281.
Turvey, M. T. (1992). Affordances and prospective control: an
outline of the ontology. Ecological Psychology, 4, 173–
187.
Ugur, E., Dogar, M. R., Çakmak, M., & Sahin, E. (2006). The
learning and use of traversability affordance using range
images on a mobile robot (Technical Report No. METUCENG-TR-2006-03). Ankara, Turkey: Dept. of Computer
Eng., Middle East Technical University.
Ugur, E., Dogar, M. R., Çakmak, M., & Sahin, E. (2007). The
learning and use of traversability affordance using range
images on a mobile robot. In Proceedings of the 2007
IEEE International Conference on Robotics and Automation (ICRA), Roma, April 2007, (pp. 1721–1726).
Warren, W. H. (1984). Perceiving affordances: Visual guidance
of stair climbing. Journal of Experimental Psychology,
105, 683–703.
Warren, W. H., & Whang, S. (1987). Visual guidance of walking through apertures: body-scaled information for
affordances. Journal of Experimental Psychology, 13,
371–383.
Wehner, R. (1987). “Matched filters” – neural models of the
external world. Journal of Comparative Physiology A:
Neuroethology, Sensory, Neural, and Behavioral Physiology, 161, 511–531.
Wells, A. J. (2002). Gibson’s affordances and Turing’s theory
of computation. Ecological Psychology, 14, 140–180.
472
About the Authors
Erol Şahin received a B.Sc. in electrical and electronics engineering from Bilkent University in 1991, Turkey, an M.Sc. in computer engineering from Middle East Technical University (METU) in 1995, and a Ph.D. in cognitive and neural systems from Boston University,
USA, in 2000. He is an assistant professor at the Department of Computer Engineering at
METU and is heading the KOVAN Research Laboratory. He has been working on the
MACS project (http://www.macs-eu.org) to develop an affordance-based robot control
architecture. He has also been working on swarm robotics, and has edited two books and
a special issue on the topic.
Maya Çakmak has received a B.Sc. in electrical and electronics engineering, a minor
degree in mechatronics and an M.Sc. in computer engineering from the Middle East
Technical University, Turkey. She worked at the KOVAN Research Laboratory for two
years. She is now pursuing a Ph.D. in robotics at the College of Computing, Georgia Institute of Technology, Computational Perception Lab, 85 5th Street, Atlanta, GA 30332-0760,
USA. E-mail: [email protected]
Mehmet Remzi Dog
ar has received his B.Sc. degree from the Department of Computer
Engineering at Middle East Technical University, Turkey. He is currently pursuing an
M.Sc. degree in the same department. His research interests include intelligent robotics,
robotics learning, and robotic behavior development. Address: Department of Computer
Engineering, Middle East Technical University, 06531, Ankara, Turkey.
E-mail: [email protected]
Emre Ug
ur is currently pursuing his Ph.D. degree at the Computer Engineering Department, Middle East Technical University, Turkey. He has a B.Sc. degree in computer
engineering from METU, 2003. He obtained his M.Sc. degree in the same department
with a thesis on Direct perception of traversability affordance on range images through
learning on a mobile robot in 2006. Since 2004, he has been working as a researcher for
the MACS project. Prior to this, he worked for the Swarm-bots project (http://swarmbots.org). Address: Department of Computer Engineering, Middle East Technical University, 06531, Ankara, Turkey. E-mail: [email protected]
Göktürk Üçoluk received his B.Sc. and M.Sc. degrees in physics from Bosphorus University, İstanbul, Turkey, and his Ph.D. degree from the Middle East Technical University, Ankara, Turkey, in 1980, 1982, and 1989, respectively. He is currently an associate
professor of computer engineering at the Middle East Technical University. His recent
interests are in evolutionary computing. His other research interests include symbolic
algebraic computing, robotics, algorithms and natural language processing. Address:
Department of Computer Engineering, Middle East Technical University, 06531, Ankara,
Turkey. E-mail: [email protected]

To Afford or Not to Afford: A New Formalization of Affordances

Transkript

Benzer belgeler

Michael Francis Gibson, writer, art historian, art critic and scholar

i A DEVELOPMENTAL APPROACH FOR AFFORDANCE AND