附录Gait Adaptation in a Quadruped Robot1. IntroductionA short time after birth a foal can walk and then run. It is remarkable that the animal learns tocoordinate the many muscles of the legs and trunk in such a short period of time. It is not likely that any learning algorithm could program a nervous system ab initio with so few training epochs. Nor is it likely that the foals locomotor controller is completely determined before birth. How can this a- bili ty be explained? How can this ability be incorporated into the control system of a walking machine?Researchers in biology have presented clear evidence of a functional unit of the central nervous system, the Central Pattern Generator (CPG), which can cause rhythmic movement of the trunk and limb muscles(Grillner and Wallen, 1985). In adult animals, the output of these cells can generate muscle activity that is very similar to activity during normal walking, even when sensory feedback has been eliminated (Grillner and Zangger, 1975). The CPG begins its ac- tivity before birth, although its activity does not appear to imitate the details of a particular walking animal, it is apparently correlated with the animals class, i.e., amphibian, reptile, mammal, etc. (Bekoff, 1985; Cohen, 1988).Apparently, the basic structure of the CPG network is laid down by evolution. How is this basic structure adapted to produce the detailed coordination needed to control a walk- ing animal?The answer to this question is important to robotics for the following reason. CPGs have been well studied as a basic coordinating mechanism (Cohen et al., 1982; Bay and Hemami, 1987; Matsuoka, 1987; Rand et al., 1988; Taga et al., 1991; Collins and Stewart, 1993; Murray, 1993; Zielinska, 1996; Jalics et al., 1997; Ito et al., 1998; Kimura et al., 1999). However, the details of how this system can automatically adapt to control a real robot are not clear. A good goal would be to describe a general strategy for matching a generic CPG to a particular robot in real-time, with a minimal amount of interaction with the environment.Reinforcement learning has been applied on long time scales to certain problems in walking (learning coordination and basic leg movement) (Ilg and Berns, 1995), but the time scales of such an approach is too long to explain the quick learning of animals just after birth.The author suggests that part of the answer may be in the use of a number of simple innate internal models to evaluate the performance of the rapidly developing nervous system. These innate internal models could be used to adaptively tune CPGs during phases of rapid development. Figure 3 illustrates the training concept. A CPG generates a signal destined for a group of actuators (muscle) as well as a second signal, which is a copy of the signal sent to the actuators destined for an innate forward model. In biology, a copy of the motor signal is called an efference copy (Sperry, 1950). A forward model as described by Kawato (Kawato and Wolpert, 1998) is a functional model of the forward dynamics of the system.We use very simplified, innate forward models. These forward models predict the sensory expectation, or the desired consequence of CPG activity. This information is compared, and an adaptive rule then modifies the CPG.The author suggests that a handful of adaptive mechanisms may be used for rapid tuning of a generic CPG. For example, the adaptive model can be used to ensure coordination of limbs with the environment.The use of simplified, innate models is the most conservative stance possible. The intent is to make the fewest assumptions possible about the knowledge that the nervous system has about the body that it is trying to control. By demonstrating how this process may be used in a physical devicea robot we give compelling evidence that this approach is sufficient.This article reports on an investigation into how a group of forward models could be used to adaptively tune a CPG in a real robot. As these models are innate, we assume they are simple; it would not be satisfying if these models were as complex as the behaviors that they help generate. Secondly, these models should not be detailed, accurate models of the forward dynamics of the robot. If innate models are used, their simplicity prohibits them from detailing accurate information about the structure of the pattern generator. The study here supposes that once the CPGs have been tuned to produce basic locomotion, other, more general learning mechanisms would take over to create a more refined gait. These learning mechanisms might include reinforcement learning or supervised learning methods.2. ExperimentsThe results of three experiments using GEO-II are described in this section. These experiments require progressively more adaptation to the environment and culminate in adaptive walking behavior. The robot learns to adjust key parameters of the CPG network to allow the robot to walk within minutes.2.1 Exper