Given the tail model that we developed, we now want controllers to get the tail into a variety of positions. We chose three positions that seem the reflect a reasonable range of useful prehensile motion, and they are shown below.
Position 1 is a common quiescent (resting) state for monkeys, with their tails wrapped loosely around the body [6]. Position 2 is a gripping state, with the tail a distance away from the body and the last few links clutching an object. Position 3 is a hanging state, with the tail gripping an object directly overhead. Gravity is considered only in the case of position 3.
We next consider the model of the controller. For search to be at all possible, we need either a simple model or a very good starting guess. In an attempt not to bias the effort, we chose to use a simple model. Tests with the simulator suggest that approximately 1.5 seconds was enough to get the tail to any of the desired states. This time was broken into fifteen 0.1 second segments, during which time each motor could be either on (in a single direction) or off. This gives a 90 bit state, which is fairly reasonable.
Simulated annealing [7] was chosen as a search method. While genetic algorithms would have been preferable (and certainly apropos), they were not applicable since the problem was not separable (each time step sets up the conditions for the next) and therefore cross-over is not possible. No obvious way around this problem presented itself.
We want a controller that will not only get to the desired position, but also with a desired velocity and preferably with the minimum energy usage. Therefore, we define our cost function as:
where the f subscript represents final state and the d subscript represents desired state. The function sums the final state errors with the number of time steps for which the motor is on. Values of the leading coefficients were found via trial and error (described below).
The search was conducted by flipping the state of a randomly chosen motor at a randomly chosen time step, and then recalculating the dynamics to be able to calculate the new cost. If the cost is lower the new controller is kept. Otherwise, the new controller is kept with probability:
where t is the iteration step value, delta E is the change in cost and alpha is a cooling constant that must be tweaked. The search is stopped when no changes have been accepted for 200 iteration steps.
Note what is being searched over and what is held constant. We are searching for good controllers given the parameters found in the previous section. We are not varying the number of controllers or the value of the simulation parameters. Both of these offer prime areas for future exploration.
The final information that we need before search is the desired states. These are given below.
Note that the desired angular velocity in position 2 is not zero. In this case, the tail is considered to be gripping an item in its final state, and therefore should have some momentum when it gets there. Note also that we are looking for angles rather than positions. This is an explicit choice, whereby we consider that having the right relative link angles is more important to a "correct" controller than the right absolute position. This point is open for debate.
We began the search with both lambdas equal to one. It became quickly apparent that the controller pulse count term was of little use. The good controllers with respect to position and velocity were those which used less energy. Therefore lambda1 was set to zero. Test with the annealing rate showed improvement until alpha = 0.01, at which point further search was of no value.
It was further discovered that the controller was going through pains to match the desired velocities to the detriment of the positions (as is to be expected). Therefore, lambda2 was dropped to 0.1, which then reversed the dilemma. Both controllers are shown in the demo below, the reader can determine which is preferable for themselves.
Several methods could solve this problem. The first would be to allow more time, so that the links had more time to slow down. This solution was implement by extending the controllers to 2.0 seconds, with some (though not great) improvement. Another possibility is to allow the motors to operate in reverse as well, thereby giving them greater ability to slow down (this would not make building the tail any harder, since it already has to go backwards to get out of the desired position). This was not attempted due to time constraints. One point to consider in such an implementation is that the random flip described above should be likely biased against going negative.
A final point worth mentioning for the search is that the process should be repeated multiple times with different random seeds, to get a better feel for the search space.
A sample annealing for position 2 is shown below. The first plot shows the cost function with time, the second plot shows the controller that was found. Note the limited number of pulses.
This controller, and others, can be seen in action in the following controller demonstration applet.
The code for this search can be found below. simanneal handles the high level operations while runonce implements a single dynamics run. Note this is code requires the rk4.c and nrutil.c files from Numerical Recipes [8] to run.
