|
Here are some highlights from our group's research. Please check out the group page pages for more details on any of these projects, as well as the group's resources page for complete list of papers. Ok.
-----------------------
Dobie T. Coyote
The ability to learn is a potentially compelling and important quality for interactive synthetic characters. My most recent work is focused on developing a practical approach to real-time learning for synthetic characters. Our implementation is grounded in the techniques of reinforcement learning and informed by insights from animal training. It simplifies the learning task for characters by (a) enabling them to take advantage of predictable regularities in their world, (b) allowing them to make maximal use of any supervisory signals, and (c) making them easy to train by humans.
We built an autonomous animated dog that can be trained with a technique used to train real dogs called ``clicker training''. Capabilities demonstrated include being trained to recognize and use acoustic patterns as cues for actions, as well as to synthesize new actions from novel paths through its motion space.
A key contribution of this work is to demonstrate that by addressing the three problems of state, action, and state-action space discovery at the same time, the solution for each becomes easier.
Here is a short video that shows Dobie in action
What is going on in the Video
When Dobie starts out his "behavior system" (the part of his brain that decides what to do) "knows" how to beg, lie-down, shake, and how to approach the trainer. In the case of beg and shake he varies how he does them. He is also hardwired to associate the clicker with getting a "good thing" and the shake of the hand with a "bad thing." He can hear acoustic patterns such as your voice or whatever but has no idea that they are important. His "motor system" is also capable of lots of actions that his "behavior system" doesn't know he knows how to do, for example, roll-over, back-up, sniff, tumble...
So, he chooses his actions based on their learned consequences. Things that lead to a click increase in frequency, things that lead to a finger wag, decrease in frequency (Thorndike's Law of Effect.) In fact he actually learns the reliability of his actions so you can train him with a Variable Reward Ratio. So that is what the first segment of the video shows.
When he performs an action that is rewarded he looks back at what was true about how he did the action or what was true about the environment and uses that to correlate the observed reliability of the action with those things. So if you repeatedly give a verbal cue while he performs an action (like down) and reward him, he will eventually learn to associate the cue with the action (actually he first learns that paying attention to verbal cues is useful, and then learns that paying attention to a specific cue for a specific action is even more important.) That is what the second segment of the video is showing. Even though I am saying "dobie down" in the video, he has no built-in hardwiring that says to look for "down", rather he learns the characteristics of the acoustic pattern. What this means is that I could have just as easily used a bell, or spoken french or whatever.
The third clip is showing luring. When the hand takes the luring pose, Dobie is hardwired to try to get his nose to the hand (it is as if there is a nice piece of steak in the hand.) As he does this, his motor system may cause him to perform movements that he is capable of, but doesn't "know" about (i.e. He wouldn't otherwise perform spontaneously.) If we reward him while luring him, he looks at the pattern of movement he just performed. If it is something his behavior system already knows about (e.g., lying down) then "lying down" gets the credit for the reward. If on the otherhand it is novel, such as Roll-Over or Backing up, he eventually adds it to his behavior repertoire and starts performing it spontaneously, at which point you can start associating a cue with it.
The fourth clip shows him after about 5 -10 minutes of training. So he has learned 3 acoustic cues for 3 different actions (down, shake, and back) and in the case of back-up, via luring he actually learned that he could back up, and then trained him to associate a cue with it.
He also can be trained using shaping, so if you preferentially reward when he shakes high, he will tend to shake high when he shakes.
More Information
Back to Top
-----------------------
Duncan
Duncan was a Highland Terrier developed to explore a variety of issues ranging from spatial learning and object permanence to temporal representations for Synthetic Characters. He was also the first character developed using our C4 architecture. Duncan was featured in an interactive installation sheep|dog: trial by eire.
Here is a short video on sheep|dog: trial by eire:
Here is a short video on Damian Isla's work on Object Persistence
Here is a short video on Robert Burke's work on Temporal Representations for Synthetic Characters. Note, the main character may not look like Duncan, but he is a Highland Terrier in all but his form.
More Information
Back to Top
-----------------------
Alpha Wolf
Alpha Wolf was an exploration into the representations and processes required to allow synthetic characters to learn dynamic social hierarchies. Alpha Wolf was the basis of Bill Tomlinson's PhD work and took the social behavior of wolves as its inspiration. It was installed at Siggraph 2001, and won Honorable Mention at Ars Electronica 2002.
Here is a video that shows the installation at Siggraph 2002. One of the interesting aspects of Alpha Wolf was balancing user control vs. the character always staying in character,
More Information
Back to Top
-----------------------
Rover@Home
Rover@Home was an exploration into computer mediated remote interaction between Humans and Dogs. It was the basis for Ben Resner's Masters Thesis. Essentially, it was an internet implementation of Clicker Training. While on the surface it may seem a bit goofy, the fact remains that there are 60 million households that have dogs and some set of those dog owners would enjoy interacting with their dog over the internet when they can't be there in person. It is also a remarkably interesting design problem to develop an interface that its rewarding and interesting for each party when the parties to the interaction differ in motivational, cognitive, perceptual and motor abilities.
Here is a short video showing me using the system with my son's Silky Terrier Sydney,
More Information
Back to Top
------------------------
Silas
In the beginning there was Silas, the focus on my PhD work.
More Information
Swamped!
Swamped!
More Information
Max
Curious Machines
|
