Cati Vaucelle
bio
blog
portfolio
publications
♣ photos
RESEARCH - by Cati Vaucelle
I decided to explore the capturing of the video, from the toy perspective, to create unexplored visual perspectives and to merge storytelling and play to construct movies.
CONTEXT
INTERACTION DESIGN
Motivation. Whereas play emphasizes spontaneity and improvisation, video making necessitates structure and composition. I investigate what technology can add to the narrative and play experience. I use technology to offer visual feedback regarding how the scene looks like from the point of view of an imaginary audience.
With character toys, children create interrelationships and plots, a means to expose their social knowledge: knowledge about human beings and social relationships (Shantz, 1975).
If the toy has an immediately accessible visual perspective, a new world is opened to the child. The toy brings her into exploring visual and narrative perspectives of character props, expanding the discovery of her environment. The child storyteller enters the world of the movie maker. Cameras become part of a toy system showing how things look from a toy’s point of view. They can be integrated in Lego people, car drivers, and even coffee mugs!
The video process, supported by gesture induced editing, benefits children in practicing social interrelationships and visual perspective taking.
Scenario of Interaction. Picture This! analyzes the child’s gestures and conducts film assembly.
Here is a scene: the bear has a camera, the helicopter has a camera. As the kid moves the toys as part of their play, the gesture analysis system relies on their movements to shift which camera is filming from one camera to another. The gestures that I look for are simply to determine which camera should be shooting at one time or another, based on where the action is taking place, who the main character is, etc…
I also capture the audio of the kid telliing stories throughout the play session. The result is a sequence of shots, taken from multiple cameras, switching amongst each other. The kid is directing the switching through gestures, and modifying their storytelling based on who and what is in the visual at a given time.
Contributions. While tangible systems invite for capturing oral stories and videos, current systems do not benefit from the gesture interaction that children do while playing with toys.
Picture This! offers a gesture language for capturing and editing suitable for children in their toy environment. In Picture This! children gradually project themselves onto their toys, embedding persons they know in their stories and character toys, adopting a “god’s eyes view” to obtain a deeper understanding of their own stories.
The children alternate between actors and movie makers, orchestrating the scene with their favorite props. The playback mode in Picture This! invites children to revisit their movie; they “step away from their performance” children reflect on the outcome of their spontaneous play and character’s conversations. Visual spatial processing guides our movement. The ability to mentally manipulate objects, and imagine how an object appears if it is moved, is spatial cognition (Henderson et al., 1991).
Goal. I aim for Picture This! to invite children to practice spatial cognition between imagining the point of view of the toy, trying it out and correcting it. I designed Picture This! so that the child’s toy becomes a camera person as opposed to having the child hold a camera directly. As a child plays with the toy that holds the camera, its video feed is projected on a screen in front of her in real time.
The toy is the camera person (a) vs what the toy “sees” from “his” video feed (b).
This visual flow aims to motivate her in composing a final movie as she plays and explores her visual story. As the two dolls interact with one another, they alternate between their respective visual scenes. The child creates the conversation using direct speech for the toy characters. The child also uses a narrator voice to introduce the story and contextualize the scene.
I chose the interaction to function like a performance to not break the flow of traditional pretend play with character toys. Thus, my system incorporates the child’s gestures with the cameras and toy’s accessories as control functions to conduct film assembly.
By combining movie making during play and the improvisatory element, Picture This! naturally extends play to creative outcomes. Integrating a video editing algorithm to automate the editing process in a gesture object interaction allows one to get closer to the object of focus in a captured scene (for instance, a specific character). The video process, supported by gesture induced editing, benefits children in practicing social interrelationships as well as visual perspective taking, thus expanding creative storytelling in video composition.
In my observations with users aged 4-10, I observe how eight children create a movie with their toys using Picture This! Because the first iteration of Picture This! is designed for individual use, to study children’s interaction I installed the system at their home, or if the children requested, they came and interacted with it at ML research laboratory. The children were asked to bring their own character toys, to be outfit with my system and to record a movie during play.
In the first couple of minutes, children explored the system without explanation. After five minutes, a researcher clarified how to operate the recording and the playback. The children were invited to play as long as they wanted. Eight children worked independently between forty-five minutes for up to two hours. Their interactions were videotaped and transcribed for analysis.
Capture of storytelling at different levels of interaction
I conducted studies to break down the ability to learn this interface environment and take advantage of it, by age groups. Picture This! allows children to capture storytelling with physical artifacts at different levels of interaction, functionalities and mode of interaction can now be distinguished with a research goal for each age group:
Children were extremely methodical and attentive with the use of the video. While in pretend play, they sometimes stopped their story and carefully worked on their camera view angle, alternating between characters. They progressed between getting to have the doll in the picture, to a full shot of the doll, integrating specific backgrounds, discovering camera distortions and various camera angles, all facilitated by the size factor of the camera and its context. Children under age six seem to forget about the screen throughout their play, being exclusively immersed in their play. After playing with the toys provided, children took out toys from their bag or from their bedroom. They had selected their favorite toys to be used with the system.
I noticed that even if some of the children removed the camera from a proposed character toy, they always put it back on top of another one. They did not use the camera detached from a toy as they would have done with a regular video camera. They were keen to explore the visual point of view of the toys they carry around. They found play-like justifications for the wires. A child said, regarding a rubber band from the camera that covers half the face of his toy: “well it’s kind of normal, cause they wear something in front of their mouth sometimes. Like a mask!”. The children above eight years old mastered the full system, being able to coordinate the dolls back-and-forth and to control the video, understanding the interaction between preview, record and playback.
After twenty minutes of playing, the gestures with the dolls became parts of the children’s vocabulary.
Even though a controlled study would support my qualitative observations, Picture This! seems to allow children to capture storytelling with physical artifacts at different levels of interaction. Functionalities and mode of interaction could be distinguished with a specific cognitive goal for each age group. For children under 6 years old, Picture This! functions as a video performance system with video snippets of the child’s play, with only one of the two toys carrying a camera. The preview seems to benefit them in developing the spatial-visual coordination while playing with their favorite toys and telling stories. For older children, Picture This! allows them to test visual angles and assemble a movie as they play with their toys and tell their stories alternating between direct speech and narrator voice, providing spatial and temporal context to their stories. The recording and playback modes seem to enable older children to use their social perspective taking visually and through storytelling.
I remarked that the youngest children (under eight years old) transferred their personal characteristics into the toys. For instance, a doll dances because the child takes dancing lessons. Or a doll takes her first picture, because this is the fist time the child takes a picture herself. Another child shakes the doll while saying: “Shake! Shake! I want to be in the camera!” and she shakes her own body. Older children (over eight years old) talked to the dolls, giving them directions for the movie. A child brought a doll to her face as if the doll had a mind of its own to say, “You don’t carry your wand like that. You don’t put the wand at people like that!”. Children navigate from transferring their own lives onto their toys and attributing human characteristics to the toys. All the children in my evaluation developed spontaneous conversations between the character toys, testing their social knowledge and perspective taking. The following is an excerpt of a video story by Jeremy, ten years old:
D1: “Hi! My name is Fred what's yours?”
D1:“Over there in the great yellow mountains, but there is a giant blocking the way. We need to take down the giant so that we can find the treasure.”
D2:“sounds good to me, when do you wanna go?”
D1:“how about right now?”
D2:“ “ok let's go” Narrator voice: and they walked off to the mountains to destroy the giant and get the Peruvian treasure.
D1: "tutututututut" (walking the dolls though the yellow mountains.) Then in front of the giant, the child says with the doll in the video frame:
D2: "hey you evil sid cops, surrender! Face the rest of us! We are superior and strong! We shall take you down!" Then the child uses one of the two dolls to take a video of the giant and says (taking the voice of the giant)
Giant: “I shall take you down first, face the rest of me!”
Real time visual perspectives by playing with video character toys in Picture This! allow children to develop visual perspective taking in determining where objects are located relative to another agent, or whether the agent can see a particular object (Michelon and Zacks, 2006). The high level of concentration exhibited by the children demonstrated how challenging it is for a child to find the right angle and distance between the object and the camera, and between the two objects.
DISCUSSION
In commercial technology with the apparition of the camcorder and its preview display, the relationship between the actor and the cameraman is transformed. The actor has more control over how her actions are represented in the global scene. Through the preview display, the actor is given a real time visual feedback and can adopt different postures accordingly. In Picture This! the traditional camera-human relationship is also modified. The perspective effort needed is demonstrated through the spatial and visual coordination, managing the right angle for the right doll at the right moment in time, and also while acting out a story with the toys. The point of focus of the movies being the characters, children are guided towards creating a conversation which provokes a shift in perspective (Ziegler et al., 2005). Children have an object to focus on for their movie that allows them to iterate back-and-forth, stepping back from the scene and immersing themselves into it. Children gradually project themselves onto their toys, embedding persons they know in their stories and character toys, and adopting a “god’s eyes view” to obtain a deeper understanding of their own stories. The children alternate between actors and movie makers, orchestrating the scene with their favorite props.
The playback mode in Picture This! invites children to revisit their movie; they “step away from their performance” and reflect on the outcome of their spontaneous play and character’s conversations. Visual spatial processing guides our movement. The ability to mentally manipulate objects, and imagine how an object appear if it is moved, is spatial cognition (Henderson et al. 1999). We observed that Picture This! invites children to practice spatial cognition between imagining the point of view of the toy, trying it out and correcting it. Children were motivated in seeing “how it looks like out of a toy’s eyes” and expressed the wish to have action figures take video at their home; to have Lego people with an eye socket in which the camera could be inserted; to mount the Picture This! system on a racing car to have the point of view of the driver; and to have a waterproof version of Picture This! to capture videos under water with the child’s toys. Because all children were keen to keep the Picture This! camera on their favorite toys instead of removing the camera system separately from a character prop, various objects could be given a camera spot and generate movies based on how they are handled.
Playing is about spontaneity and improvisation, while editing a movie is about structure and composition. Movie making can have a bit of both. For Picture This!, I chose a gesture based interaction for movie making because of its advantage to integrate well with play. The trade-off of my system consists of being a movie making system more than being exclusively about role playing. Picture This! invites for the discovery of unique angles and point of view, its gesture based interaction facilitating the movie making flow. I found that Picture This! invites children to experience with movie editing while playing with their toys. It works as a new mode of video expression and creation through which children are drawn to explore unique visual and storytelling perspectives.