A child playing with Picture This and his Naruto action figure

System components of Picture This

Picture This! in action!

 

Picture This! detail on the sensor used

 

"

 

Picture This! is a new input device for video capturing and editing! Designed for young children, ages five and up, it allows them to craft compelling movies through the motion analysis of their interaction with toys.

Algorithmic film assembly using toy gestures

Picture This, 2007

 

Vision

Children use toys to externalize and elaborate their mental representations. With character toys they create interrelationships and plots, a means to expose a child’s social interactions. If the toy had a visual perspective immediately accessible to the child, a new world would be opened to her. The toy could potentially bring the child into exploring visual and narrative perspectives of these character props, expanding her discovery and understanding of social interrelationships.

I propose that children could benefit from video to express and reflect thoughts on social interactions. One needs to be immersed in experience in order to learn from it, but sometimes adopting a “god’s eyes view” is a must for deeper grounding and understanding. After all, human development is a lifelong attempt to resolve the unsolvable tension between getting embedded and emerging from embeddedness. “Dwelling in” and “stepping back” are equally important to get the cognitive dance going. In Picture This, I research the relationship of perspective taking with personal video and personal voice using tangible materials.

 

Description

Picture This! is a video editing and capturing device designed for young children, ages five and up, allowing them to craft compelling movies through the motion analysis of their interaction with toys. Children’s favorite props alternate between characters and cameramen in a film. As children play with the toys to act out a story, they conduct algorithmic film assembly.

The Picture this tool is an audiovisual device that combines two digital video cameras and two accelerometers. The tool captures motions, video and sound in real-time while an algorithmic video editing system composes a movie from these inputs. A motion based editing engine fluidly assembles the film as its story is being narrated, while respecting the conventions of continuity editing, namely, a sequence of shots that appear to be continuous. This style of film editing is made possible in Picture This by detecting turn taking behaviors between the toys. Two toy props are augmented with video cameras and custom accelerometer hardware. They use the Picture This tool both as a doll hand-bag or a doll audiovisual recorder. The tool is flexible for a child to take the perspective of props she selected for her movie.

Picture This! detail on the camera

 

Interaction design

To engage with Picture This, children rely on their usual gesture interaction with their toys while narrating a story. I implemented a quick interaction language for movie editing with dolls: If a doll wants to be on the video, the doll needs to move. If it wants to be recorded it has to move rapidly twice. Two dolls alternate back and forth between being cameraman and the actor. The camera doll carries the camera to control the angle of view. To play back the movie, the two dolls have to be moved in synchrony, in essence, jumping together.

The motions that are detected by the system are anthropomorphized, jumping in synchrony at completion, shaking for attention, as if the doll wants to say: “film me, film me!” To master the interaction with Picture This, a child needs to alternate between projecting herself onto her toys and being the scene’s master-mind.

Example of the first line of a child's story, playing with Picture This!

 

More information

Download Picture This! leaflet.

A one minute video of Picture This!

A six minute video of children playing with Picture This!

Academic technical work

Full paper published and presented at Ubicomp 2008

Vaucelle, C. and Ishii, H. 2008. Picture this!: film assembly using toy gestures. In Proceedings of the 10th international Conference on Ubiquitous Computing (Seoul, Korea, September 21 - 24, 2008). UbiComp '08, vol. 344. ACM, New York, NY, 350-359.

Abstract

We present Picture This! a new input device embedded in children's toys for video composition. It consists of a new form of interaction for children's capturing of storytelling with physical artifacts. It functions as a video and storytelling performance system in that children craft videos with and about character toys as the system analyzes their gestures and play patterns. Children's favorite props alternate between characters and cameramen in a film. As they play with the toys to act out a story, they conduct film assembly. We position our work as ubiquitous computing that supports children's tangible interaction with digital materials. During user testing, we observed children ages 4 to 10 playing with Picture This!. We assess to what extent gesture interaction with objects for video editing allows children to explore visual perspectives in storytelling. A new genre of Gesture Object Interfaces as exemplified by Picture This relies on the analysis of gestures coupled with objects to represent bits.

 

Children playing with Picture This!