These two MPEG movies are two of many possible playouts of the production. One presents the story from John's subjective point of view, and the other presents Kathy's point of view. You will notice differences in shot composition, close-up placement, backgrounds, ambient sounds, and acoustics. The system is also capable of blending these two "extremes" to produce versions that mix the two story perspectives to different extents. In the actual piece, the viewer has real-time control over the subjective story perspective and the camera position.
The Isis object-based media prototyping environment was used to script the entire presentation. Several problems needed to be addressed: How could we restrict the spatial movement of the viewer? How would the presentation change to present different subjective points of view? How could we manage the master shot and close-ups in a manner that could change for different perspectives or display conditions? The approach taken was to create a small set of high-level presentation variables that would affect playout in different ways, each of which could be represented by a single number. Some or all of these variables then would be placed under the control of the viewer:
Cubic interpolation was used to synthesize camera angles between these key points. These parameters were used to pre-render and store several views of the three-dimensional model of the attic which are recalled and used as backgrounds before the characters are composited into the frame. Since the actors were shot from a limited number of angles, the resulting composited output might look slightly odd if the virtual camera is far from any real captured angle.
As long as the camera stays within a certain ``distance'' of one of the real angles, this effect can usually go unnoticed. The script determines which actor view is the most appropriate for the current camera pose.
The other three system variables are used to create a three-dimensional story space to determine exactly what should be showing on the screen for a given scene time, story perspective, and closeupivity setting. Nested timelines greatly simplify the creation of this space inside of the Isis interpreter. At every point in the space, indication is given of whether or not a close-up should be shown, and if it should, a frame number of a particular close-up is specified along with a background to use behind it. Other aspects of the video could be controlled also, such as brightness or color tone, but the playback system does not currently support modifications to these attributes.
For example, at one particular point near the beginning of the scene, the viewer may see the master shot or a close-up of Kathy typing, depending on the size of the output window.
More interestingly, near the middle of the scene, you might see John grabbing the rag off of the dress form in a dark gray corner of the room, or you might see the same action superimposed over a bright cloud background, or you might see John grabbing a handkerchief from the hand of a third mysterious character, all depending on the current setting of the story perspective variable.
The virtual acoustics of the attic are smoothly interpolated between two extremes, as are the volumes of the ambient sounds in the attic. A small room with no echo or reverberation is at one extreme (John's perspective), while a much larger and softer room with a lot of reverberation is at the other extreme (Kathy's perspective).