Sun Microsystems Labs
Mountain View, CA, USA
Massachusetts Institute of Technology
Cambridge, MA, USA
Boston, MA, USA
A good user interface brings you face-to-face
with whatever is being manipulated and experienced. For example,
the steering in a sports car, especially with a mid-engine layout,
lets you feel the road surface so you can tell when the road is
slippery, just by the feeling of the steering. A good tool, like
a high-quality wrench becomes a part of your hand, so you feel
that you are turning the bolt yourself. The machinery disappears,
and you feel connected to the object in question. A programming
environment can also convey an experience of immediacy, drawing
the programmer in closer to his program. When that happens, debugging
becomes easier. This principle of immediacy can serve as a guide,
keeping programming environment builders on the path to productive
Like your favorite pizza, a programming environment
can be described as a layered yet synergistic stack of ingredients:
The principle of immediacy can be brought
to bear on each of these layers.
For example, in the user interface, it will
help if each chunk of information (e.g. each procedure) shows
up as a visually distinguished and clearly legible unit. Here,
at the low level of visual perception, designers frequently employ
cues that mimic the real visual world in order to exploit a user's
preexisting perceptual abilities. That is why, for example, the
Self user interface  renders each object as a slab with stereotypical
stage lighting, from above and left. Careful attention to perceptual
legibility can lighten the burden on the programmer, so that more
attention can be focused on finding bugs and less attention on
finding objects on the screen. The user interface must allow programmers
to manipulate programs as well as see them. Here again, immediacy
can be achieved by employing affordances that match user's prior
experiences, or that reduce the need for the user to change focus.
A bad user interface draws so much attention to itself that it
distracts from debugging, and a good one lets a tired programmer
debug much later into the night (if need be!).
But the UI is only the lowest-level of the
system. The programming environment provides the lenses and levers
that let the programmer inspect, manipulate, and debug the program.
But if the lenses and levers feel like lenses and levers, the
programmer is distanced from the program, has to mentally work
harder to keep track of it, and is encumbered in attempts to fix
The principle of immediacy leads us to understand
that the environment must make it effortless to examine a program,
with all of its connections. If the programming environment you
use cannot rapidly and effortlessly show you all the callers of
a function, or all the definitions of a message, or all the places
that can change a particular variable, it is distancing you from
the logical structure of your program, and making it that much
harder for you to debug it.
And when debugging, what goes for a static
program also goes for a (running or suspended) process. When you
find an argument that has an erroneous value, the environment
should be able to take you to where the value was ultimately passed
in, even if it is ten stack frames up (although I know of none
that can do this today). Immediacy says that the important relationships
must be made manifest.
Immediacy and debugging
We'll illustrate how the principle of immediacy applies to debugging environments by describing ZStep 95 [3, 4], a program debugging environment based on the principle of immediacy. Debugging is often a search that starts from observed effects of running a program and proceeds to deduce the possible causes in the underlying program code. So ZStep 95 strives to facilitate debugging by bringing the relationship between cause and effect closer to the programmer.
The separation between cause and effect can
occur in at least three dimensions: time, space and semantics,
thus designers can strive for three kinds of immediacy that are
important for debugging: temporal immediacy, spatial immediacy
and semantic immediacy.
Human beings recognize causality without concious
effort only when the time between causally related events is
kept to a minimum. If there is a delay between a change in a
steering wheel and the response of a car, we describe the steering
as "mushy" and it destroys the feeling of immediacy
of control of the car. In programming, delay between an effect
and observing related events or data in the program puts a strain
on the programmer's short term memory to hold all the relevant
information in their head while waiting for the programming environment
to "catch up". In Eisenstadt's paper in this issue
, we see that a full 15% of bugs are directly attributable
to temporal distance between cause and effect, the largest single
source of error in his survey.
In debugging, it is important to reason backwards
from effects to causes, so temporal immediacy backward in time
is just as important, if not more important, than temporal immediacy
in the forward direction. ZStep 95 is built upon the notion of
reversibility. It keeps a history of the computation and
can be run either forward or backward using "video recorder"
controls. The history includes expressions, values, and the graphic
output of the program. Important also is fine control over the
level of detail displayed. It is not only immediacy between temporally
adjacent events that is important, but immediacy between temporally
relevant events. We provide both "fast" and "slow"
stepping speeds in both directions, manual and automatic control.
By allowing the programmer to choose speed and direction, ZStep
makes it easier for him to find a mode where the temporal distance
between relevant causal connectionns occurs on the appropriate
scale for preconcious perception.
We also provide a novel "Graphic Step
Forward" and "Graphic Step Backward" that works
in terms of events where something is drawn on the screen, rather
than in terms of expressions in the code. This allows you to step
the behavior of the program rather than step the code, and supports
temporal immediacy between graphic events and their causes in
Whether you choose to step expression-by-expression,
graphic event by graphic event, forward or backward, all views
of the program -- code, value, stack, and graphics are kept in
sync and present a consistent view of the program execution. Self
also stives to keep the information presented to the user consistent
with the dynamic state of the system.
The principal of immediacy explains why these
and other designers made this choice: by providing automatic display
updates, the link between the system's state and the display state
is removed from the programmer's concious attention and becomes
immediate and unconcious; the programmer unconciously comes to
identify one with the other, reducing the cognive burden of the
Spatial immediacy means that the physical
distance between causally related events should be kept to a minimum.
The reason is the same as for temporal immediacy -- events that
are widely separated by space on the screen force the user to
devote more concious effort to link them, forcing the user to
shift attention and putting a strain on he user's short-term memory.
Many steppers show the value of the current expression in a fixed
window off to the side of the window containing the program code
display. This causes a "ping-pong" effect as the user's
attention bounces from the code to the values and back again.
And the existence of a separate, constant area that seqentially
reflects the values of different expressions adds cognitive distance
as well as spatial by forcing the user to deal with the changing
link between the value display region and the expression being
ZStep 95 takes the approach of showing a floating
window that follows the point of program execution. The code is
displayed in an editor buffer, and as the program runs, either
forward or backward, fast or slow, the expression currently being
evaluated is highlighted, and, right next to it, a floating
window shows the code's value, if it has been evaluated.
The user's attention never need wander to
see the value of the current expression. The user need never devote
precious short-term memory to the task of remembering which expression's
value is being shown. If the user is working backwards in the
history, the notation "Will be:" along with the value
is displayed, to indicate that the value lies further along in
the history. "Working on it..." is shown if the value
is only partially computed, so that the event is situated in the
temporal event stream.
Spatial immediacy is important because it
maintains the visual context. Seeing two related pieces of information
next to each other fosters making semantic connections in the
mind between those two pieces of information. If they are spatially
separated, "out of sight" becomes "out of mind".
In ZStep 95, a click on the floating window containing the value
brings up a data inspector. The code, the resulting value, and
an inspector on that value are all spatially co-located, reinforcing
the feeling of immediacy.
If an error occurs, the error message is shown
in the floating window, again, spatially located near where the
error is manifest, so both the cause and effect can be seen at
once. Another click on the error message brings up documentation
about the error, and another click still evokes documentation
for related functions.
Semantic immediacy means that information
that the conceptual distance between pieces of information that
are semantically related is kept to a minimum. In interactive
interfaces, the conceptual distance between two pieces of information
is often represented by how many user interface operations, such
as mouse clicks, it takes you to get from one to the other. We
have already seen some examples of semantic immediacy, between
code expressions and their values, and between error messages
and documentation. There are others.
Because ZStep 95 keeps a history of all the
graphic events in the program, we can associate each graphic object
drawn on the screen with the event that drew it. This makes it
possible to click on a graphic object, and go immediately to the
place in the execution where that object was drawn. Not only do
we see the source code that drew the object, but the entire state
of the stepper is backed up to the point in time where that object
was drawn, including the appearance of the screen at the time,
the values of expressions, the stack. Similarly, we can point
to an expression and jump the stepper immediately to that point
in time, keeping all views consistent, including the graphics.
To summarize, the process of debugging a program
involves understanding the relationships between the following
An expression in the source code [not "lines of code"!]
The value of that expression at a given point in time
The graphic output, if any, of that expression
Expressions that are executed before or after that expression
Other expressions or values that are semantically related
[for example, the stack, inspecting data
ZStep's interface is designed to allow you
to go from any one of these objects to any one of the others with
practially no concious effort. It never takes more than one mouse
click. That's semantic immediacy.
Immediacy and programming language design
An environment creates the experience of immediacy
by allowing the programmer to make any change at any time, and
by taking less than half a second to do so, which keeps response
time below the level of perceivable delay. That way, the program
feels right in your hands. I once attended a panel session representing
three great programming environments at PARC: InterLisp, Smalltalk,
and Cedar. The Cedar panelist boasted that, since multitasking
had been added to Cedar, he now had the luxury of reading his
e-mail while waiting for compiles. But the Smalltalk user had
the greater luxury of having any change take effect in the time
it took to press the accept button and see the resulting screen
flash. He or she could then go on and continue debugging without
loss of continuity. Just as immediacy applies to both stimulus
and response sides of the UI, it also applies-at a higher level
of abstraction-to both browsing and changing code and data in
the programming environment.
Like Smalltalk, Self, APL, and most Lisps
let the programmer breakpoint a program, and insert extra code
to print information, or to check conditions, or even to attempt
a bug fix-while the program is suspended. The program can be continued
without restarting, the change takes effect immediately, runs
at full speed, can be browsed the same way as any other part of
the program. Few other environments-especially those in wide use-can
boast this level of manipulative immediacy. Many C++ systems take
minutes to change class definitions in a large program, forcing
the programmer to waste mental effort devising workarounds, or
leading to either boredom (leaving the flow state) or distraction
(reading e-mail, etc.). Immediacy at the programming-environment
manipulation level is crucial for debugging productivity, and
underlies many of the good results reported on in the rest of
Finally, programming language design is often
viewed as high art, mystical religion, or as putting together
a menu for a restaurant (this is the principle of orthogonality--
the diner can choose sauce independently of meat to give everyone
something that they like). One can also appraise the design of
a language by returning to the principle of immediacy. In order
for a language to bring programs closer to the user, it must be
built around a small number of concepts. For example, APL embodies
linear algebra with its arrays, functions, and functions-on-functions
(a.k.a operators). As a result, to an engineer already familiar
with linear algebra, there is hardly any learning curve at all!
Just as you can write any matrix equation on a piece of paper,
you can write almost any APL expression, unlike many languages
that impose extra constraints. For example, APL though typesafe,
has no static type checking. Although the absence of static checking
requires the programmer to find and fix some bugs at runtime (the
traditional rationale for static typing), its presence compels
the programmer to reason much more indirectly about his program.
Like a scratch on a skipping CD (or LP), static semantics such
as type checking ram home the disturbing fact that the program
does not execute as written; there is a translation and checking
stage, which "executes" the program in its own way,
so that what finally runs is less directly related to what the
programmer can write or see.
As another example, macros are a popular variety
of static translation on programs. They often help the programmer
make up for a lack in the language design itself. But what happens
when there is a bug in the macro? The programmer may wind up staring
at "preprocessor output" in order to understand some
particularly abstruse and unexpected behavior of the program.
In effect the poor programmer has to debug two programs, the one
he wrote, and the one that eventually ran. The principle of immediacy
dictates that the programmer need not worry about static invariants
unless he or she wants to, but many existing languages
force doing this all the time.
By bringing the programmer closer to the program,
an experience of immediacy helps the programmer understand, change,
and ultimately debug. Attaining a reasonable level of immediacy
is a precondition for effective debugging and can serve as a useful
guide to the design of all aspects of programming environments.
 Bay-Wei Chang, Objective reality for self:
concreteness and animation in the Seity user interface. Ph.D.
Dissertation, Electrical Engineering Dept., Stanford University,
 Marc Eisenstadt, "My Hairiest Bug
War Stories", Communcations of the ACM, April 1997.
 Henry Lieberman and Christopher Fry, Bridging
the Gap Between Code and Behavior in Programming, ACM Conference
on Computers and Human Interface, CHI '95, Denver, Colorado, April
 Henry Lieberman and Christopher Fry, ZStep
95, A Reversible, Animated Source Code Stepper, in Software
Visualization: Programming as a Multimedia Experience, John
Stasko, John Domingue, Marc Brown, and Blaine Price, eds., MIT
Press, Cambridge, MA, 1997.
is a Senior Staff Scientist at Sun Microsystems Laboratories.
Author's present address: Sun Microsystems Laboratories, 2550
Garcia Ave, MTV29-117, Mountain View, CA 94043; e-mail: David.Ungar@sun.com
is a Research Scientist at the Media Laboratory of the Massachusetts
Institute of Technology. His interests lie at the intersection
of artificial intelligence and human interface concerns, and among
his topics of interest are intelligent agents for the Web, software
visualization, visual programming and debugging, machine learning,
and programming by example. Author's present address: Media Laboratory,
MIT, 20 Ames St., Room 305 A, Cambridge, MA 02139 USA. E-Mail:
Christopher Fry is the Chief Technical Officer of PowerScout
Corporation in Boston. His current primary interest is the visualization
of complex data structures as diverse as the World Wide Web or
intermediate values within a program development environment.
Over the past couple of decades, Fry has hacked at MIT and numerous
MIT spin-offs. Author's Present Address: PowerScout Corp., 21
Crescent Rd., Lexington, MA 02173 USA. Email: email@example.com