Did the Assistant provide effective assistance for the Searcher?
His directions were good, or at least no worse than the user's idea.

Were their interactions smooth and communication between them good?
The expert had a style that favored search engines, while the novice
favored looking at potential expert sites for the answer.  Often, the
"novice" tried his way first, then if it failed he followed the expert's
advice.  Other times (especially at first), the novice had ideas but asked
for buy-in from the expert (e.g. "Now back to altavista?")

>From the Clinton question (novice in all caps):

I HAVE NO IDEA, YOU'LL HAVE TO TELL ME
type clinton
WITH CAPITOL C OR COMAS OR ANYTHING?
no, just "clinton+may+1994"
[only one hit]
ok, let's go back
[GOES TO THE ALTAVISTA HIT ANYWAY]
oh, maybe it'll work anyway... go down
[not it]
WHAT ABOUT GOING TO WHITEHOUSE?
i assume they won't have such information, so we should try like the
press.  try clinton+press+1994.  [try, fail]
WHAT ABOUT GOING TO THE WHITEHOUSE HOMEPAGE?  [TRIES SEARCHING ON
WHITEHOUSE]
try president+usa?

Missunderstandings:
There weren't any missunderstandings in the transcript, though there were
times when the novice had to ask for clarification.  E.g.:

Expert:	type clinton
Novice:	WITH CAPITOL C OR COMAS OR ANYTHING?

What could either the Searcher or the assistant have done to facilitate
working together?

It would have been much more efficient if the expert could have had his own
browser, and was able to say "hey, look at what's on my screen" whenever
his search bore fruit.  With only one computer, the expert is shooting in
the dark, and doesn't know if something will work till it's tried.

Having a seperate screens for both expert and novice, and allowing both
expert and novice to see both screens at once is important.  It allows them
to compare and look for differences.  Quick state-copies (with undo), would
also be useful.

A comment on the expriment as a whole:

In the hunt problem given the user in this experiment, there are two
potential goals for the expert:
1.  Help the user with their explicit goal.
2.  Teach the user learn to use the web to better satisfy goals in the
future.

Helping a user with their task-at-hand will always be a valid goal for an
agent, but teaching the user to better search is only useful in limited
situations.  For example, if a user will always have the agent available
when performing web-searches, why shouldn't they always go to the agent,
just as they would go to any other meta-search engine?  I can see only
three reasons one might want a teaching agent instead of a "showing agent:"

1.  The agent won't always be around to help the user.  This was the case
with the expert help in the wizard-of-oz experiment.  This is also the case
for helper-agents in simulations (e.g. training software), where there
won't be an agent in the outside world.
2.  Using the agent's help is actually slower than doing the work yourself,
once you know how.  This is the case with Emacs-19 and MS-Word's helper
system that points out shortcuts to long commands.
3.  The agent is only useful in a limited domain, but what it can teach is
more generally useful.  I can't think of any situation where this would be
true though.
4.  Where education is the ultimate goal.  For example, a chess tutoring
program is valuable even if the user could always just have the agent play
for him, because the education itself has intrinsic value.
5.  Where understanding the agent's underlying mechanism will help use the
agent more effectively.  E.g. a tip-of-the-day based on your current task
to help make the agent work better.

Did the Assistant provide effective assistance for the Searcher?
His directions were good, or at least no worse than the user's idea.

Were their interactions smooth and communication between them good?
The expert had a style that favored search engines, while the novice
favored looking at potential expert sites for the answer.  Often, the
"novice" tried his way first, then if it failed he followed the expert's
advice.  Other times (especially at first), the novice had ideas but asked
for buy-in from the expert (e.g. "Now back to altavista?")

>From the Clinton question (novice in all caps):

I HAVE NO IDEA, YOU'LL HAVE TO TELL ME
type clinton
WITH CAPITOL C OR COMAS OR ANYTHING?
no, just "clinton+may+1994"
[only one hit]
ok, let's go back
[GOES TO THE ALTAVISTA HIT ANYWAY]
oh, maybe it'll work anyway... go down
[not it]
WHAT ABOUT GOING TO WHITEHOUSE?
i assume they won't have such information, so we should try like the
press.  try clinton+press+1994.  [try, fail]
WHAT ABOUT GOING TO THE WHITEHOUSE HOMEPAGE?  [TRIES SEARCHING ON
WHITEHOUSE]
try president+usa?

Missunderstandings:
There weren't any missunderstandings in the transcript, though there were
times when the novice had to ask for clarification.  E.g.:

Expert:	type clinton
Novice:	WITH CAPITOL C OR COMAS OR ANYTHING?

What could either the Searcher or the assistant have done to facilitate
working together?

It would have been much more efficient if the expert could have had his own
browser, and was able to say "hey, look at what's on my screen" whenever
his search bore fruit.  With only one computer, the expert is shooting in
the dark, and doesn't know if something will work till it's tried.

Having a seperate screens for both expert and novice, and allowing both
expert and novice to see both screens at once is important.  It allows them
to compare and look for differences.  Quick state-copies (with undo), would
also be useful.

A comment on the expriment as a whole:

In the hunt problem given the user in this experiment, there are two
potential goals for the expert:
1.  Help the user with their explicit goal.
2.  Teach the user learn to use the web to better satisfy goals in the
future.

Helping a user with their task-at-hand will always be a valid goal for an
agent, but teaching the user to better search is only useful in limited
situations.  For example, if a user will always have the agent available
when performing web-searches, why shouldn't they always go to the agent,
just as they would go to any other meta-search engine?  I can see only
three reasons one might want a teaching agent instead of a "showing agent:"

1.  The agent won't always be around to help the user.  This was the case
with the expert help in the wizard-of-oz experiment.  This is also the case
for helper-agents in simulations (e.g. training software), where there
won't be an agent in the outside world.
2.  Using the agent's help is actually slower than doing the work yourself,
once you know how.  This is the case with Emacs-19 and MS-Word's helper
system that points out shortcuts to long commands.
3.  The agent is only useful in a limited domain, but what it can teach is
more generally useful.  I can't think of any situation where this would be
true though.
4.  Where education is the ultimate goal.  For example, a chess tutoring
program is valuable even if the user could always just have the agent play
for him, because the education itself has intrinsic value.
5.  Where understanding the agent's underlying mechanism will help use the
agent more effectively.  E.g. a tip-of-the-day based on your current task
to help make the agent work better.