Chapter 10: User Interfaces and Visualization - by Marti Hearst
Contents
|
Modern Information Retrieval Chapter 10: User Interfaces and Visualization - by Marti Hearst |
Contents |
The user interface designer must make decisions about how to arrange various kinds of information on the computer screen and how to structure the possible sequences of interactions. This design problem is especially daunting for a complex activity like information access. In this section we discuss design choices surrounding the layout of information within complex information systems, and illustrate the ideas with examples of existing interfaces. We begin with a discussion of very simple search interfaces, those used for string search in `find' operations, and then progress to multiwindow interfaces and sophisticated workspaces. This is followed by a discussion of the integration of scanning, selecting, and querying within information access interfaces and concludes with interface support for retaining the history of the search process.
A common simple search need is that of the `find' operation,
typically run over the contents of a document that is currently being
viewed. Usually this function does not produce ranked output, nor
allow Boolean combinations of terms; the main operation is a simple
string match (without regular expression capabilities). Typically a
special purpose search window is created, containing a few simple
controls (e.g., case-sensitivity, search forward or backward). The
user types the query string into an entry form and string matches are
highlighted in the target document (see Figure ![[*]](cross_ref_motif.gif)
).
The next degree of complexity is the `find' function for searching
across small collections, such as the files on a personal computer's
hard disk, or the history list of a Web browser. This type of
function is also usually implemented as a simple string match. Again,
the controls and parameter settings are shown at the top of a
special purpose search window and the various options are set via
checkboxes and entry forms. The difference from the previous example
is that a results list is shown within the search interface itself
(see Figure ).
A common problem arises even in these very simple interfaces. An ambiguous state occurs in which the results for an earlier search are shown while the user is entering a new query or modifying the previous one. If the user types in new terms and but then does not activate the search, the interface takes on a potentially misleading state, since a user could erroneously assume that the old search hits shown correspond to the newly typed-in query. One solution for this problem is to clear the results list as soon as the user begins to type in a new query.
However, the user may want to refer to terms shown in the search results to help reformulate the query, or may decide not to issue the new query and instead continue with the previous results. These goals would be hampered by erasing the current result set as soon as the new query is typed. Another solution is to bring up a new window for every new query. However, this requires the user to execute an additional command and can lead to a proliferation of windows. A third, probably more workable solution, is to automatically `stack' the queries and results lists in a compact format and allow the user to move back and forth among the stacked up prior searches.
Simple interfaces like these can be augmented with functionality that can greatly aid initial query formulation. Spelling errors are a major cause of void result sets. A spell-checking function that suggests alternatives for query terms that have low frequency in the collection might be useful at this stage. Another option is to suggest thesaurus terms associated with the query terms at the time the query terms are entered. Usually these kinds of information are shown after the query is entered and documents have been retrieved, but an alternative is to provide this information as the user enters the query, in a form of query preview.
For search tasks more complex than the simple string matching find operations described above, the interface designer must decide how to lay out the various choices and information displays within the interface.
As discussed above, traditional bibliographic search systems use TTY-based command-line interfaces or menus. When the system responds to a command, the new results screen obliterates the contents of the one before it, requiring the user to remember the context. For example, the user can usually see only one level of a subject hierarchy at a time, and must leave the subject view in order to see query view or the document view. The main design choices in such a system are in the command or menu structure, and the order of presentation of the available options.
In modern graphical interfaces, the windowing system can be used to divide functionality into different, simultaneously displayed views [myers88]. In information access systems, it is often useful to link the information from one window to the information in another, for example, linking documents to their position in a table of contents, as seen in SuperBook. Users can also use the selection to cut and paste information from one window into another, for example, copy a word from a display of thesaurus terms and paste the word into the query specification form.
When arranging information within windows, the designer must choose between a monolithic display, in which all the windows are laid out in predefined positions and are all simultaneously viewable, tiled windows, and overlapping windows. User studies have been conducted comparing these options when applied to various tasks [shneiderman97][billingsley88]. Usually the results of these studies depend on the domain in which the interface is used, and no clear guidelines have yet emerged for information access interfaces.
The monolithic interface has several advantages. It allows the designer to control the organization of the various options, makes all the information simultaneously viewable, and places the features in familiar positions, making them easier to find. But monolithic interfaces have disadvantages as well. They often work best if occupying the full viewing screen, and the number of views is inherently limited by the amount of room available on the screen (as opposed to overlapping windows which allow display of more information than can fit on the screen at once). Many modern work-intensive applications adopt a monolithic design, but this can hamper the integration of information access with other work processes such as text editing and data analysis. Plaisant et al. [plaisant95] discuss issues relating to coordinating information across different windows to providing overview plus details.
A problem for any information access interface is an inherent limit in how many kinds of information can be shown at once. Information access systems must always reserve room for a text display area, and this must take up a significant proportion of screen space in order for the text to be legible. A tool within a paint program, for example, can be made quite small while nevertheless remaining recognizable and usable. For legibility reasons, it is difficult to compress many of the information displays needed for an information access system (such as lists of thesaurus terms, query specifications, and lists of saved titles) in this manner. Good layout, graphics, and font design can improve the situation; for example, Web search results can look radically different depending on spacing, font, and other small touches [mullet95].
Overlapping windows provide flexibility in arrangement, but can quickly lead to a crowded, disorganized display. Researchers have observed that much user activity is characterized by movement from one set of functionally related windows to another. Bannon et al. [bannon83] define the notion of a workspace -- the grouping together of sets of windows known to be functionally related to some activity or goal -- arguing that this kind of organization more closely matches users' goal structure than individual windows [billingsley88]. Card et al. [card84] also found that window usage could be categorized according to a `working set' model. They looked at the relationship between the demands of the task and the number of windows in use, and found the largest number of individual windows were in use when users transitioned from one task to another.
Based on these and other observations, Henderson and Card [hendersen86] built a system intended to make it easier for users to move between `multiple virtual workspaces' [billingsley88]. The system uses a 3D spatial metaphor, where each workspace is a `room,' and users transition between workspaces by `moving' through virtual doors. By `traveling' from one room to the next, users can change from one work context to another. In each work context, the application programs and data files that are associated with that work context are visible and readily available for reopening and perusal. The workspace notion as developed by Card et al. also emphasizes the importance of having sessions persist across time. The user should be able to leave a room dedicated to some task, work on another task, and three days later return to the first room and see all of the applications still in the same state as before. This notion of bundling applications and data together for each task has since been widely adopted by window manager software in workstation operating system interfaces.
Elastic windows [kandogan97a] is an
extension to the workspace or rooms notion to the organization of 2D tiled
windows. The main idea is to make the transition easier from
one role or task to another, by adjusting how much of the screen real
estate is consumed by the current role. The user can enlarge an entire
group of windows with a simple gesture, and this resizing automatically
causes the rest of the workspaces to reduce in size so they all still fit
on the screen without overlap.
The following sections describe the information layout and management approaches taken by several modern information access interfaces.
The InfoGrid system [rao92] is a typical example of a monolithic
layout for an information access interface. The layout assumes a
large display is available and is divided into a left-hand and
right-hand side (see Figure ). The left-hand
side is further subdivided into an area at the top that contains
structured entry forms for specifying the properties of a query, a
column of iconic controls lining the left side, and an area for
retaining documents of interest along the bottom. The main central
area is used for the viewing of retrieval results, either as thumbnail
representations of the original documents, or derived organizations of
the documents, such as Scatter/Gather-style cluster results. Users
can select documents from this area and store them in the holding area
below or view them in the right-hand side. Most of the right-hand side of
the display is used for viewing selected documents, with the upper
portion showing metadata associated with the selected document. The
area below the document display is intended to show a graphical
history of earlier interactions.
Designers must make decisions about which kinds of information to show in the primary view(s). If InfoGrid were used on a smaller display, either the document viewing area or the retrieval results viewing area would probably have to be shown via a pop-up overlapping window; otherwise the user would have to toggle between the two views. If the system were to suggest terms for relevance feedback, one of the existing views would have to be supplanted with this information or a pop-up window would have to be used to display the candidate terms. The system does not provide detailed information for source selection, although this could be achieved in a very simple way with a pop-up menu of choices from the control panel.
The layout of the InfoGrid is quite similar to that of SuperBook (see
section ). The main difference is that
SuperBook retains the table of contents-like display in the main
left-hand pane, along with indicators of how many documents containing
search hits occur in each level of the outline. Like InfoGrid, the
main pane of the right-hand side is used to display selected documents.! !
Query formulation is done just below the table of contents view (although in earlier versions this appeared in a separate window). Terms related to the user's query are shown in this window as well. Large images appear in pop-up overlapping windows.
The SuperBook layout is the result of several cycles of iterative design [landauer93]. Earlier versions used overlapping windows instead of a monolithic layout, allowing users to sweep out a rectangular area on the screen in order to create a new text box. This new text box had its own set of buttons that allowed users to jump to occurrences of highlighted words in other documents or to the table of contents. SuperBook was redesigned after noting results of experimental studies [hauptmann83][maclean85] showing that users can be more efficient if given fewer, well chosen interaction paths, rather than allowing wide latitude (A recent study of auditory interfaces found that although users were more efficient with a more flexible interface, they nevertheless preferred the more rigid, predictable interface [walker98]). The designers also took careful note of log files of user interactions. Before the redesign, users had to choose to view the overall frequency of a hit, move the mouse to the table of contents window, click the button and wait for the results to be updated. Since this pattern was observed to occur quite frequently, in the next version of the interface, the system was redesigned to automatically perform this sequence of actions immediately after a search was run.
The SuperBook designers also attempted a redesign to allow the interface to fit into smaller displays. The redesign made use of small, overlapping windows. Some of the interaction sequences that were found useful in this more constrained environment were integrated into later designs for large monolithic displays.
The DLITE system [cousins97][cousins97b] makes a number of
interesting design choices. It splits functionality into two parts: control of the search process and display of results . The control portion is a graphical direct manipulation
display with animation (see Figure). Queries, sources,
documents, and groups of retrieved documents are represented as graphical
objects. The user creates a query by filling out the editable fields within
a query constructor object. The system manufactures a query object, which is
represented by a small icon which can be dragged and dropped onto
iconic representations of collections or search services. If a service
is active, it responds by creating an empty results set object and
attaching the query to this. A set of retrieval results is
represented as a circular pool, and documents within the result set
are represented as icons distributed along the perimeter of the pool.
Documents can be dragged out of the results set pool and dropped into
other services, such as a document summarizer or a language
translator. Meanwhile, the user can make a copy of the query icon and
drop it onto another search service. Placing the mouse over the iconic
representation of the query causes a `tool-tips' window to pop up to
show the contents of the underlying query. Queries can be stored and
reused at a later time, thus facilitating retention of previously
successful search strategies.
A flexible interface architecture frees the user from the restriction of a rigid order of commands. On the other hand, as seen in the SuperBook discussion, such an architecture must provide guidelines, to help get the user started, give hints about valid ways to proceed, and prevent the user from making errors. The graphical portion of the DLITE interface makes liberal use of animation to help guide the user. For example, if the user attempts to drop a query in the document summarizer icon -- an illegal operation -- rather than failing and giving the user an accusatory error message [cooper95], the system takes control of the object being dropped, refusing to let it be placed on the representation for the target application, and moves the object left, right, and left again, mimicking a `shake-the-head-no' gesture. Animation is also used to help the user understand the state of the system, for example, in showing the progress of the retrieval of search results by moving the result set object away from the service from which it was invoked.
DLITE uses a separate Web browser window for the display of detailed information about the retrieved documents, such as their bibliographic citations and their full text. The browser window is also used to show Scatter/Gather-style cluster results and to allow users to select documents for relevancefeedback. Earlier designs of the system attempted to incorporate text display into the direct manipulation portion, but this was found to be infeasible because of the space required [cousins97b]. Thus, DLITE separates the control portion of the information access process from the scanning and reading portion. This separation allows for reusable query construction and service selection, while at the same time allowing for a legible view of documents and relationships among retrieved documents. The selection in the display view is linked to the graphical control portion, so a document viewed in the display could be used as part of a query in a query constructor.
DLITE also incorporates the notion of a workspace, or `workcenter,' as it is known in this system. Different workspaces are created for different kinds of tasks. For example, a workspace for buying computer software can be equipped with source icons representing good sources of reviews of computer software, good Web sites to search for price information and link to the user's online credit service.
The guiding principle behind the SketchTrieve interface
[hendry97] is the depiction of information access as an informal
process, in which half-finished ideas and partly explored paths can be
retained for later use, saved and brought back to compare to later
interactions, and the results can be combined via operations on
graphical objects and connectors between them. It has been observed
[nardi93][shipman95] that users use the physical layout of
information within a spreadsheet to organize information. This idea
motivates the design of SketchTrieve, which allows users to arrange
retrieval results in a side-by-side manner to facilitate comparison
and recombination (see Figure ).
The notion of a canvas or workspace for the retention of the previous
context should be adopted more widely in future. Many issues are not
easily solved, such as how to show the results of a set of
interrelated queries, with minor modifications based on query
expansion, relevance feedback, and other forms of modification. One
idea is to show sets of related retrieval results as a stack of cards
within a folder and allow the user to extract subsets of the cards and
view them side by side, as is done in SketchTrieve, or compare them
via a difference operation.
Sometimes conversion from a command-line-based interface to a graphical display can cause problems. Hancock-Beaulieu et al. [hancock-beaulieu95] describe poor design decisions made in an overlapping windows display for a bibliographic system. (An improvement was found with a later redesign of the system that used a monolithic interface [hancock-beaulieu97].) Problems can also occur when designers make a `literal' transformation from a TTY interface to a graphical interface. The consequences can be seen in the current LEXIS-NEXIS interface, which does not make use of the fact that window systems allow the user to view different kinds of information simultaneously. Instead, despite the fact that it occupies the entire screen, the interface does not retain window context when the user switches from one function to another. For example, viewing a small amount of metadata about a list of retrieved titles causes the list of results to disappear, rather than overlaying the information with apop-up window or rearranging the available space with resizable tiles. Furthermore, this metadata is rendered in poorly-formatted ASCII instead of using the bit-map capabilities of a graphical interface. When a user opts to see the full text view of a document, it is shown in a small space, a few paragraphs at a time, instead of expanding to fill the entire available space.
Section discusses information seeking
strategies and behaviors that have been observed by researchers in the
field. This discussion suggests that the user interface should show
what the available choices are at any given point, as well as what
moves have been made in the past, short-term tactics as well as
longer-term strategies, and allow the user to annotate the choices made and
information found along the way. Users should be able to bundle
search sessions as well as save individual portions of a given search
session, and flexibly access and modify each. There is also
increasing interest in incorporating personal preference and usage
information both into formulation of queries and use of the results of
search [freeman95].
For the most part these strategies are not supported well in current user interfaces; however some mechanisms have been introduced that begin to address these needs. In particular, mechanisms to retain prior history of the search are useful for these tasks. Some kind of history mechanism has been made available in most search systems in the past. Usually these consist of a list of the commands executed earlier. More recently, graphical history has been introduced, that allows tracking of commands and results as well. Kim and Hirtle [kim95] present a summary of graphical history presentation mechanisms. Recently, a graphical interface that displays Web page access history in a hierarchical structure was found to require fewer page accesses and require less time when returning to pages already visited [hightower98].
An innovation of particular interest for information access interfaces
is exemplified by the saving of state in miniature form in a `slide
sorter' view as exercised by the VISAGE system for information
visualization [roth97] (see Figure ).
The VISAGE application has the added advantage of being visual in
nature and so individual states are easier to recognize.
Although intended to be used as a presentation creation facility, this
interface should also be useful for retaining search action history.
User interfaces for information access in general do not do a good job of supporting strategies, or even of sequences of movements from one operation to the next. Even something as simple as taking the output of retrieval results from one query and using them as input to another query executed in a later search session is not well supported in most interfaces.
Hertzum and Frokjaer [hertzum96] found that users preferred an
integration of scanning and query specification in their user
interfaces. They did not, however, observe better results with such
interactions. They hypothesized that if interactions are too
unrestricted this can lead to erroneous or wasteful behavior, and
interaction between two different modes requires more guidance. This
suggests that more flexibility is needed, but within constraints (this
argument was also made in the discussion of the SuperBook system in
section ).
There are exceptions. The new Web version of the Melyvl system
provides ways to take the output of one query and modify it later for
re-execution (see Figure ). The
workspace-based systems such as DLITE and Rooms allow storage and
reuse of previous state. However, these systems do not integrate the
general search process well with scanning and selection of information
from auxiliary structures. Scanning, selection, and querying needs to
be better integrated in general. This discussion will conclude with
an example of an interface that does attempt to tightly couple
querying and browsing.
The Cat-a-Cone interface integrates querying and browsing of very large
category hierarchies with their associated text collections. The
prototype system uses 3D+animation interface components from the
Information Visualizer [card96], applied in a novel way, to
support browsing and search of text collections and their category
hierarchies. See Figure .
A key component of the interface is the separation of
the graphical representation of the category hierarchy from the
graphical representation of the documents. This separation allows for
a fluid, flexible interaction between browsing and search, and between
categories and documents. It also provides a mechanism by which a
set of categories associated with a document can be viewed along
with their hierarchical context.
Another key component of the design is assignment of first-class status to the representation of text content. The retrieved documents are stored in a 3D+animation book representation [card96] that allows for compact display of moderate numbers of documents. Associated with each retrieved document is a page of links to the category hierarchy and a page of text showing the document contents. The user can `ruffle' the pages of the book of retrieval results and see corresponding changes in the category hierarchy, which is also represented in 3D+animation. All and only those parts of the category space that reflect the semantics of the retrieved document are shown with the document.
The system allows for several different kinds of starting points.
Users can start by typing in a name of a category and seeing which
parts of the category hierarchy match it. For example,
Figure shows the results of searching on
`Radiation' over the MeSH terms in this subcollection. The word
appears under four main headings (Physical Sciences, Diseases,
Diagnostics, and Biological Sciences). The hierarchy
immediately shows why `Radiation' appears under Diseases -- as
part of a subtree on occupational hazards. Now the user can select
one or more of these category labels as input to a query specification.
Another way the user can start is by simply typing in a free text query into an entry label.This query is matched against the collection. Relevant documents are retrieved and placed in the book format. When the user `opens' the book to a retrieved document, the parts of the category hierarchy that correspond to the retrieved documents are shown in the hierarchical representation. Thus, multiple intersecting categories can be shown simultaneously, in their hierarchical context. Thus, this interface fluidly combines large, complex metadata, starting points, scanning, and querying into one interface. The interface allows for a kind of relevance feedback, by suggesting additional categories that are related to the documents that have been retrieved. This interaction model is similar to that proposed by [agosti92].
Recall the evaluation of the Kohonen feature map representation
discussed in section . The
experimenters found that some users expressed a desire for a visible
hierarchical organization, others wanted an ability to zoom in on a
subarea to get more detail, and some users disliked having to look
through the entire map to find a theme, desiring an alphabetical
ordering instead. The subjects liked the ease of being able to jump
from one area to another without having to back up (as is required in
Yahoo!) and liked the fact that the maps have varying levels of
granularity.
These results all support the design decisions made in the Cat-a-Cone. Hierarchical representation of term meanings is supported, so users can choose which level of description is meaningful to them. Furthermore, different levels of description can be viewed simultaneously, so more familiar concepts can be viewed in more detail, and less familiar at a more general level. An alphabetical ordering of the categories coupled with a regular expression search mechanism allows for straightforward location of category labels. Retrieved documents are represented as first-class objects, so full text is visible, but in a compact form. Category labels are disambiguated by their ancestor/descendant/sibling representation. Users can jump easily from one category to another and can in addition query on multiple categories simultaneously (something that is not a natural feature of the maps). The Cat-a-Cone has several additional advantages as well, such as allowing a document to be placed at the intersection of several categories, and explicitly linking document contents with the category representation.
