Informedia Goes to School

Introduction

The Informedia Digital Video Library Project at Carnegie Mellon University studies how multimedia digital libraries can be established and utilized efficiently. Based on feedback from a series of prototype demonstrations, focus groups, and teacher workshops, an initial IDVL system was fielded at a K-12 school here in Pittsburgh in the Spring of 1996. We will use the acronym "IDVL" (for Informedia Digital Video Library) to describe both the contents of the video library and the interface to that data.

Three high school science teachers utilized the IDVL system in their Biology and Physics classes, where students made use of six client stations in a science lab to access the video library. In addition, two client stations were placed in the school library for general use. The system subsequently was used by students ages 7 to 17 enrolled in various summer school programs. This paper will report on the lessons learned from these first experiences with the IDVL, describe how that feedback has been used to design the next iteration, and conclude with future plans as the new IDVL system is delivered this month.

The Informedia Project focuses on the video medium. The project began with the premise that full content retrieval in a video library can be a difficult and time-consuming process, as evidenced by anyone trying to find a clip of interest in a videotape, or worse from a stack of videotapes. In order to improve access to relevant information, the initial IDVL logically partitions video into small-sized segments and provides alternate representations and abstractions of video content. In the tradeoff between recall and precision, we chose to err on the side of recall and provide tools to users supporting their focusing strategies. We return a set of video segments for a query, and provide browsing mechanisms and content abstractions so that the set can be quickly reviewed by a user to find the segments of interest.

The Initial IDVL System

The initial IDVL contained 1600 video segments spanning 45 hours of video from QED Communications in Pittsburgh, the U.K.'s Open University, and other sources. The video material concentrated in the areas of biology, math, and physics. The interface to this library was a Microsoft Windows application with query and browsing capabilities, an early prototype of which was described in a September 1995 D-Lib article. Users included teachers and students in high school biology and physics classes, and students in summer school programs emphasizing science topics. Primary tasks were fact-finding and the creation of multimedia essays [56 KB] using video drawn from the library.

Transaction logs were captured automatically, recording the system's usage by teachers and students. IDVL users were also invited to record their comments concerning the system. A few informal interviews were conducted. Some students were given a subset of QUIS and other on-line questionnaires similar to those employed by the Alexandria Project as part of a more formal evaluation of a portion of the interface.

Findings

Table 1 details the changes made to the IDVL as a result of the initial user feedback. The presentation style mirrors that used in the D-Lib article by Van House, Butler, Ogle, and Schiff, who stress the importance of iterative, user-centered design. We also adhere to and recognize the importance of that design philosophy. The changes in the IDVL shown in Table 1 were derived from user feedback, will need to be validated with user testing, and most likely will be refined further in the future. The remainder of this section will highlight a few findings and discuss the user feedback and any corrective action taken for each finding.

Automatic transaction logging is a useful feedback tool

Often students are unwilling or unable to be interviewed because of time constraints and personality traits. However, a log of their actions with the IDVL provides a rich data set for analysis leading to IDVL improvements. For example, queries in the transaction logs which produced no results were analyzed. With the addition of better word stemming and a spell checker, over half of those queries would have became meaningful and would have returned results, and so these features were added to the new IDVL.

The transaction log helps identify how users spend their time with the IDVL. Timing every transaction, including all mouse clicks and media control interface commands, rather than only a subset of user actions, was added to the new IDVL. Future analyses can now determine the percentage of time users spend watching videos, browsing for them, formulating queries, and so on. Ideally such complete transaction data can then be used in support of further research into digital library activities. For example, Paepcke identifies five broad categories of activities in digital libraries: discover, retrieve, interpret, manage, and share. Better logs can be used to support research into how time is spent on these tasks in an educational setting with a video library.

Presentation choices will affect system usage

The manner in which search results are presented definitely influences which result items are chosen by the user. The first users of the IDVL had the results presented to them in a pyramid layout [32 KB]. The access pattern [22 KB] reveals that the top result was accessed twice as much as those on the second row of the pyramid, while the second row items were accessed twice as much as those on the third pyramid row. Obviously the pyramid layout biased which results were selected for subsequent viewing. Given that accessing video can be expensive in terms of viewing time and perhaps data transfer, the user should be encouraged to browse through a set of results to better select an appropriate video segment, rather than being enticed into selecting the top of a pyramid layout. The new IDVL will present results in a grid of three columns, with better search results browsing tools so that the user can quickly see which words in a query phrase correspond to which search result items.

Users accessed the most visible features of the system (word query, playing a video) much more frequently than advanced features (focusing on words in a query, browser interface, filmstrips). To encourage use of other features believed to be beneficial, on-screen instructions have been added and the new built-in help system describes these features more completely. To test whether such features actually are of benefit in fact-finding and essay creation tasks, small structured interface evaluation studies have been conducted this summer focusing on a few of these features; the data is currently being analyzed.

During interviews with users they frequently asked for the ability to do conjunctive searches ("and" all of the query words together), rather than the default disjunctive "or" processing. Like most Web search engines, the IDVL allowed conjunctive search but only by making a choice in a "Search Options" dialog which was displayed only after a menu action. Because the option was hidden from view, it was not accessed. The new IDVL has put this option on the displayed query form.

Reduce the cognitive load on the user

Users left the system in its default setting (for color, maximum results to return, style of results presentation, and numerous other interface settings) in over 90% of the sessions. The wealth of options useful for Informedia project demonstrators, e.g., data library locations, dialog synchronization features, and speech recognition settings, overwhelmed users in the school. Even beneficial options were hence ignored. The options were simplified and organized so that only a small but relevant subset is accessible to the school users. Also, context-sensitive help was extended to better describe available options to the users.

In browsing the search results, users could move the mouse over a search results item and its one line abstract would be displayed in a pop-up text window. However, this worked only if the search results window was the active window which received the mouse events, and it took a mouse click by the user to activate the search results window. Many users were frustrated by forgetting to click the search results window in order to activate it. As a result, the extra click is not necessary in the new IDVL and the search results window becomes the active window automatically when the mouse moves into its area. Similarly, other extra mouse or key clicks are no longer needed in the new IDVL.

One of the primary tasks for the users in the school was embedding relevant clips from the IDVL into their own multimedia essays and presentations. The "embed" operation involved the use of a separate dialog box to specify the start and end times for the video to be copied by reference. (The attribution for the video segment, such as producer and title, was included in the reference to satisfy one of the first requirements we received from the school librarian.) Having the dialog box and the video playback window be separate windows meant that they could overlap with one another, one could be hidden from view, and so on. The new IDVL fixes this obvious deficiency and puts all controls which act on the video, such as selecting a portion of the video for copying, into the video window.

Plans for Future IDVLs

The teachers are excited by the potential of using the Informedia system in their classrooms. They and the students would like to see more video content in the library, which will more than double in size with the new IDVL. Many teachers in other subjects expressed an interest to incorporate the library in their curricula if the domain would be extended beyond science topics, and so the project is pursuing copyright issues with news corporations and other content providers.

We are currently moving towards a WWW Informedia client interface, utilizing commonly available technology to allow access over the Internet. The inclusion of public domain video in the new IDVL makes it easier to share this data with others.

We will continue to work closely with teachers and students to iteratively refine the IDVL based on their needs. Initially the video itself was compelling enough to motivate both teachers and students to use the library. We may find the need to provide more scaffolding for educational use of the library in the future. For example, one teacher in particular had great success in motivating the use of the initial IDVL for the creation of multimedia essays on student-selected topics. Other teachers may need help to replicate such experiences, perhaps in the form of workshops, Web pages, and other documentation outlining and supporting the practice of inquiry-based learning via digital resources. The UMDL Teaching and Learning Project is especially relevant to such an effort.

We realize that through better communication with the teachers and students we can improve upon the iterative, user-centered design process used to refine the IDVL. Through incorporation of others' work in the area, including work discussed at the Allerton Institute and UCLA Social Aspects of Digital Libraries Workshop, we can better track the needs and activities of our DL users. Following our first full iteration with the teachers and students, we believe the new IDVL will be used more and with greater satisfaction and benefit than was the initial system, and that this new IDVL is a step closer toward the goal of educationally beneficial digital video libraries.

Acknowledgements

This project is funded by the NSF/NASA/ARPA Digital Libraries Initiative. A complete list of sponsors is available at http://www.informedia. cs.cmu.edu/sponsors.ht ml.

Table 1. Changes to the Informedia Digital Video Library (IDVL)

FEATURE	INITIAL IDVL	FALL '96 IDVL
Library Size	45 hours	100 hours
Platform	Windows 3.1	Windows 95
Query Form	example with misspelling[4 KB]	example with misspelling[3 KB]
Spell Checking	none	potential misspellings flagged quickly for user; suggestions/corrections take place at user request
Results Display	example [33 KB]	example [46 KB]
Browsing Results Display	simple tools	extended tools with better support (less clicking; on-screen instructions)
Video Embedding Tool	example [34 KB]	example [35 KB]
Options	many tailorable features	reduced feature set for school users
Transaction Logs	simple	extended

Copyright © 1996 Michael G. Christel, Krishna Pendyala

hdl://cnri.dlib/september96-christel