Situated Documentaries

The Computer Graphics and User Interfaces Laboratory

Tobias Höllerer, Steven Feiner, and John Pavlik

Columbia University
Computer Graphics & User Interfaces Lab

Introduction

Our experimental wearable augmented reality system enables users to experience hypermedia presentations that are integrated with the actual outdoor locations to which they are are relevant. The mobile prototype uses a tracked see-through head-worn display to overlay 3D graphics, imagery, and sound on top of the real world, and presents additional, coordinated material on a hand-held pen computer. We have used these facilities to create several situated documentaries that tell the stories of events that took place on our campus.

Example movies

The following four movie sequences demonstrate our system. They show how an outdoor user experiences hyperlinked multimedia presentations. Interlinked virtual flags serve as the main icons, denoting story nodes. They can be selected in various ways and explored in detail to learn about the (in this case historic and architectural) information associated with them. The video material was captured by a camera pointing through the system's AR-glasses.

5.18 MB MPEG-1 movie 7.25 MB MPEG-1 movie 6.2 MB MPEG-1 movie 10.44 MB MPEG-1 movie
User looking around Columbia's campus, visually selecting different flags denoting story elements. User is selecting one of the stories associated with a "1968 student revolt" flag. Multimedia story includes audio narrative, images, and video material, shown on visor and hand-held displays. User is selecting a flag with information on the Columbia tunnel system. A texture mapped hemisphere surrounds the user, providing an omnidirectional view of a Columbia tunnel. User is selecting a flag on the Bloomingdale Asylum complex. By means of a "time slider" applet provided on the handheld computer the user can get a real life impression of the different buildings that once covered the space of today's Low Library.

Example Pictures

Each thumbnail image is a link to the originally captured (mostly VGA resolution) image in JPG format. Note that the gamma values for the images are not adjusted. The images may need to be processed (gamma, contrast) before they can be used. Please contact Prof. Steven Feiner if you are interested in obtaining permission to use the images.

© Computer Graphics and User Interfaces Lab, Columbia University


Situated documentaries. Clockwise: a) Our backpack-based testbed, with tracked see-through head-worn display and pen-based hand-held computer. b) An image photographed by a video camera that wears our testbed's see-through head-worn display. The labels and virtual flags are part of the user interface. c) Related information displayed on our hand-held computer.
Virtual flags denoting points of interest, photographed from the top of a campus building.
This and the next image show two different menu designs for listing multimedia snippets about the 1968 Columbia student revolt. a) World-stabilized circular menu around Low Library (photographed through an earlier, low-resolution, see-through, head-worn display).
Menu design for listing multimedia snippets: b) Head-stabilized list with anchor to its flag (screen dump of the system running in indoor test mode, with an omnidirectional image as a backdrop).
Imagery documenting the student revolt in 1968: a) Still image, overlaid on top of Low Library
Imagery documenting the student revolt in 1968: b) video material displayed on the hand-held computer
Exploring Columbia's tunnel system: a) Schematic view of how a user experiences an omnidirectional camera image.
Exploring Columbia's tunnel system: b) The omnidirectional camera image seen from a user's perspective.
Information on historical architecture: a) A simplified 3D model of the main Bloomingdale asylum building overlaid at its original position on Columbia's campus
Information on historical architecture: b) Documentary material displayed on the hand-held computer

Publications

T. Höllerer, S. Feiner, J. Pavlik, Situated Documentaries: Embedding Multimedia Presentations in the Real World, In: Proc. ISWC '99 (Third Int. Symp. on Wearable Computers), San Francisco, CA, October 18-19, 1999, pp. 79-86
( 2MB Acrobat version of paper)( 3.1MB gzipped Postscript version of paper)

Acknowledgments

This research is supported in part by Office of Naval Research Grants N00014-97-1-0838, N00014-00-1-0249, and N00014-99-1-0394; NSF Grant EIA-97-29844; and hardware and software gifts from IBM, Intel, Mitsubishi, and Microsoft.

Any opinions, findings, and conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF or any other organization supporting this work.
Back to the Mobile Augmented Reality page.

Please send comments to Tobias Höllerer at <htobias@cs.columbia.edu>