MARS - Mobile Augmented Reality Systems
Steven Feiner, Tobias Höllerer, Elias Gagas, Drexel Hallaway, Tachio Terauchi, Sinem Güven, and Blair MacIntyre		Columbia University Computer Graphics and User Interfaces Lab

This sketch illustrates how a MARS unit could be used

Objective

In exploring user interfaces, systems software, and application scenarios for MARS, our main focus is on the following lines of research:

• Identifying generic tasks a mobile user would want to carry out using a context-aware computing system
• Defining a comprehensive set of reusable user interface components for mobile augmented reality applications.
• Making combined use of different display technologies, ranging from head-worn, to hand-held, to palm-top to best support mobile users.

Projects

		In our first outdoor MARS, the Touring Machine, our MARS unit acts as a campus information system, assisting a user in finding places and allowing her to pose queries about items of interest, such as buildings and statues.
		The Mobile Journalist's Workstation extends the campus tour application to present additional multimedia information (sound, text, image, video) in the spatial context of the campus. The current prototype was used to present several Situated documentaries to roaming users, including stories about the student revolt on Columbia's Campus in 1968, about the tunnel system underneath Columbia's campus, and about the early history of our campus.
		UIs for Indoor/Outdoor Collaboration allow a roaming outdoor user to be monitored and provided with guidance by remote experts. In exchange, outdoor users can report their observations to the indoor personnel. For this project we developed a distributed infrastructure that allows us to connect diverse user interfaces (wearable, hand-held, stationary desk-top, stationary wall-sized, and stationary immersive AR) to the same repository of campus-related information. A key goal is to explore collaboration in such heterogeneous computing environments.
		The MARS Authoring Tool has a 3D graphical user interface that makes it possible for non-programmers to create and edit situated documentaries. Situated documentaries are 3D hypermedia narratives that are interwoven with a wearable computer user's surrounding environment. (We introduced Situated documentaries in our earlier work on the Mobile Journalist's Workstation, described above, but had previously created them through tedious conventional programming.) Using the MARS Authoring Tool, we have authored several situated documentaries that tell the stories of events that took place on our campus. Our situated documentaries can be experienced outdoors, using our MARS backpack, as well as with indoor desktop, augmented reality, and virtual reality systems.

Implementation Framework

• Hardware

  The main components of our system are a computer (with 3D graphics acceleration), a GPS system (originally differential GPS, and now real-time kinematic GPS+GLONASS), a see-through head-worn display (with orientation tracker), and a wireless network (originally a custom spread spectrum system developed at Columbia, and now IEEE 802.11a/b), all attached to the backpack. The user also holds a small stylus-operated computer that can talk to the backpack computer via the spread spectrum radio channel. Thus we can control the material presented on the headworn display from the handheld screen. We also provide a more direct control mechanism of a cursor in the headworn display by mounting a trackpad on the back of the handheld display where it can easily be manipulated (we inverted the horizontal axis) while holding the display upright.

  While we want our system to be as lightweight and comfortable as possible, we decided to use only off-the-shelf hardware to avoid the expense, effort, and time involved in building our own. Consequently we often settled for items that were far bulkier than we would like them to be, in return for the increased flexibility that they offered. For example, our first system built in 1996, used a FieldWorks laptop machine for the backpack computer, which offered three PCI and three EISA expansion slots (used among others for a powerful 3D graphics adapter and a 6-serial port expansion card). Over the years, we later built our own lighter and faster battery-powered computers with 3D graphics cards, and finally graduated to laptops with 3D graphics processors when they became available in 2001.

• Software

  Our original software infrastructure was Coterie, a prototyping environment that provided language-level support for distributed virtual environments. The main mobile AR application ran on the backpack computer and received continuous input from the GPS system, the orientation head tracker, and the trackpad (mounted on the back of the handheld computer). It generated and displayed at an interactive frame rate the overlaid 3D graphics and user interface components on the headworn display.
  On the handheld computer we ran arbitrary applications that talked to the main backpack application via Coterie/Repo object communications. In our first prototype, we simply ran a custom HTTP server and a web browser on the handheld computer, intercepted all URL requests and link selections, and thus established a two-way communication channel between the backpack and the handheld.
  Our later systems were developed using Java and Java 3D.

Publications

S. Güven, S. Feiner, Authoring 3D Hypermedia for Wearable Augmented and Virtual Reality, Proc. ISWC '03 (Seventh International Symposium on Wearable Computers), White Plains, NY, October 21-23, 2003, 118-226.
( 2MB Acrobat version of paper)

T. Höllerer, S. Feiner, T. Terauchi, G. Rashid, D. Hallaway, Exploring MARS: Developing Indoor and Outdoor User Interfaces to a Mobile Augmented Reality System, Computers and Graphics, 23(6), Elsevier Publishers, Dec. 1999, 779-785

T. Höllerer, S. Feiner, J. Pavlik, Situated Documentaries: Embedding Multimedia Presentations in the Real World, Proc. ISWC '99 (Third Int. Symp. on Wearable Computers), San Francisco, CA, October 18-19, 1999, 79-86
( 3.2MB gzipped Postscript) ( 2MB Acrobat)

Acknowledgments

Any opinions, findings, and conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF or any other organization supporting this work.