Goblin XNA: A Platform for 3D AR and VR Research and Education Ohan Oda Steven Feiner

Columbia University Computer Graphics & User Interfaces Lab

Introduction

Goblin XNA is a platform for research on 3D user interfaces, including mobile augmented reality and virtual reality, with an emphasis on games. It is written in C# and based on the Microsoft XNA platform. Goblin XNA inherits some of the goals of the earlier Goblin project, but with a shift in emphasis toward core 3D user interface functionality, leveraging the existing functionality of DirectX 3D game engines and development environments. The platform currently supports 6DOF (six degree of freedom) position and orientation tracking using marker-based camera tracking through ARTag with OpenCV or DirectShow, and InterSense hybrid trackers. Physics is supported through BulletX and Newton Game Dynamics. Networking is supported through Lidgren library. Goblin XNA also includes a 2D GUI system to allow the creation of classical 2D interaction components.

Goblin XNA AR Racing Game

The Goblin XNA AR Racing Game is an augmented reality mod of the XNA Racing Game Starter Kit. It is the first game being created using Goblin XNA, and demonstrates the use of fiducial tracking, 2.5D and 3D widgets, and assorted 3D user interface techniques. The Goblin XNA AR Racing Game uses an optically tracked physical gameboard, controller, and tokens.  Instead of racing around a preset virtual track, the driver must pass a sequence of waypoints, while avoiding obstacles. Waypoints and obstacles are attached to physical tokens that can be moved dynamically during gameplay to rearrange the course.

		The user drives a virtual car with a hand-held controller on a tracked physical gameboard. Developers: Ohan Oda, Levi Lister, and Wei Teng
		The car traverses a course containing waypoints and obstacles. Developers: Ohan Oda, Levi Lister, and Wei Teng
		Waypoints and obstacles can be attached to physical tokens and moved during gameplay. Developers: Ohan Oda, Levi Lister, and Wei Teng

Goblin XNA Domino Knockdown

Goblin XNA Domino Knockdown was created to explore how well Goblin XNA games would work on a UMPC with a modest computational profile: a Sony VGN-UX390N with Intel Core Solo U1500 (1.33 GHz), Intel GMA 950, 1 GB DDR2, and a 1024×600 touchscreen display, running Windows Vista Business. The user begins by positioning virtual dominoes on a tracked physical game board. When ready, the user fires balls by tapping on the touchscreen.  The object of the game is to knock all the dominoes off the board.  Game physics are simulated using Goblin XNA's interface to Newton Game Dynamics. 

		The user fires balls at a board filled with dominoes, trying to knock all the dominos off the board. Developer: Ohan Oda

Spring 2007 Final Projects for 3D User Interface Design (COMS W4172)

Goblin XNA was first implemented as Goblin v2, built on top of Truevision3D, an indie 3D development environment that supports C#. Goblin v2 was used by students for individual and team assignments in 3D User Interface Design (COMS W4172), taught by Prof. Steve Feiner in Spring 2007.  3D position and orientation tracking was accomplished using inexpensive web cameras and the Goblin v2 interface to ARTag,

Team final projects used augmented reality to create situated visualizations that overlay with additional information the site of Columbia’s proposed Manhattanville campus, both in miniature and at full scale. Guided by Prof. Petia Morozov from the Graduate School of Architecture, Planning and Preservation, and Sean White, students used web-based information sources to determine questions to address, and designed visual representations and techniques for interacting with their visualizations indoors and outdoors. Each of the screen shots shown here was captured live from a final project by one of the six teams. 

		•Buildings overlaid on tracked image of urban site are selected using a two-handed controller created from a hand-held box covered with tracker markers. Team: Augmented Realness (Mike Sorvillo, Levi Lister, Taran Singh, and Sasha Stoeva)
		User navigates with arm-worn and hand-worn virtual controls. Team: SKY (Seungjin Nam, Kyunghwan Kim, and Yonghan Kim)
		Tracked finger selects from fisheye menu of buildings. Team: Delta (Hadi Fadaifard, Zhiyang Cao, Cheng-Chih Yang, and Wei Teng)
		2D slider is controlled with tracker marker to filter buildings. Team: Quirk (Cindy Liao, Connie Shi, Nick Tan, and Jessica Parker)
		Physical marker is used to position and orient virtual representation of a large marker array employed in full-scale outdoor tracking. Team: The View (Varun Maithel, Armando Ramirez, Michael Wasserman, and William Yin)
		The site model is registered relative to a large marker array. Team: 76ers (Steven Chaitoff, Jia Hou, and Ho Yin Cheng)

Downloads

Acknowledgments

This research is supported by a generous gift from Microsoft Research.

Michael Sorvillo, Levi Lister, and Sean White helped to develop the platform.

Back to the Columbia CGUI Lab home page.