MobiVR - A User Interface for Mobile Devices
MobiVR is a novel, pragmatic gestural user interface for future hand-held computing.
MobiVR won the first prize in the Innovation Contest 2003
of the Finnish New Technology Foundation.
A conference paper was published
on the 4th International Workshop on Mobile Computing (IMC'2003), Rostock, Germany, June 17-18, 2003.
Communications and computing are converging due to the underlying digital
technologies. The dominant paradigm of computing is changing towards mobile
computing and wireless Internet. Communication and access to information
seem to become increasingly important. Future personal computers may mostly
be hand-held devices. They should be small, yet their displays, interfaces,
and buttons should be big and convenient to use. This imposes new requirements
for the user interfaces (UI).
The current UI approaches like Windows, Icons, Menus, and Pointing (WIMP)
user interfaces employing I/O devices like mice and keyboards are mostly inappropriate
for the future hand-held computers, which will come in diverse form factors
and functions like personal digital assistants (PDAs), hand-held game consoles,
smart phones, digital audio players, smart cameras, wristwatches, etc. The
pen-based UI of PDAs is good for some purposes, but the small, low-resolution
screen will not be enough for many mobile tasks like web browsing.
Near-eye microdisplays would provide lightweight and high-resolution virtual screens
for wearable computers and hand-held devices. Some prototype mobile phones and PDAs with
near-eye microdisplays have already been presented. Microdisplay units
will be embedded into mobile phones and Internet appliances.
A microdisplay can solve the display issue, but the UI becomes then a problem. Cursor movement could be handled by e.g.,
a touchpad or a tiny joystick, but they may be slow and clumsy to operate
with the same hand that is holding the hand-held device. Hand gestures could
be a more natural and intuitive way of pointing.
MobiVR is a novel, multimodal approach for the user interface of mobile devices.
The Basic Idea of the MobiVR Concept
The basic components of MobiVR are a wireless communication channel, a near-eye
microdisplay, and a tracking system to track a finger or a pointing device (see
Figure 1). Finger tracking based on camera and computer vision is a convenient
and intuitive method of pointing, as no hand-held or finger-mounted artifacts
are needed. The display can show e.g., a PC desktop view in a pocket-sized
device, and the pointing finger can replace a mouse.
Figure 1. The basic components of the MobiVR user interface. The optics (1)
and the microdisplay (2) show the UI of a remote appliance. The camera (3)
tracks the user’s finger. Optional IR light sources (4) and IR pass filter
(5) can simplify tracking.
MobiVR provides a kind of a “virtual touch screen” for the user. The user
can point to the items on the virtual display with a finger (see Figure 2),
and thus replace a mouse. In addition to pointing, gestures could be used
also for selection and other purposes. Selection can also be made e.g., by
a conventional button or other methods.
Figure 2. The user can point a finger to the objects on the virtual screen.
We have searched and gone through a massive amount of prior art and research.
There are some works, which are a little similar to MobiVR, but there are
also several major differences. The MobiVR seems to be a novel innovation.
MobiVR can be thought of and applied in many ways. It could be seen as a
hand-held web browser, an integrated UI component of a hand-held appliance,
a universal information appliance, a ubiquitous remote control, etc. Its
design can resemble e.g., a camera, a stick, a remote control, or a binocular,
as in our early prototype.
Contact: Dr.Tech. Ismo Rakkolainen
University of Tampere, Finland
Ismo.Rakkolainen @ uta.fi