Since the introduction of the iPhone, the mainstream mobile user has had a new revelation about the power and abilities of touchscreens. I encounter people who are regularly in awe about what any device with a touchscreen can do.
Granted, part of this is because of the terrible user interfaces that mobile device users have been subjected to over the years. In many ways, one can say that touchscreens are a revelation mainly because we’ve been forced to learn something new when we already had digits to point and select with that we already knew how to use.
That being said, touchscreens as we know them are still probably not truly paradigm changing until they do more. They need to get out and touch our worlds with enhanced abilities beyond just point and show.
With the iPhone, Instinct, and several other tablet-style mobiles here and coming soon, what could be next in terms of touchscreens and interactive mobile displays? Some things are just a matter of changing the design a bit, but others are going to be a bit of un-learning on our parts. And it might open up some incredible possibilities.
Gestures, or the act of using some flowing finger or hand motion to initiate an action, is something that we’ve seen since the early years of computing. With devices like the iPhone and Instinct however, gestures have become an artful way to add usability to mobile devices without adding additional buttons.
Gestures can take either of the following forms:
- physically moving the device from a stationary point in a uniform motion
- physically moving a finger or fingers across the screen in a uniform motion
The recognition of that motion by the software and hardware delivers an effect such as flipping a page or scrolling a website.
What is starting to happen with gestures is that developers are seeing the fun that people are having with the Nintendo Wii, and integrating some of that functionality into mobile games and other applications.
For example, a few racing games in development for the iPhone and Nokia S60 devices allow the use of the mobile device as a steering wheel. The screen shows where the user is going, and either pressing a button or tilting the device causes the virtual car move forward or stop.
Another example is the navigation software on the T-Mobile G1. This will allow you to view a panoramic image of a remote location, and then scroll around on the image by holding the device up and moving it around.
Projection with Gestures
Taking the idea of gestures a bit further, I can see the idea of using projectors built into mobile devices working along with gestures to create new mobile interfaces.
Currently, projection technology is being developed that will fit a projector into a mobile device that would allow it to use any a large, flat surface as a display. Taking that and adding the idea of gestures – with either moving the device itself or the user moving their hands in front of the mobile device – can create other ways of interacting with content. We already have a basic semblance of this in the wireless laser keyboard.
An idea that I see here is being able to take a picture of an unfamiliar area, and then project the image onto the ground. Then, using the built-in GPS of the device along with gesturing, a person would be able to not only find out where they are, but also search the image’s context for online references for points of interest — parks, ATMs, restaurants, etc. The idea being to take the physical environment and meld it with a virtual one with the mobile device as the “controller.”
We’ll Have to Wait and See
What is to come is not really known at this point. New mobile interfaces come onto the scene every 18 to 24 months, and it usually takes something major for them to become paradigm shifting. We saw this with the QWERTY keyboards RIM and Palm made famous, and then with the capacitive and multi-touch screens that Apple has brought onto the scene in the past year.
What ever new mobile-friendly input mechanism will have to answer to the challenge of little space and high efficiency. No matter the input element, if a person cannot find information quickly and easily, the technology will quickly fail the test of what works best for the next generations of mobile device users.