²

To start with, I would like to touch on the potential applications of this technology. It is my belief that we have only just begun to harness the potential afforded by such a naturally intuitive interface system. Gesture and handwriting recognition have come so far recently, along with other advanced body language reading (eye/face tracking etc.), that it is theoretically possible to control a computer without the use of a mouse or keyboard at all. This has so far been the most use to high input creative applications, especially ones that take place live, such as musicians (Beardy Man) and television studios (Campbell Live). The freedom of placing anything on the screen and having it instantly become a usable tool is an immense breakthrough in my opinion, it is like being able to have a hammer that can also be used to weld a car together.

I feel that the nature of this technology is such that it should be available on a mainstream level. There is no reason I can see that the keyboard and mouse should be held in favor of something with the potential for the immense freedom that this has. To this end I propose a project that looks into the current level of this technology and perhaps take it in a direction that the large corporations have not yet thought of.
The breakdown of where I would like this to go is as follows.

• Research multi-touch technology, its past, present, and potential future.
• Identify the areas we will focus on developing.
• Conceptualise potential applications.
• Develop a working prototype.
• Develop those areas using that prototype.
• Produce an advanced result.
• Potentially market that result?

For me, the problem begins where AUT owns any idea we come up with, but I also believe in always having something up my sleeve!

As for the direction this project will take, I am thinking of a 1-person work station that combines screen and input device in an ergonomic fashion. Such a device may wrap around the user and run a kind of Shell OS that combines several different programs with the same style of intuitive gesture inputs. These pieces of software can vary, from virtual DJ/VJ equipment, to image editing software, to advanced web browsing and more. Ideally I would like to see a set of drivers that means that the device and its gestures can be used with any 3rd party software, something like recording a gesture as a keyboard shortcut and sending that to the application.

The potential for recruiting students in other years to work on software and applications is also high. For instance, if a student was interested in creating a piece of software that manipulated video using a touch interface, they would not need to go back to basics to create the touch interface in the first place.

Ideally I would like to continue to do other projects over the course of the next few years, but have them relate back to the development of this as much as possible (for instance having a robot that is controlled through the interface, or an exhibition in which a sculpture reacts dynamically to touch input, etc.). The idea being, to have a collection of valuable work that ties together by the end of my time in BCT.