Multitouch in Firefox 4

The updated Firefox 4 Beta was released today, and one the new features in this beta is the multitouch API that we’ve been working on for the past months. This means that regular webpages are capable of using  multitouch input, allowing webdevelopers  to create new interactive experiences for their websites. You can read all about it and see a really cool demo of what’s possible in Paul Rouget’s article on hacks.mozilla.org.

On this post I’ll give a broader overview of the APIs, samples and development methods.

How to use the APIs

The multitouch API is very simple and meant to get web developers started quickly. It works through the use of DOM Events, similarly to how other kinds of input work (e.g. mouse, keyboard). It consists of three different events: MozTouchDown, MozTouchMove, MozTouchUp, which are very similar to the mouse events already familiar to web developers.

They have all the common properties from mouse events, plus an extra field called streamId, which is a random number that is an unique identifier for each point of contact, and this number is kept constant while you don’t lift that finger from the screen. It makes it possible to accurately track the movement of each finger without mixing them.

// Some simple usage example:
document.addEventListener("MozTouchDown", function(event) {
  trackedFingers[event.streamId] = {
    X: event.clientX,
    Y: event.clientY
  }
}, false);

document.addEventListener("MozTouchMove", function(event) {
  //update X/Y positions, do something interesting
}, false);

document.addEventListener("MozTouchUp", function(event) {
  delete trackedFingers[event.streamId];
}, false);

Supporting API: .mozInputSource

There are two other supporting APIs. If you don’t need to deal with multiple input, but still want to handle touch or pen input differently, you can access the .mozInputSource property of regular mouse events. With this property, you can tell if that event was generated by a mouse, by pen or by touch input. This property is not only available on low-level mouse input events, but also on click, dblclick and drag&drop events.

document.addEventListener("click", function(event) {
alert(event.mozInputSource);
}, false);

Supporting API: CSS media query

If you want to make style changes for your webpage when being accessed using a touch-enabled device, you can use the -moz-touch-enabled CSS media query and add your custom style inside that selector

@media all and (-moz-touch-enabled) {
  /* touch specific css goes here */
}

Code snippets and examples in the wild

While the APIs were being developed, there was a handful of demos created to test the implementation and to highlight some interesting use cases for touch. You can see some of those demos here (video and source code):

Other people have been trying the work in progress and built more examples. Al MacDonald has a very nice generative music mash-up of multitouch with the experimental Audio data API.

How to get started on development

There are various ways to get started developing for multitouch, depending on the equipment and OS that you use. At the moment, we support devices that are compatible with the Windows 7 multitouch input, such as various models of notebooks and touchscreen-monitors.

However, it’s possible to get homemade devices that uses the TUIO protocol to work, using projects such as MultitouchVista. It features a set of drivers that gets the input from various sources, such as homemade touch tables, multiple mice, among others, and outputs data to the Windows’ standard APIs.

If you have a spare webcam and a free weekend for a DIY project you might be able to build one yourself following various good tutorials that exist.

In addition to that, Hernán Colmeiro has created a multitouch simulator for his Google Summer of Code project. This is a platform-independent Firefox add-on that is able to replay multitouch input from hand-written or recorded JSON files. This add-on is built using Jetpack, and you don’t even need to restart Firefox to install it. You can use it to develop and test a webpage for multitouch input and your code doesn’t need any special handling for the add-on; it works as is. You can read more about it, watch a screencast of how it works and see a standalone demo of its playback feature over at Hernan’s blog.

Feedback

We hope this API is useful to get this feature in the hands of web developers. We will soon be starting dicussions to see how to standardize these APIs at the W3C.  Seeing what people will create with it and getting feedback will be invaluable to make sure that the standard is useful for the web developers.