Posted by: felipe | August 11, 2010

Multitouch in Firefox 4

The updated Firefox 4 Beta was released today, and one the new features in this beta is the multitouch API that we’ve been working on for the past months. This means that regular webpages are capable of using  multitouch input, allowing webdevelopers  to create new interactive experiences for their websites. You can read all about it and see a really cool demo of what’s possible in Paul Rouget’s article on hacks.mozilla.org.
On this post I’ll give a broader overview of the APIs, samples and development methods.

How to use the APIs

The multitouch API is very simple and meant to get web developers started quickly. It works through the use of DOM Events, similarly to how other kinds of input work (e.g. mouse, keyboard). It consists of three different events: MozTouchDown, MozTouchMove, MozTouchUp, which are very similar to the mouse events already familiar to web developers.
They have all the common properties from mouse events, plus an extra field called streamId, which is a random number that is an unique identifier for each point of contact, and this number is kept constant while you don’t lift that finger from the screen. It makes it possible to accurately track the movement of each finger without mixing them.
// Some simple usage example:
document.addEventListener("MozTouchDown", function(event) {
  trackedFingers[event.streamId] = {
    X: event.clientX,
    Y: event.clientY
  }
}, false);

document.addEventListener("MozTouchMove", function(event) {
  //update X/Y positions, do something interesting
}, false);

document.addEventListener("MozTouchUp", function(event) {
  delete trackedFingers[event.streamId];
}, false);

Supporting API: .mozInputSource

There are two other supporting APIs. If you don’t need to deal with multiple input, but still want to handle touch or pen input differently, you can access the .mozInputSource property of regular mouse events. With this property, you can tell if that event was generated by a mouse, by pen or by touch input. This property is not only available on low-level mouse input events, but also on click, dblclick and drag&drop events.

document.addEventListener("click", function(event) {
alert(event.mozInputSource);
}, false);

Supporting API: CSS media query

If you want to make style changes for your webpage when being accessed using a touch-enabled device, you can use the -moz-touch-enabled CSS media query and add your custom style inside that selector
@media all and (-moz-touch-enabled) {
  /* touch specific css goes here */
}

Code snippets and examples in the wild

While the APIs were being developed, there was a handful of demos created to test the implementation and to highlight some interesting use cases for touch. You can see some of those demos here (video and source code):
Other people have been trying the work in progress and built more examples. Al MacDonald has a very nice generative music mash-up of multitouch with the experimental Audio data API.

How to get started on development

There are various ways to get started developing for multitouch, depending on the equipment and OS that you use. At the moment, we support devices that are compatible with the Windows 7 multitouch input, such as various models of notebooks and touchscreen-monitors.
However, it’s possible to get homemade devices that uses the TUIO protocol to work, using projects such as MultitouchVista. It features a set of drivers that gets the input from various sources, such as homemade touch tables, multiple mice, among others, and outputs data to the Windows’ standard APIs.
If you have a spare webcam and a free weekend for a DIY project you might be able to build one yourself following various good tutorials that exist.

In addition to that, Hernán Colmeiro has created a multitouch simulator for his Google Summer of Code project. This is a platform-independent Firefox add-on that is able to replay multitouch input from hand-written or recorded JSON files. This add-on is built using Jetpack, and you don’t even need to restart Firefox to install it. You can use it to develop and test a webpage for multitouch input and your code doesn’t need any special handling for the add-on; it works as is. You can read more about it, watch a screencast of how it works and see a standalone demo of its playback feature over at Hernan’s blog.

Feedback

We hope this API is useful to get this feature in the hands of web developers. We will soon be starting dicussions to see how to standardize these APIs at the W3C.  Seeing what people will create with it and getting feedback will be invaluable to make sure that the standard is useful for the web developers.
About these ads

Responses

  1. [...] Multi-touch Firefox 4 Beta now has multi-touch support for Windows 7 to enable people to interact with Web content with their hands naturally. Developers can use multi-touch capabilities to leverage touch-enabled devices and provide people with an intuitive and fun browsing experience. For more details, see the blog posts by Paul Rouget and Felipe Gomes. [...]

  2. [...] Edit: If you want more details, take a look at Felipe’s latest blog post. [...]

  3. [...] This post was mentioned on Twitter by Planet Mozilla and Firefox Help, Planet Repeater. Planet Repeater said: Felipe Gomes: Multitouch in Firefox 4 http://dlvr.it/3gNSL [...]

  4. [...] compatibility on Windows 7. This means that developers will be able to take advantage of the new multitouch API, which is expected to eventually filter down to users through [...]

  5. [...] compatibility on Windows 7. This means that developers will be able to take advantage of the new multitouch API, which is expected to eventually filter down to users through [...]

  6. Hello Felipe,

    Are any of the demos online available.
    I’ve got some Multitouch-devices running Windows 7 were I’d like to test the new Multitouch-capabilites of Firefox with.

  7. yeah, no demos. am i blind or is it fake

  8. [...] aportadas por CSS3 y HTML5. Y por otro lado, se basa en los experimentos de Felipe Gomes, quien creo la API y se encargo del desarrollo en si. Después del salto, un video [...]

  9. [...] to Mozilla’s browser and are currently only implemented on the Windows platform. In a recent blog entry, Firefox developer Felipe Gomes says that Mozilla hopes to bring the new APIs to W3C so that they [...]

  10. [...] to Mozilla's browser and are currently only implemented on the Windows platform. In a recent blog entry, Firefox developer Felipe Gomes says that Mozilla hopes to bring the new APIs to W3C so that they [...]

  11. [...] to Mozilla’s browser and are currently only implemented on the Windows platform. In a recent blog entry, Firefox developer Felipe Gomes says that Mozilla hopes to bring the new APIs to W3C so that they [...]

  12. [...] to Mozilla's browser and are currently only implemented on the Windows platform. In a recent blog entry, Firefox developer Felipe Gomes says that Mozilla hopes to bring the new APIs to W3C so that they [...]

  13. [...] in Websites zu implementieren. Ausführlichere Informationen gibt es im Blog von Felipe Gomes. Firefox 4.0 mit Multitouch #2Web-Entwickler können so direkt Touch-Events von Windows 7-Nutzern [...]

  14. I understand that what apple has done for touch events is not a standard, but what is wrong with what they did? This just seems like a subset of what apple has done. Well based on my reading of this a more awkward to use version of what apple is done. And it looks to be missing the GestureEvent set of events as well as a bunch of the handy attributes that are on the apple events.

    Please do not be different for the sake of it. Apple has done a nice job of touch events, they are quite easy to use. Why not work with them to bring their touch APIs to the W3C instead of creating a competing standard?

    The apple API is already out there on every apple and android touch device. The apple API is better than this one. It is not like all the webkit devices are going to drop a more featured API for a less featured one. This will just create a separate API for everyone to support.

    • In my experience some apple events are a little clumsy. And not everything is functional. For example: in Apples implementation:
      1) Start touching with fingerA
      2) Start touching with fingerB
      - At this point both fingers are being tracked.
      3) Eelease one of the fingers
      - Touch end is fired for both fingers and no more tracking is done for the finger that still on the screen.

      This sems a problem related with the array of touch points that apple uses. I dislike it a lot. I like the Mozilla implementation better. AFAIK some of the demos shown here would not be possible by the current Apple API.

      • Actually, that is not true. When you release one finger, you get a touchend event, with the released touch in changedTouches. The second finger will still send touchmove events when moved, and eventually a touchend event of its own.

      • Anders, you’re right. My bad.
        I’ve read it somewhere, I’ve tryed to make it work, dispite of what I’ve read and I failed to notice that you can keep track of the touchs in a way that the array indexes would not affect the code. I even found a good example that works:

        http://www.floydprice.com/2010/09/ipad-multi-touch-in-javascript/
        (demo link with code on the last paragraph)

        Thank you very much to point my mistake =)

  15. [...] “This means that regular webpages are capable of using multitouch input, allowing web developers to create new interactive experiences for their websites,” noted Mozilla developer Felipe Gomes in a blog post. [...]

  16. [...] compatibility on Windows 7. This means that developers will be able to take advantage of the new multitouch API, which is expected to eventually filter down to users through [...]

  17. [...] to Mozilla’s browser and are currently only implemented on the Windows platform. In a recent blog entry, Firefox developer Felipe Gomes says that Mozilla hopes to bring the new APIs to W3C so that they [...]

  18. [...] compatibility on Windows 7. This means that developers will be able to take advantage of the new multitouch API, which is expected to eventually filter down to users through [...]

  19. [...] compatibility on Windows 7. This means that developers will be able to take advantage of the new multitouch API, which is expected to eventually filter down to users through [...]

  20. [...] aportadas por CSS3 y HTML5. Y por otro lado, se basa en los experimentos de Felipe Gomes, quien creo la API y se encargo del desarrollo en si. Después del salto, un video [...]

  21. [...] allowing webdevelopers  to create new interactive experiences for their websites," said this post on a Firefox developer’s blog (head there if you want to read about how the API works – [...]

  22. [...] 7 でのマルチタッチへの対応 (詳細は Paul Rouget と Felipe Gomes [...]

  23. [...] Multi-touch Firefox 4 Beta now has multi-touch support for Windows 7 to enable people to interact with Web content with their hands naturally. Developers can use multi-touch capabilities to leverage touch-enabled devices and provide people with an intuitive and fun browsing experience. For more details, see the blog posts by Paul Rouget and Felipe Gomes. [...]

  24. [...] making way cool demos. For further information about multitouch in Firefox 4 I recommend reading Felipe’s blog post about it as well as Paul’s article on [...]

  25. [...] far as I know (from here) FireFox 4 on Windows 7 supports similar [...]

  26. Which API is better: iOS Safari touch event or Firefox 4 touch event?…

    Currently, touch event API is not finalized by W3C. Since Apple leads touch device (especially multitouch device), its implementation in Safari can be de facto standard. Firefox 4 is planned to be released next month; and its touch event API is quite d…

  27. [...] with HTML5 canvas?with CSS3 and jquery?with a javascript framework?with touch events like this: http://felipe.wordpress.com/2010… What do you recommend?  BIU     @   Edit Link [...]

  28. [...] check out Felipe Gomes excellent post on multitouch in Firefox 4. [...]

  29. szq96j http://dhY3n0fjvTtj48mG9sFnCv.com

  30. Do you have a spam problem on this site; I also am a blogger, and I was curious about your situation; many of us have developed some nice methods and we are looking to trade solutions with others, why not shoot me an e-mail if interested.

  31. Could you sent me your example? the link was invalid.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: