Posted by: felipe | August 21, 2009

Sneak peak on multitouch events

I’ve been anxious to demonstrate the progress on our multitouch support for Firefox, and here’s a video which showcases some possible interactions and use cases for what web pages and webapps can do with a multitouch device.

We’re working on exposing the multitouch data from the system to regular web pages through DOM Events, and all of these demos are built on top of that. They are simple html pages that receives events for each touch point and use them to build a custom multitouch experience.

We’ll also add CSS support to detect touchscreen devices. Using the pseudo-selector :-moz-system-metric(touch-enabled), you can apply specific styles for your page only if it’s viewed by a touchscreen user. That, along with physical CSS units (cm or in), makes it possible to adjust your webapp for a touchscreen experience.

Check out a short video with the demos in action (.ogg), .mp4, or embedded below, and some screenshots for reference (I added some marks to the images showing where the contact points are).

Here is an example of what the API looks like for now. We have three new DOM events (MozTouchDown, MozTouchMove and MozTouchRelease), which are similar to mouse events, except that they have a new attribute called streamId that can uniquely identify the same finger being tracked in a series of MozTouch events. The following snippet is the code for the first demo where we move independent <div>s under the X/Y position of each touch point.

var assignedFingers = {};
var lastused = 0;

function touchMove(event) {
var divId;
if (lastused < = 4) return;
if (assignedFingers[event.streamId]) {
divId = assignedFingers[event.streamId];
} else {
divId = "trackingdiv" + (++lastused);
assignedFingers[event.streamId] = divId;
}
document.getElementById(divId).style.left = event.clientX + 'px';
document.getElementById(divId).style.top  = event.clientY + 'px';
}
document.addEventListener("MozTouchMove", touchMove, false);
document.addEventListener("MozTouchRelease", function() { lastused--; }, false);

On the wiki page you can see code snippets for the other demos. Any comments regarding the demos or the API are really welcome. We hope to start some good discussion on this area. Hopefully as touch devices (mobile and notebooks) are getting more and more popular we’ll see new and creative ways to use touch and multitouch on the web. Code snippets for the demos.

About these ads

Responses

  1. [...] This post has also been cross-posted to Felipe’s personal blog. [...]

  2. [...] intern Felipe Gomes has put up a very nice video showcasing a few demos of what can be done with the new multitouch support coming on Windows [...]

  3. [...] [Felipe’s Blog] Also Read Show Me your FirefoxFirefox Mobile: Download Fennec Maemo 1.0 Beta 3Skype 2.0 beta for [...]

  4. Why -moz-system-metric and not a new CSS media query value?

    • @roc: we went and used the css selector because it was a simple and traditional way to add a moz specific value for now, and it suited well for making small CSS changes instead of a whole new file or section as would be with a media query or a conditional file in browser/skin/.

      However, a media query makes sense to be considered too. Although, as I understand, they’re mutually exclusive, so there might be a problem in this (e.g. a screen user agent could support touch as well as a mobile user agent).

      Another thing that will make sense to add in the future will be a property to check for touch capabilities from JS as this is CSS only.

      • I’m not talking about CSS media values like ‘screen’ etc, but CSS media queries, which support an extensible set of independent queryable properties:
        http://www.w3.org/TR/css3-mediaqueries/

        There is a draft spec to provide script access to those values:
        http://www.w3.org/TR/2009/WD-cssom-view-20090804/#dom-media-matchmedium

      • Cool, thanks for the pointer Roc. I didn’t know about those. Let’s see if I’ll have an implementation of that in the following week :)
        I couldn’t find any info regarding this in the spec, but: is the list of features there considered complete, or just a minimal set for implementors? Should we suggest adding _touch_ as a new feature for the spec?

  5. [...] través de Mozilla Links me entero que nuestro amigo (?) Felipe Gomes ha grabado una demostración de lo que es el soporte multitouch de Firefox. No tiene nada que ver con la compatibilidad multitouch para el touchpad, es para dispositivos que [...]

  6. [...] través de Mozilla Links me entero que nuestro amigo (?) Felipe Gomes ha grabado una demostración de lo que es el soporte multitouch de Firefox. No tiene nada que ver con la compatibilidad multitouch para el touchpad, es para dispositivos que [...]

  7. [...] disponibili solamente alcuni frammenti di codice, e qualche informazione, reperibile sul blog di Felipe Gomes, uno degli ideatori. Ciao! Benvenuto su Geektwice! Ti consiglio di sottoscrivere il nostro RSS [...]

  8. [...] Gomes has extended multitouch support in Firefox to reach into content space (i.e. us Web devs can use it!). Checking out the sexy demos in the [...]

  9. [...] Gomes has extended multitouch support in Firefox to reach into content space (i.e. us Web devs can use it!). Checking out the sexy demos in the [...]

  10. [...] Gomes has extended multitouch support in Firefox to reach into content space (i.e. us Web devs can use it!). Checking out the sexy demos in the [...]

  11. [...] info: felipe's Blog Loading google.load('search', '1'); google.setOnLoadCallback(function(){ new [...]

  12. [...] Sneak peak on multitouch events « felipe’s Blog felipe.wordpress.com/2009/08/21/sneak-peak-on-multitouch-events – view page – cached #felipe’s Blog RSS Feed felipe's Blog » Sneak peak on multitouch events Comments Feed felipe's Blog Soon to be started The multitouch project status — From the page [...]

  13. [...] Gomes has extended multitouch support in Firefox to reach into content space (i.e. us Web devs can use it!). Checking out the sexy demos in the [...]

  14. [...] dokunmatik ekranlarda web tarayıcı kullanılabilmesini sağlıyor. Mozilla programcılarından Felipe Gomes‘un geliştirdiği Firefox multitouch’un demosunda tarayıcı içinde çalışan [...]

  15. [...] Gomes, interno de Mozilla, ha subido un muy buen vídeo mostrando unas pruebas de lo que se puede hacer con el nuevo soporte multitáctil que vendrá en [...]

  16. Nice work.

    Can you please let me know what kind of device (e.g. Dell Laptop some model number) you are using to support multi-touch with Firefox?

    Many thanks.

    Cheers Craig

    • @craig: there are a couple of multitouch devices in the market right now. The one in this video is an HP Touchsmart TX2.

  17. [...] προγραμματιστής Felipe Gomes εργάζεται, όπως αναφέρει στο blog του, πάνω στην ενσωμάτωση της υποστήριξης του [...]

  18. [...] Felipe Gomes Blog [...]

  19. [...] Source [...]

  20. [...] go to the top or bottom of a Web page, for example, or two-finger pinch gestures to zoom out. But Felipe Gomes, a Brazilian computer science student who just finished a stint as a Mozilla intern, has [...]

  21. [...] go to the top or bottom of a Web page, for example, or two-finger pinch gestures to zoom out. But Felipe Gomes, a Brazilian computer science student who just finished a stint as a Mozilla intern, has [...]

  22. [...] go to the top or bottom of a Web page, for example, or two-finger pinch gestures to zoom out. But Felipe Gomes, a Brazilian computer science student who just finished a stint as a Mozilla intern, has [...]

  23. [...] sulla creazione di un interfaccia multi-touch per la vostra applicazione, raggiungete il sito di Felipe Gomes dove troverete anche alcuni codici di [...]

  24. Just a quick question is the currently plan to also support single touch devices for the events and css as well?

    Still interesting to see where this goes…

  25. Nice work. Is there going to be a draft specification submitted by mozilla or webkit for a multitouch API? I tend to lean more towards webkits implementation in the iPhone of ontouchstart, ontouchmove, ontouchend.

    I agree with robert that CSS styling would be better used in a media query, like webkit does for their transitions etc using @media (-webkit-transition)

  26. [...] Gomes programador de la comunidad Mozilla presenta en su blog el soporte multitouch que incorporará la versión 3.6 de Firefox. Esta funcionalidad sólo pordrá [...]

  27. [...] Mozilla-Entwickler Felipe Gomez arbeitet zur Zeit an einer Multitouch-Unterstützung für Firefox. Die Multitouch-Daten sollen über DOM-Ereignisse mit Elementen einer Webseite [...]

  28. [...] [...]

  29. [...] que quem apresenta esta “funcionalidade” é um brasileiro que está trabalhando lá, o Felipe Gomes. Ponto para o [...]

  30. [...] Can you think of any other cool stuff that you could do with a touchscreen interface on a web browser? Share your ideas in the comments below. var a_vars = Array(); var pagename=''; var phpmyvisitesSite = 8; var phpmyvisitesURL = "http://www.potpot.fr/stats/php.php&quot;; Read [...]

  31. [...] y casos de uso para sitios web y aplicaciones.”, dice Felipe Gomes, interno de Mozilla, desde su blog oficial. “Hemos estado trabajando en exponer la información multitáctil del sistema a [...]

  32. [...] you are interested in multitouch and Firefox make sure to read this blog post by Felipe. var addthis_pub = ''; var addthis_language = 'en';var addthis_options = 'email, [...]

  33. [...] go to the top or bottom of a Web page, for example, or two-finger pinch gestures to zoom out. But Felipe Gomes, a Brazilian computer science student who just finished a stint as a Mozilla intern, has [...]

  34. [...] Fuente y videos:  Felipe’s Blog [...]

  35. What about using firefox multitouch from my macbook ?

    The Macbook’s trackpads include multi-touch technology too.

  36. [...] Daha fazla bilgi için: Sneak Peak on Multitouch Events [...]

  37. [...] Felipe Gomes is te zien in een video-demo waarin hij de status toont van het “MultiTouch Polish” [...]

  38. [...] Gomes has extended multitouch support in Firefox to reach into content space (i.e. us Web devs can use it!). Checking out the sexy demos in the [...]

  39. [...] artık çoklu dokunmatik destek verecek. Geliştirici Felipe Gomes bu projenin ilk sonuçlarını kendi web günlüğünde yayımladı. Açıklamalara göre Firefox’taki dokunmatik yönetim, Html sayfalarındaki DOM [...]

  40. [...] Multitouch-tauglich werden. Erste Ergebnisse dieses Projekts zeigt Entwickler Felipe Gomes jetzt in seinem Blog. Die Multitouch-Funktionalität wird demzufolge auf normalen HTML-Seiten mittels DOM-Ereignissen [...]

  41. [...] que Mozilla proporciona para su comunidad. Ahora, como usuario, ¿qué opinas? Enlaces Fuente: Felipe Gomes Ejemplos: Wiki [...]

  42. [...] Ein brasilianischer Student namens Felipe Gomes zeigt in diesem Video, wie man einen Multi-Touch Screen im Firefox integrieren könnte. Gegen sich selbst Pong am Touchscreen zu zocken, ganz schön nerdig aber formidabel! Mehr Infos in Felipe’s Blog. [...]

  43. [...] out the groovy clip here, then hit the jump and tell us what you [...]

  44. [...] out the groovy clip here, then hit the jump and tell us what you [...]

  45. [...] out the groovy clip here, then hit the jump and tell us what you [...]

  46. [...] Check out the demo  from Felipe Gomes. [...]

  47. [...] out the groovy clip here, then hit the jump and tell us what you [...]

  48. This is very nice work! From what source do you capture the events from? Is it a TUIO server, or are they events available in windows7 ?

    I’m also a student computer engineering in Ghent (Belgium), and last semester we too worked on integrating multitouch with Firefox by means of a Firefox extension called PookyTouch — you may have allready seen it around (http://pooky.sourceforge.net/). We used a different API however, inspired by the one used by Safari and currently intensively used on the iPhone and such.

    We were wondering what your future plans are with this project, and when or how Mozilla plans to bundle your work in a future release of Firefox. We would also be very interested in what you think about the API we used, and about the extension as a whole. Perhaps we could join forces in order to establish a speedy introduction of multitouch in the next Firefox release?

    Please contact me at my e-mailadres, or via our website. Be sure to check out the demo’s we’ve provided there as well :-)

    Greetings,
    Michael.

  49. Thats really good to here that having multi touch support in a browser.

    But what kind of applications can be developed other than the sample examples with the multi touch support.

  50. [...] dokunmatik yönetime de destek verecek. Geliştirici Felipe Gomes bu projenin ilk sonuçlarını kendi web günlüğünde yayımladı. Açıklamalara göre çoklu dokunmatik yönetim HTML sayfalarındaki DOM hareketleri [...]

  51. Does these multitouch events available in Firefox 3.5? Or these will appear in Firefox 3.6?

    • They arn’t available in 3.5 yet.

  52. Very Interesting. I belong to NUIgroup and I’ve been investigating in “indirect” multitouch, which means using low cost multitouch pads and overlap webcam stream on window where multitouch app is running. Check this out:


    http://www.youtube.com/watch?v=klQ3M8oGakk

    there are also some threads on NUIgroup.com forums about this topic. My question is this one: can you add support to display visual feedback in window tabs in Firefox?

    • From what I understand of this project it should be pretty straight forward. Events are received in JavaScript, so it should be very easy to display touch-events on the screen.

      When you take the PookyTouch extension for Firefox alongside the jPooky plugin for jQuery you should allready be able to get visual feedback with only one line of code: $.visualizeTouches(). Please try out your multi-touch pad on our website http://pooky.sourceforge.net/ and the demo’s we’ve provided there :-)

  53. Thanks, I downloaded PookyTouch and I was able to use the pad with your multitouch apps on the website (great!) . Anyway It’s not clear to me what you said about visualizeTouches(). Visual feedback for me it’s not only touch points on the webpage, but it’s also needed the image streamed by the webcam in order to see how the hand is going on the screen. Let me know if it’s possible with your plugin to add a transparent video on a webpage in order to do that.

    • At the moment that isn’t possible with PookyTouch (and from what I know of Filipe’s implementation, it isn’t possible with that either).

      We have considered this possibility however, but to be honest it it’s not one of our priorities to implement that. Mostly because the complexity is rather large as we take our input from a TUIO server, and handle the events from there with Java/JavaScript.

  54. I highly enjoyed reading this post, keep on creating such interesting stuff.

  55. [...] arrived, so I dived head-first into Processing.js, equipped with a very special build of Firefox by Felipe Gomes. I have been following Felipe since I found out he was working on Multi-Touch JavaScript events for [...]

  56. [...] dokunmatik yönetime de destek verecek. Geliştirici Felipe Gomes bu projenin ilk sonuçlarını kendi web günlüğünde yayımladı. Açıklamalara göre çoklu dokunmatik yönetim HTML sayfalarındaki DOM hareketleri [...]

  57. [...] arrived, so I dived head-first into Processing.js, equipped with a very special build of Firefox by Felipe Gomes. I have been following Felipe since I found out he was working on Multi-Touch JavaScript events for [...]

  58. i am making a online javascript desktop, and would like to state THANK YOU!!! =)

  59. [...] My demo would not have been complete without the ability to create sound by touch. In this example I am rendering black squares on a white background so the touch points are visible on the low-quality web cam. Apart from the styles, the code is the same as the example above. I am using the HP TouchSmart TX2 with Felipe Gomes’ Firefox Touch API. [...]

  60. Hi,

    I have been trying to get this working on Firefox 3.7 alpha on Windows 7 Professional and an Acer T230H monitor but without luck. I have made a very simple test page available on http://gert.dk/moztouch

    Firefox does respond to the -moz-system-metric(touch-enabled) css selector but I can’t get any reactions in my javascript, regular taps on the monitor is reconized as mouse clicks.

    The drawing example app at http://www.mgalli.com/development/drawing/demoapp.html does not work either (found it through this blog post: http://www.labnol.org/internet/firefox-logo-drawn-using-firefox/10579).

    I have looked through about:config to see if I should enable something to get Mozilla touch events to work but I cannot find anything.

    Does anybody know what it is that I am missing? I’m sure it’s simple once I know but right know I’m stuck.

    Full browser info: Mozilla/5.0 (Windows; U; Windows NT 6.1; WOW64; en-US; rv:1.9.3a5pre) Gecko/20100602 Minefield/3.7a5pre

  61. Oh forgot to mention that the microsoft touch pad example applications all work fine.

  62. Hi Felipe, thank you for your fast reply, I will test it tomorrow.

    Can you tell me (you don’t have to promise anything) if they will make it into 3.7 final or are they scheduled for 4.0 first?

    And thanks a lot for doing this work, kiosks will never be the same again in a very good way :)

  63. [...] Felipe Gomes is te zien in een video-demo waarin hij de status toont van het “MultiTouch Polish” [...]

  64. Hi again Felipe,

    I forgot to write you that everything worked perfectly with your special build.

    Just wanted to hear if there is any news regarding which version of FF you are aiming for this to appear in and/or a donation towards specific features is possible?

    All the best and great work.

  65. submitted to social bookmark, thanks for your useful informations ^_^

    wait before you buy touchscreen monitor

  66. [...] never-tiring folks at Mozilla are already hard at work on implementing multitouch events in Firefox. Felipe Gomes has posted a short demonstration of very cool multitouch capabilities via a few [...]

  67. [...] arbeitet an Multitouch-Events und wartet noch auf Feedback der anderen [...]

  68. […] une démonstration de Felipe Gomes qui nous montre les derniers progrès en matière de multi-touche sous Firefox. Le système en est […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: