Here are this week’s news about our project to support multitouch screens. Besides moving forward on an events API, last week I filed some bugs that aim to improve the overall experience for touchscreens devices (single and multitouch) when surfing on the web. They are:

• Bug 547997 – Perform hit target detection/correction on tap clicks
• Bug 547996 – Be able to tell when a click was generated by touch
• Bug 548005 – Browser zoom should stay centered when coordinates are available
• Bug 548012 – Browser zoom should be cancelable to be handled by webpage

And there are some other already existing bugs so there’s now the meta bug 548100 to keep track of them all.

On the DOM Events API front, on the previous week I unbitrotted the experimental patch to apply to current trunk. And at the moment I’m investigating the Windows 7 API to understand how to deal with the limitation that you can either get raw touch input or touch gestures data, but not both at the same time.

My next goals is to work on the bugs to improve the user experience and to keep moving with the events api making sure to coordinate this work with mobile’s interests too.

Last month, the recently opened brazilian W3C office made an interesting campaign on the UN International Day of Disabled Persons to generate awareness of the importance that is accessibility on the web. During that day, they modified their website in 3 different ways in order to try to simulate how people with certain accessibility needs view and interact with the web.

On one version, they increased the font-size of the whole page, similarly to how users with sight difficulties might increase the page zoom until they’re able to read the content. The second style featured the regular page but with mouse interactions disabled, so you can only navigate on the page using the arrows and the Tab key, as do users who are not able to handle the mouse cursor effectively. The third one featured a blacked out page where only one element is visible at a time, and again you navigate through the page using the Tab key. This experience relates to the interaction that screen readers’ users are used to.

These pages, beyond being examples of the different interactions that users might face on the web, are a great incentive for web designers to start testing what the experience for such users are on their own sites. Usability has fortunately become a popular trend recently, where more designers and web developers are aware of its importance. A recent Jakob Nielsen’s article stated that Anybody can do Usability [tests], where he says that with simple tasks and minimal testing, many usability problems can be detected (and hopefully improved) on a web page. Here’s to hope that the same trend follows to accessibility.

I asked Willie Walker, who I’ve met at JRSL Chile and works with Accessibility on Gnome, what he thought about this campaign, and he said it would be nice if there was some sort of tool that could help web developers to make these tests on their websites. This would indeed be cool. For the first example, you only need to hit Ctrl++ a few times and it’s done. For the other ones, you can work a few modifications to your webpage to get the desired effect, or… or you can have it all automatically done by these two bookmarklets that I prepared! Try it on your own page or your favorite page and see how well it fares. These bookmarklets are by no means perfect, but they should be a good starting point of inspiration. Some things to consider when doing these tests:

• Zoomed page: Hit Ctrl++ and see how well your layout stands. How many times can you zoom it without breaking everything? Try reducing the window size too.
• Mouse disabled: How less obvious does your links and navigation become without any mouse hover? Does your CSS menus break? How is the order of navigation on your links via the Tab key? Hint: you can have a good ordering of elements  via a well-thought mark-up, but you can also explicit change them using the tabindex attribute.
• Screen readers: How well is your content organized? Is the hierarchy too deep and confusing? Does the most important content on your page comes first?

Accessibility bookmarklets

Having noticed that Mozilla is at a moment where there are quite a number of different opportunities to contribute and get involved, for users with all kinds of skills and interests (app developers, web developers, artists, designers), I set out to compile a list for the Mozilla Brazil community.

Alix pointed me to the contribute page, and there’s this page from the mozilla.com website, but I also wanted to list the timely events going on, because they’re hard to keep track if you don’t follow all the news pretty closely, and my friend Marcio commented that he believes that these kind of events have a great engaging component, which I absolutely agree: the challenge of a deadline together with a specific goal is a great motivation to get started.

So here’s a list of what I know is currently going on, bonus added: the target audience, which is useful in helping the new members themselves to find their way in the community and their interests.

• Extend Firefox 3.5 Contest (developers)
  October 2nd
• Jetpack Contest (web developers)
  October 15th
• Fennec call for addons (developers)
  Exciting times ahead for Fennec
• Fall Design Challenge (power users, usability/ux students, infoviz lovers)
  Last week of September
• Firefox Goes Mobile (designers, artists)
  September 30th
• Mozilla Service week (everybody!)
  Do something today or tomorrow!
• Love the web? Show the love and celebrate it (everybody!)
  September 22nd

Some of these are close to their deadlines, but the list is not intended to be a final resource, just a beginning Listing all of these helps put on perspective how many cool and interesting things are going on.

There are other activities which are not time sensitive but are relatively new engaging points worth mentioning: Creating Ubiquity commands or new Personas, participating in the Creative Collective and the Test Pilot. (Or our great Nightly testers)

Hope this comes in handy the next time you want to help a new community member seeking advice on how to be part of the Mozilla world.

I’ve been anxious to demonstrate the progress on our multitouch support for Firefox, and here’s a video which showcases some possible interactions and use cases for what web pages and webapps can do with a multitouch device.

We’re working on exposing the multitouch data from the system to regular web pages through DOM Events, and all of these demos are built on top of that. They are simple html pages that receives events for each touch point and use them to build a custom multitouch experience.

We’ll also add CSS support to detect touchscreen devices. Using the pseudo-selector :-moz-system-metric(touch-enabled), you can apply specific styles for your page only if it’s viewed by a touchscreen user. That, along with physical CSS units (cm or in), makes it possible to adjust your webapp for a touchscreen experience.

Check out a short video with the demos in action (.ogg), .mp4, or embedded below, and some screenshots for reference (I added some marks to the images showing where the contact points are).

Here is an example of what the API looks like for now. We have three new DOM events (MozTouchDown, MozTouchMove and MozTouchRelease), which are similar to mouse events, except that they have a new attribute called streamId that can uniquely identify the same finger being tracked in a series of MozTouch events. The following snippet is the code for the first demo where we move independent <div>s under the X/Y position of each touch point.

var assignedFingers = {};
var lastused = 0;

function touchMove(event) {
var divId;
if (lastused < = 4) return;
if (assignedFingers[event.streamId]) {
divId = assignedFingers[event.streamId];
} else {
divId = "trackingdiv" + (++lastused);
assignedFingers[event.streamId] = divId;
}
document.getElementById(divId).style.left = event.clientX + 'px';
document.getElementById(divId).style.top  = event.clientY + 'px';
}
document.addEventListener("MozTouchMove", touchMove, false);
document.addEventListener("MozTouchRelease", function() { lastused--; }, false);

On the wiki page you can see code snippets for the other demos. Any comments regarding the demos or the API are really welcome. We hope to start some good discussion on this area. Hopefully as touch devices (mobile and notebooks) are getting more and more popular we’ll see new and creative ways to use touch and multitouch on the web. Code snippets for the demos.

Last week 3 patches regarding our touch support for Firefox successfully landed on trunk, and with them I’d say that interacting with Firefox using a touchscreen is getting quite comfortable. In addition, more progress was done in the events front, and I’ve got some cool examples that I hope to demonstrate and blog about soon.

Status
- Beginning of the week: landed Bug 503042 (css selector) and bug 503889 (css changes)
- Bug 503541 (widget gesture registration): worked on it, got r+ from smaug and roc, landed!
- built demos using the multitouch events
- brainstormed with smaug about touch events; wrote details of the current implementation (+ questions) of the events on a wiki page to get comment and feedback

Loose threads
- Publish video with demos and write about them

Next targets
- get feedback and iterate on the current events implementation
- investigate how to tie together the events and the OS gestures

Last week

As part of my internship project on Multitouch for Firefox, last week the main patch for the polish of touch interactions was finally finished and is in review process now. Following that I blogged about the customization UI mockups to receive feedback, and started working on sending touch events to web pages. Here’s a summary of the ongoing activities:

Progress
- finished patch to support touch interactions (bug 503541, going on review now)
- blogged about customization UI mockups
- worked up prototypes to expose multitouch events to web pages
Loose threads
- need to get the patch reviewed and in the tree
Next target
- continue working on the events, determine what data to expose
- eventually produce video of multitouch in action

Plans for this week

- The week is just beginning but we already got some news: the patch for the moz-system-metric(touch-enabled) [bug 503042] selector got sr+, so now we can land that patch and the CSS patch [bug 503889] that was waiting on it.

- Keep working on exposing touch events to content. I am currently writing what we have so far on a wiki page to get feedback on this front, and will have a blog post about that soon.

- Will build 2 or 3 demos of what is already possible to do with these new touch events.

- The patch from last week already got some comments on the review process so I’ll address them for a new review iteration.

- If there’s enough time will work on a new round of the customization mockups taking into consideration the feedback received.

In my internship project on the Firefox team I’ve been working on the browser support for [multi-]touch interactions. If you’re using a tablet computer with support for touch input in Windows 7, you can already interact with Firefox using basic touch actions and gestures, such as panning the webpages, drag ‘n ‘drop things around, scrolling menus and listboxes, and pinching in/out for zoom.

Looking further, we will want to support customization of these gestures, both in the sense of what gestures you can use and what each gesture actually do. For that, I’ve created some UI mockups to see what are the directions we can follow and how we can allow customization of both gestures and actions with a simple UI.

The following screens shows two possibilities for the main customization window. On the left side we have a list of the available gestures for the user. The big icons should be a graphic representation of each gesture, and the small ones icons for the assigned actions. On the right side we have two different views for the list of possible actions. Note that both designs feature easy scrolling and large target areas which are fundamental for touch interactions, where the input is less precise and the screen is partially covered by the user’s hand.

Main View - 1

Main View - 2

Also, it should be interesting to allow the user to practice the gestures and to create new custom gestures. For both of these scenarios, we could use a “gesture canvas”, which would be a big blank area where the user will be free to interact. A subtle grid background provides a better affordance indicating how to interact with the canvas.

Gesture Canvas

When a gesture is being made, a trail displays the current gesture path and the initial touch point for that gesture. This graphic representation should match the icons for gestures on the left. If in practice mode, a light green or red background can quickly fade indicating a success or an error on that gesture. This same fading background can be used on the gesture creation process to indicate if there’s some conflicting gesture already defined.

Gesture Trail

Gesture success

(Some more details if you’re interested)

These are rough first versions of these mockups. I’ll be working on some more iterations soon, so all comments and suggestions are more than welcome.

Making Ubiquity work in different languages is a harder problem than simply translating its strings and making simple adjustments so that the translation feels correct in the target language. Since Ubiquity provides a natural language interface between the user and the computer, the way that the user interacts with the commands should feel natural at his language, conforming (although not strictly necessary) with the language’s grammar, and specially conforming with how the user thinks and expects to give commands in his own language. Here are some thoughts of how the Ubiquity NL interface could work in portuguese.

In portuguese, the verbs inflect a lot. A verb is said slightly differently for each variation of tense/mood/person/number. However, except for the so-called irregular verbs (happens a lot in linking verbs, which are not very useful for ubiquity), every inflection keeps the same root (or stem) from the infitive form. This means that we can let the user write in any form, and from that we can detect the verb root and match that root against the database of commands.

This would work well for most of the verbs commands from Jono’s list. Most of those verbs would be written in imperative form, in a direct interpretation of giving a command to the browser. Some users would prefer to write it in infinitive form, because it’s common in portuguese, but the parsing wouldn’t have a problem with that. So far so good.

However, not for all verb commands that works. For example, the command map is troublesome. The verb map (pt: mapear) in portuguese is somewhat rare, being much more common its noun form (pt: mapa).  We would think of the command “map san francisco” much more as:

Cool, so the 7 facts meme got me! I was tagged by the awesome guy Chris Hofmann, and now that I’ve linked back to him (rule #1), I’ve got to share 7 embarassing facts about me that I should never tell anybody! Let’s go!

The Rules:

1. Link back to your original tagger and list the rules in your post.
2. Share seven facts about yourself.
3. Tag 7 people by leaving name and link to his blog.
4. Let them know they’ve been tagged

The facts:

1. How many of you have spent endless hours watching an screensaver please raise your hands! o/  I have. This awesome screensaver made by Sierra back in the day featured an lonely man trapped in an island, and it was so funny watching him eating coconuts, firing wood to cook, trying to build his never-finished canoe, and other things. There were special events on holidays, and even some rare events where some boats, mermaids, airplanes appeared, mostly to be missed by the guy in a funny way. It was very creative and entertaining.
2. Once, me and some friends from university got together to study overnight for a test on the next day. Instead, we played Line Rider for 4 hours long. We basically mastered the art of the game. And everybody did well on the test.
3. Until 2 years ago, I knew absolutely nothing about movies, except for Matrix or very random movies watched during boring afternoons on TV. Every time a friend would ask “hey, have you watched movie X”, the answer would certainly be No, even if it was the most obvious of the movies. So I decided I needed to do something serious to change this: I bought a book about cinema history, got to know everything at imdb.com, and set to watch as many movies as possible every month. The clerk at the nearby video shop must love me, because I’ve rented so many movies since then there.
4. I’m a huge fan of video games, specially classic consoles (Nintendo 64 and older). Many of the best memories of my kid’s years comes from playing Super Nintendo and mainly Mega-Drive (Genesis), which I swear have some of the best games made ever. During my childhood, my uncle had a game rental shop beside my house, and thus I had an unlimited number of games and possibilities to play every day. A kid’s dream! I also played Frogger on a CP-500 and had an Atari which I played endlessly. My first computer game, the one thing that got me into computers and this beautiful world of technology , is Lemmings.
5. When I was learning C in university, I had read the entire K&R but was still uneasy about the tiny little details that sometimes made difference between the “it works” to “What is happening here” to the even worse “Oh well, WHY is this happening”. So I set a goal: I would consider I had learned C well enough only after I mastered over 90% of the comp.lang.c FAQ. It was a tough task, but in the end it helped me winning a local Obscure C Contest =)
6. Before I was 10 years old, I had to do a simple surgery because of a Trigger Finger, a little problem in one tendon of a finger. I was told this was due to much computer and video-game playing (wikipedia denies), but I guess I loved the concept at the time.
7. I love web culture and watching how the web evolves and the people evolve with it. Things like data visualization, Twitter, new media, they all drive me crazy (specially when combined!). I was a very active member of an online art community for long years, even without the least ability for art! I don’t participate in general forums anymore, but I believe they are a very important part of the web, both of what it is and where it goes.

So, that’s it! My 7 nerd facts. I tried to keep then non-nerdy, but apparently failed. Anyway, it was very fun to make the list, trying to remember some funny facts from old and recent times to write!

I won’t tag anyone, because I guess that by now most of the people here on the planet (ha ha) have already been tagged, but there are some interesting ones who haven’t answered the meme yet, so keep the cool facts coming!

I just can’t describe how happy I am now! My newest extension, Tagmarks, which I created and submitted to the Extend Firefox 3 contest, is one of the winners of the contest!

Tagmarks

The news about the addon and the contest are all over the place! Some days ago, when I was contacted about being among the finalists and submitting it to Addons.mozilla.org, I submitted it. Quickly, just by being on the “Newest Add-ons” listing, some blog posts and e-mails and comments started popping up. It appeared on Firefox Facts, Cybernet News and Lifehacker! This was all so unexpected and made me very happy and by that time I realized that the users really liked the idea behind the extension, and wanted to use it! Many people had lots of nice suggestions about it, and I’m working on many of these suggestions. I’ve got to say thanks to everyone who used their time to write nice words and great suggestions for it!

I’ve also got to say thanks a lot for many great friends who have encouraged me for participating in the contest and sending me the congrats about the results! Marcio, Fabricio, Marek, Matheus, Filípi, and all the dudes from college!

I’ll soon be posting about the future ideas I’m planning for the extension, and the UIs and interactions I’m working on to implement the new suggestions!