Things to know after you get the build

The last remaining patch to get Firefox to open and stand up with electrolysis enabled was landed late last week, and, after a few days baking, it has stuck! This means that anyone with a mozilla-central tree can now compile and test Firefox with the content running in a separated process, and start trying your own features and see how well they work or not (or if you can even get to them in the first place). To do this you just need to add one extra configure flag to your mozconfig:

ac_add_options ‐‐enable-e10s-compat

You’re now wondering why this is a compile-time flag and not a simple pref. This flag doesn’t do a whole lot, it mainly switches the main pref browser.tabs.remote to true, but it also switches off hardware acceleration and disables some code that partially break the UI with the pref on. The goal is to get rid of the flag relatively soon (and only use the pref), but for now it’s the easiest way to make sure everyone can focus on the important bits rather than having broken pieces getting in the way.

The first tab of each window is not remote. All the other tabs that you open are. Progress indicators, favicons, session history and titles don’t work yet (session history and titles are coming soon). Most input like mouse, mouse scroll, keyboard and focus works (although there are corner cases not covered yet). Ah, do not use it with your main profile! Results are unknown.

The content is hosted by the plugin-container process. You should be able to kill it and the content from the remote tabs will disappear (when Firefox is re-rendered, usually when you re-focus the window). You can then open new tabs and the process is recreated just fine.

With these patches in mozilla-central the goal is to make sure everyone can see the progress and test their new features with electrolysis on, and as importantly, do not break what currently already works.

The front-end tasks are being tracked by bug 653064, and platform bugs marked with [e10s] in the whiteboard. Those lists are in no way comprehensive, so do file bugs or ask around with important things that you notice are not tracked yet.

If you want to know more about the patches

On the platform side, the patches that landed added support to input events and were done in bug 583976. nsEventStateManager and nsFocusManager were made aware of the content separation, and when input/focus events are targeted at the content area (i.e. the <browser> element), they’re automatically forwarded (asynchronously) to the content process. On the front-end side, some code was changed to not assume that the content docShell exists, and messages are passed through the message manager when appropriate (e.g. loadURI). I’ll talk more about these details in other posts.

Next steps

Now that these base patches are in the tree, on the front-end side we’re working on making sure more operations work with electrolysis. Drew has got a neat logging for forbidden chrome->content operations in bug 666713, and the goal is to reduce as much as possible these warning for basic navigation tasks in Firefox, so that the noise can be reduced and we can tackle more features more efficiently. I’ve got more code ready (and some already reviewed) but which I can’t land right now because they’re triggering other [un-]related bugs. I’m already filing these bugs and asking around about them but expect me to start pinging more frequently to ask for help in figuring out and fixing bugs on broader areas of the code.

I’ve been meaning to write a simple add-on for some time now, and today felt the perfect day to do it. I’ve noticed that all the sites that I use as app tabs are indeed web apps (or close enough); they manage their own navigation and URLs, such that I rarely need to use the navigation bar on those sites. So why not get even more screen real estate for them?

Aaapptabs does just that. Like the Add-ons manager, it hides the browser toolbars for all app tabs. The result is that, for Windows 7 with a maximized screen, the Firefox UI takes a grand total of 27 pixels in height.

Not unexpectedly, writing the add-on using the Jetpack Add-on SDK was a blast. The SDK is in 1.0 beta 4 and the existing APIs are really solid. The whole add-on code is less than 50 lines, it’s restartless, it deals correctly with being disabled/re-enabled. If you temporarily need to view the hidden toolbars, just go to the Add-ons manager and disable the add-on. Changes take effect instantly.

Install it from AMO or view the source.

Hat tip: combine it with Easy app tabs to toggle them with a double click

Patrick Walton has been demonstrating how useful some of the new animation APIs are with his framerate monitor add-on, which has already helped to understand some of our performance characteristics and to find various possible improvements for opportunistic wins in our rendering speed.

This got me thinking on how we could use this code to create tests to measure animation speeds and make sure that they don’t regress. One important goal that we have coming soon is to ensure that our UI is always responsive and animations are smooth, and that will certainly need some kind of testing framework.

So I set out on a little weekend project to include the framerate monitor in our mochitest framework. The main goal was that it should be useful to measure both:

• stand-alone animations, like creating tests for various -moz-transition combinations and sequences;
• animations in the UI itself, in a way that every time we add a new animated feature to the UI we should be able to write a test to trigger it and profile its FPS

With that in mind, I went to the simple approach of including the monitor as part of  the SimpleTest, such that it can be used from mochitest-plain, mochitest-browser-chrome, etc.

I wrote two tests to begin with and see how it behaves. One is a simple test that triggers a bunch of -moz-transition properties on various elements. The other one came from an interesting observation that Patrick saw some weeks ago: one of the untold benefits of the new and gorgeous about:home is that it’s also faster to resize. Of course, we should never have an entry page that is slow to resize, so the test opens a new browser window and resizes it during some seconds. Currently, for simplicity’s sake, both tests get the average FPS during the process and fails if it was less than 30.

The current patch can be found in this hg queue. Also, for those interested in this area, these problems are also being tackled from different angles too: for example, Ted is working on adding event loop instrumentation, and already has some patches for Linux and Win.

Last month we started working on a project to stand up a Firefox instance running e10s. Now we’ve got some first tryserver builds (links below) in which it’s possible to interact with the webpages, so if you’re interested I’m going to talk about how was the process to get to this point so far, and what are the next steps to do.

First we started by using the test-ipc.xul to check how the platform support for e10s was working. This is a simple XUL window with a <browser remote=”true”> element and some test functions. Upon disabling hardware acceleration and making same-compartment checks non-fatal, the window stood up and displayed the webpage.

After that we went to get the regular Firefox browser window standing up. There were various points during the startup functions (BrowserStartup and delayedStartup) that threw exceptions and thus the functions never completed. Most of the cases are calls trying to access .contentDocument or .docShell properties, which won’t exist for content anymore and will have to be replaced for messages between the processes. After commenting out a few places and making small changes to others, those two functions completed and the UI began to work (for a very broad definition of work, of course).

The next major step was to set up a mechanism to pass events from chrome to content. After talking with smaug, we went for a mechanism tied to the EventStateManager that will automatically forward selected events to content if their target ends up being the <browser> element, and one that is hopefully easy to extend for each new event that we want to add later. This just involves adding them to a whitelist and writing some simple code to {de}serialize them at the IPC level.

With this mechanism in place we already hooked up mouse, mouse scroll and basic keyboard events, plus a kinda-broken focus (and a simple plan to try to hook something to the focus manager)

At this point, we have a Firefox window starting up where you can load up tabs and interact with the pages at a basic level. There’s not progress indicators or tab titles yet, which makes it sometimes hard to tell if things are working or not. Also, focus get lost often, specially if you try to close tabs, which is not working either .

In any case, a lot of people are curious to test it and to get a feeling of how things are working, so there are try builds available: OS X, Linux, and Windows.
Remember that all of this is only planned for future versions of Firefox. Use the builds with a testing profile and expect things to break!

Update: forgot to mention that you need to type full URLs, including the protocol:// part. Also, if you’re interested, these are the changesets pushed to this build, and I’m keeping the patches (plus some others) in a patch queue.

In the next blog posts I’ll talk more about some of the specific fixes so far, the attempts to run tests on these builds, and some of the approaches that will be needed to adapt the front-end to e10s.

How to use the APIs

The updated Firefox 4 Beta was released today, and one the new features in this beta is the multitouch API that we’ve been working on for the past months. This means that regular webpages are capable of using  multitouch input, allowing webdevelopers  to create new interactive experiences for their websites. You can read all about it and see a really cool demo of what’s possible in Paul Rouget’s article on hacks.mozilla.org.

On this post I’ll give a broader overview of the APIs, samples and development methods.

The multitouch API is very simple and meant to get web developers started quickly. It works through the use of DOM Events, similarly to how other kinds of input work (e.g. mouse, keyboard). It consists of three different events: MozTouchDown, MozTouchMove, MozTouchUp, which are very similar to the mouse events already familiar to web developers.

They have all the common properties from mouse events, plus an extra field called streamId, which is a random number that is an unique identifier for each point of contact, and this number is kept constant while you don’t lift that finger from the screen. It makes it possible to accurately track the movement of each finger without mixing them.

// Some simple usage example:
document.addEventListener("MozTouchDown", function(event) {
  trackedFingers[event.streamId] = {
    X: event.clientX,
    Y: event.clientY
  }
}, false);

document.addEventListener("MozTouchMove", function(event) {
  //update X/Y positions, do something interesting
}, false);

document.addEventListener("MozTouchUp", function(event) {
  delete trackedFingers[event.streamId];
}, false);

Supporting API: .mozInputSource

There are two other supporting APIs. If you don’t need to deal with multiple input, but still want to handle touch or pen input differently, you can access the .mozInputSource property of regular mouse events. With this property, you can tell if that event was generated by a mouse, by pen or by touch input. This property is not only available on low-level mouse input events, but also on click, dblclick and drag&drop events.

document.addEventListener("click", function(event) {
alert(event.mozInputSource);
}, false);

Supporting API: CSS media query

If you want to make style changes for your webpage when being accessed using a touch-enabled device, you can use the -moz-touch-enabled CSS media query and add your custom style inside that selector

@media all and (-moz-touch-enabled) {
  /* touch specific css goes here */
}

Code snippets and examples in the wild

While the APIs were being developed, there was a handful of demos created to test the implementation and to highlight some interesting use cases for touch. You can see some of those demos here (video and source code):

Other people have been trying the work in progress and built more examples. Al MacDonald has a very nice generative music mash-up of multitouch with the experimental Audio data API.

How to get started on development

There are various ways to get started developing for multitouch, depending on the equipment and OS that you use. At the moment, we support devices that are compatible with the Windows 7 multitouch input, such as various models of notebooks and touchscreen-monitors.

However, it’s possible to get homemade devices that uses the TUIO protocol to work, using projects such as MultitouchVista. It features a set of drivers that gets the input from various sources, such as homemade touch tables, multiple mice, among others, and outputs data to the Windows’ standard APIs.

If you have a spare webcam and a free weekend for a DIY project you might be able to build one yourself following various good tutorials that exist.

In addition to that, Hernán Colmeiro has created a multitouch simulator for his Google Summer of Code project. This is a platform-independent Firefox add-on that is able to replay multitouch input from hand-written or recorded JSON files. This add-on is built using Jetpack, and you don’t even need to restart Firefox to install it. You can use it to develop and test a webpage for multitouch input and your code doesn’t need any special handling for the add-on; it works as is. You can read more about it, watch a screencast of how it works and see a standalone demo of its playback feature over at Hernan’s blog.

Feedback

We hope this API is useful to get this feature in the hands of web developers. We will soon be starting dicussions to see how to standardize these APIs at the W3C.  Seeing what people will create with it and getting feedback will be invaluable to make sure that the standard is useful for the web developers.

On the last couple of days I’ve started playing with the new Jetpack SDK, and I’m still working on understanding the main concepts of its development environment. I’m interested to find a good workflow while working on Jetpack (and to know how others are doing it), and since more people might be wondering the same, I’ll blog here about the dev env that I started building for me.

By far, my favorite tool to work with JS is the extremely awesome (this is an understatement) JS Shell running from the Extension Developer add-on. So, what I wanted was to be able to get Jetpack’s context inside the shell.

To do this, I first created a new Firefox profile (called dev) and installed said add-on on it. Now, we need to run Jetpack using that profile. To run Jetpack on Firefox, except when running it from the test harness, you need to run it from a package with a main.js module. So, I created a simple package from the `cfx docs` tutorial and added a main.js module. Now, we can load up our Firefox profile from this package:

(jetpack-sdk)felipe: my-first-package$ cfx run -a firefox -P path/to/dev/profile

With Firefox loaded, Tools > Extension Developer > Javascript Shell gets our shell. Now, we need to get Jetpack’s context. For this, we have to import the Cuddlefish module that has the Loader, and then load both the jetpack-core and our package libs. The path to the resources depends on the package you’re running from, so considering our package is called “my-first-package”, we can do:

var a = {};
Components.utils.import("resource://my-first-package-jetpack-core-lib/cuddlefish.js", a);
[Object BackstagePass]
var jetpack = a.Loader({rootPaths:
["resource://my-first-package-jetpack-core-lib",
"resource://my-first-package-my-first-package-lib/"]
});

Done. Now the jetpack object have jetpack’s main functionality, and you can now require your modules from it. Let’s see it using the example module from the tutorial, which have an add function:

jetpack.console.log("Hello World"); //message was printed at the console
var mymodule = jetpack.require("my-module");
mymodule.add
function add(a, b) {
  return a + b;
}
mymodule.add(1, 2)
3

And that’s it! Thanks to Drew for giving the ideas on how to make this work.

While we’re at it, there’s another thing that I’m still trying to understand. It looks to me that there’s just a thin line between developing an “add-on using the jetpack sdk” and “new modules for the jetpack core sdk”. Is that true? My current plan for developing these modules goes like this: develop it into an indepedent package, (so I can cfx run from that package and have some dummy consumers on main.js to help development), and after it’s finished, I move the module-name.js and tests to jetpack-core and get rid of my original package. Does this sound about right?

I don’t know about you, but I’m always amazed every time I see some clever usage of generator functions (the yield keyword). From things like speeding up your UI, to implementing thread pools, or computing prime numbers and infinite sequences, they’re always interesting. Last week, Shawn showed me a cool async pattern using generators which I really liked so I decided to blog about it.

Let’s start with the problem: I was writing some set of tests which were highly dependent on triggering actions and waiting events propagation to continue. This usually involves a lot of callbacks from short timeouts or event listeners, and you end up with code like this: [semi-pseudocode]

function goOn() {
  doThing3();
  setTimeout(finishUp, 10);
}

function test() {
  doThing1();
  setTimeout(function() {
    addEventListener("interesting-event", goOn, false);
    doThing2();
  }, 10);
}

Which is the modern day equivalent of spreading gotos everywhere on your code (“oh, the horror!”), and makes it very hard to follow the intended logic, specially if you didn’t aptly name your functions with numbers to help you out.

To improve that, we can avoid nesting all of the calls and write a sequential function, and use the yield keyword to exit the function at one point and then continue running it from that point on. It’s like a set of sleep/wakeUp calls, and all the wakeUp control won’t be so mixed with our logic. So we would write as follows:

function wakeUp() {
  testGen.next();
}
function test() {
   doThing1();
   setTimeout(wakeUp, 10); yield;
   doThing2();
   addEventListener("interesting-event", wakeUp, false); yield;
   doThing3();
   setTimeout(wakeUp, 10); yield;
   finishUp();
}

Cleaner, isn’t it? Oh, and just a note, when you use this make sure you don’t end up subtly changing the logic of your calls, unlike “a friend of mine” who did this and broke the tests he had written.

Here are this week’s news about our project to support multitouch screens. Besides moving forward on an events API, last week I filed some bugs that aim to improve the overall experience for touchscreens devices (single and multitouch) when surfing on the web. They are:

• Bug 547997 – Perform hit target detection/correction on tap clicks
• Bug 547996 – Be able to tell when a click was generated by touch
• Bug 548005 – Browser zoom should stay centered when coordinates are available
• Bug 548012 – Browser zoom should be cancelable to be handled by webpage

And there are some other already existing bugs so there’s now the meta bug 548100 to keep track of them all.

On the DOM Events API front, on the previous week I unbitrotted the experimental patch to apply to current trunk. And at the moment I’m investigating the Windows 7 API to understand how to deal with the limitation that you can either get raw touch input or touch gestures data, but not both at the same time.

My next goals is to work on the bugs to improve the user experience and to keep moving with the events api making sure to coordinate this work with mobile’s interests too.

Last month, the recently opened brazilian W3C office made an interesting campaign on the UN International Day of Disabled Persons to generate awareness of the importance that is accessibility on the web. During that day, they modified their website in 3 different ways in order to try to simulate how people with certain accessibility needs view and interact with the web.

On one version, they increased the font-size of the whole page, similarly to how users with sight difficulties might increase the page zoom until they’re able to read the content. The second style featured the regular page but with mouse interactions disabled, so you can only navigate on the page using the arrows and the Tab key, as do users who are not able to handle the mouse cursor effectively. The third one featured a blacked out page where only one element is visible at a time, and again you navigate through the page using the Tab key. This experience relates to the interaction that screen readers’ users are used to.

These pages, beyond being examples of the different interactions that users might face on the web, are a great incentive for web designers to start testing what the experience for such users are on their own sites. Usability has fortunately become a popular trend recently, where more designers and web developers are aware of its importance. A recent Jakob Nielsen’s article stated that Anybody can do Usability [tests], where he says that with simple tasks and minimal testing, many usability problems can be detected (and hopefully improved) on a web page. Here’s to hope that the same trend follows to accessibility.

I asked Willie Walker, who I’ve met at JRSL Chile and works with Accessibility on Gnome, what he thought about this campaign, and he said it would be nice if there was some sort of tool that could help web developers to make these tests on their websites. This would indeed be cool. For the first example, you only need to hit Ctrl++ a few times and it’s done. For the other ones, you can work a few modifications to your webpage to get the desired effect, or… or you can have it all automatically done by these two bookmarklets that I prepared! Try it on your own page or your favorite page and see how well it fares. These bookmarklets are by no means perfect, but they should be a good starting point of inspiration. Some things to consider when doing these tests:

• Zoomed page: Hit Ctrl++ and see how well your layout stands. How many times can you zoom it without breaking everything? Try reducing the window size too.
• Mouse disabled: How less obvious does your links and navigation become without any mouse hover? Does your CSS menus break? How is the order of navigation on your links via the Tab key? Hint: you can have a good ordering of elements  via a well-thought mark-up, but you can also explicit change them using the tabindex attribute.
• Screen readers: How well is your content organized? Is the hierarchy too deep and confusing? Does the most important content on your page comes first?

Accessibility bookmarklets

Having noticed that Mozilla is at a moment where there are quite a number of different opportunities to contribute and get involved, for users with all kinds of skills and interests (app developers, web developers, artists, designers), I set out to compile a list for the Mozilla Brazil community.

Alix pointed me to the contribute page, and there’s this page from the mozilla.com website, but I also wanted to list the timely events going on, because they’re hard to keep track if you don’t follow all the news pretty closely, and my friend Marcio commented that he believes that these kind of events have a great engaging component, which I absolutely agree: the challenge of a deadline together with a specific goal is a great motivation to get started.

So here’s a list of what I know is currently going on, bonus added: the target audience, which is useful in helping the new members themselves to find their way in the community and their interests.

• Extend Firefox 3.5 Contest (developers)
  October 2nd
• Jetpack Contest (web developers)
  October 15th
• Fennec call for addons (developers)
  Exciting times ahead for Fennec
• Fall Design Challenge (power users, usability/ux students, infoviz lovers)
  Last week of September
• Firefox Goes Mobile (designers, artists)
  September 30th
• Mozilla Service week (everybody!)
  Do something today or tomorrow!
• Love the web? Show the love and celebrate it (everybody!)
  September 22nd

Some of these are close to their deadlines, but the list is not intended to be a final resource, just a beginning Listing all of these helps put on perspective how many cool and interesting things are going on.

There are other activities which are not time sensitive but are relatively new engaging points worth mentioning: Creating Ubiquity commands or new Personas, participating in the Creative Collective and the Test Pilot. (Or our great Nightly testers)

Hope this comes in handy the next time you want to help a new community member seeking advice on how to be part of the Mozilla world.