ac_add_options ‐‐enable-e10s-compat

You’re now wondering why this is a compile-time flag and not a simple pref. This flag doesn’t do a whole lot, it mainly switches the main pref browser.tabs.remote to true, but it also switches off hardware acceleration and disables some code that partially break the UI with the pref on. The goal is to get rid of the flag relatively soon (and only use the pref), but for now it’s the easiest way to make sure everyone can focus on the important bits rather than having broken pieces getting in the way.

Things to know after you get the build

The first tab of each window is not remote. All the other tabs that you open are. Progress indicators, favicons, session history and titles don’t work yet (session history and titles are coming soon). Most input like mouse, mouse scroll, keyboard and focus works (although there are corner cases not covered yet). Ah, do not use it with your main profile! Results are unknown.

The content is hosted by the plugin-container process. You should be able to kill it and the content from the remote tabs will disappear (when Firefox is re-rendered, usually when you re-focus the window). You can then open new tabs and the process is recreated just fine.

With these patches in mozilla-central the goal is to make sure everyone can see the progress and test their new features with electrolysis on, and as importantly, do not break what currently already works.

The front-end tasks are being tracked by bug 653064, and platform bugs marked with [e10s] in the whiteboard. Those lists are in no way comprehensive, so do file bugs or ask around with important things that you notice are not tracked yet.

If you want to know more about the patches

On the platform side, the patches that landed added support to input events and were done in bug 583976. nsEventStateManager and nsFocusManager were made aware of the content separation, and when input/focus events are targeted at the content area (i.e. the <browser> element), they’re automatically forwarded (asynchronously) to the content process. On the front-end side, some code was changed to not assume that the content docShell exists, and messages are passed through the message manager when appropriate (e.g. loadURI). I’ll talk more about these details in other posts.

Next steps

Now that these base patches are in the tree, on the front-end side we’re working on making sure more operations work with electrolysis. Drew has got a neat logging for forbidden chrome->content operations in bug 666713, and the goal is to reduce as much as possible these warning for basic navigation tasks in Firefox, so that the noise can be reduced and we can tackle more features more efficiently. I’ve got more code ready (and some already reviewed) but which I can’t land right now because they’re triggering other [un-]related bugs. I’m already filing these bugs and asking around about them but expect me to start pinging more frequently to ask for help in figuring out and fixing bugs on broader areas of the code.

Aaapptabs does just that. Like the Add-ons manager, it hides the browser toolbars for all app tabs. The result is that, for Windows 7 with a maximized screen, the Firefox UI takes a grand total of 27 pixels in height.

Not unexpectedly, writing the add-on using the Jetpack Add-on SDK was a blast. The SDK is in 1.0 beta 4 and the existing APIs are really solid. The whole add-on code is less than 50 lines, it’s restartless, it deals correctly with being disabled/re-enabled. If you temporarily need to view the hidden toolbars, just go to the Add-ons manager and disable the add-on. Changes take effect instantly.

Install it from AMO or view the source.

Hat tip: combine it with Easy app tabs to toggle them with a double click

This got me thinking on how we could use this code to create tests to measure animation speeds and make sure that they don’t regress. One important goal that we have coming soon is to ensure that our UI is always responsive and animations are smooth, and that will certainly need some kind of testing framework.

So I set out on a little weekend project to include the framerate monitor in our mochitest framework. The main goal was that it should be useful to measure both:

• stand-alone animations, like creating tests for various -moz-transition combinations and sequences;
• animations in the UI itself, in a way that every time we add a new animated feature to the UI we should be able to write a test to trigger it and profile its FPS

With that in mind, I went to the simple approach of including the monitor as part of  the SimpleTest, such that it can be used from mochitest-plain, mochitest-browser-chrome, etc.

I wrote two tests to begin with and see how it behaves. One is a simple test that triggers a bunch of -moz-transition properties on various elements. The other one came from an interesting observation that Patrick saw some weeks ago: one of the untold benefits of the new and gorgeous about:home is that it’s also faster to resize. Of course, we should never have an entry page that is slow to resize, so the test opens a new browser window and resizes it during some seconds. Currently, for simplicity’s sake, both tests get the average FPS during the process and fails if it was less than 30.

The current patch can be found in this hg queue. Also, for those interested in this area, these problems are also being tackled from different angles too: for example, Ted is working on adding event loop instrumentation, and already has some patches for Linux and Win.

First we started by using the test-ipc.xul to check how the platform support for e10s was working. This is a simple XUL window with a <browser remote=”true”> element and some test functions. Upon disabling hardware acceleration and making same-compartment checks non-fatal, the window stood up and displayed the webpage.

After that we went to get the regular Firefox browser window standing up. There were various points during the startup functions (BrowserStartup and delayedStartup) that threw exceptions and thus the functions never completed. Most of the cases are calls trying to access .contentDocument or .docShell properties, which won’t exist for content anymore and will have to be replaced for messages between the processes. After commenting out a few places and making small changes to others, those two functions completed and the UI began to work (for a very broad definition of work, of course).

The next major step was to set up a mechanism to pass events from chrome to content. After talking with smaug, we went for a mechanism tied to the EventStateManager that will automatically forward selected events to content if their target ends up being the <browser> element, and one that is hopefully easy to extend for each new event that we want to add later. This just involves adding them to a whitelist and writing some simple code to {de}serialize them at the IPC level.

With this mechanism in place we already hooked up mouse, mouse scroll and basic keyboard events, plus a kinda-broken focus (and a simple plan to try to hook something to the focus manager)

At this point, we have a Firefox window starting up where you can load up tabs and interact with the pages at a basic level. There’s not progress indicators or tab titles yet, which makes it sometimes hard to tell if things are working or not. Also, focus get lost often, specially if you try to close tabs, which is not working either .

In any case, a lot of people are curious to test it and to get a feeling of how things are working, so there are try builds available: OS X, Linux, and Windows.
Remember that all of this is only planned for future versions of Firefox. Use the builds with a testing profile and expect things to break!

Update: forgot to mention that you need to type full URLs, including the protocol:// part. Also, if you’re interested, these are the changesets pushed to this build, and I’m keeping the patches (plus some others) in a patch queue.

In the next blog posts I’ll talk more about some of the specific fixes so far, the attempts to run tests on these builds, and some of the approaches that will be needed to adapt the front-end to e10s.

How to use the APIs

// Some simple usage example:
document.addEventListener("MozTouchDown", function(event) {
  trackedFingers[event.streamId] = {
    X: event.clientX,
    Y: event.clientY
  }
}, false);

document.addEventListener("MozTouchMove", function(event) {
  //update X/Y positions, do something interesting
}, false);

document.addEventListener("MozTouchUp", function(event) {
  delete trackedFingers[event.streamId];
}, false);

Supporting API: .mozInputSource

There are two other supporting APIs. If you don’t need to deal with multiple input, but still want to handle touch or pen input differently, you can access the .mozInputSource property of regular mouse events. With this property, you can tell if that event was generated by a mouse, by pen or by touch input. This property is not only available on low-level mouse input events, but also on click, dblclick and drag&drop events.

document.addEventListener("click", function(event) {
alert(event.mozInputSource);
}, false);

Supporting API: CSS media query

@media all and (-moz-touch-enabled) {
  /* touch specific css goes here */
}

Code snippets and examples in the wild

How to get started on development

Feedback

By far, my favorite tool to work with JS is the extremely awesome (this is an understatement) JS Shell running from the Extension Developer add-on. So, what I wanted was to be able to get Jetpack’s context inside the shell.

To do this, I first created a new Firefox profile (called dev) and installed said add-on on it. Now, we need to run Jetpack using that profile. To run Jetpack on Firefox, except when running it from the test harness, you need to run it from a package with a main.js module. So, I created a simple package from the `cfx docs` tutorial and added a main.js module. Now, we can load up our Firefox profile from this package:

(jetpack-sdk)felipe: my-first-package$ cfx run -a firefox -P path/to/dev/profile

With Firefox loaded, Tools > Extension Developer > Javascript Shell gets our shell. Now, we need to get Jetpack’s context. For this, we have to import the Cuddlefish module that has the Loader, and then load both the jetpack-core and our package libs. The path to the resources depends on the package you’re running from, so considering our package is called “my-first-package”, we can do:

var a = {};
Components.utils.import("resource://my-first-package-jetpack-core-lib/cuddlefish.js", a);
[Object BackstagePass]
var jetpack = a.Loader({rootPaths:
["resource://my-first-package-jetpack-core-lib",
"resource://my-first-package-my-first-package-lib/"]
});

Done. Now the jetpack object have jetpack’s main functionality, and you can now require your modules from it. Let’s see it using the example module from the tutorial, which have an add function:

jetpack.console.log("Hello World"); //message was printed at the console
var mymodule = jetpack.require("my-module");
mymodule.add
function add(a, b) {
  return a + b;
}
mymodule.add(1, 2)
3

And that’s it! Thanks to Drew for giving the ideas on how to make this work.

While we’re at it, there’s another thing that I’m still trying to understand. It looks to me that there’s just a thin line between developing an “add-on using the jetpack sdk” and “new modules for the jetpack core sdk”. Is that true? My current plan for developing these modules goes like this: develop it into an indepedent package, (so I can cfx run from that package and have some dummy consumers on main.js to help development), and after it’s finished, I move the module-name.js and tests to jetpack-core and get rid of my original package. Does this sound about right?

Let’s start with the problem: I was writing some set of tests which were highly dependent on triggering actions and waiting events propagation to continue. This usually involves a lot of callbacks from short timeouts or event listeners, and you end up with code like this: [semi-pseudocode]

function goOn() {
  doThing3();
  setTimeout(finishUp, 10);
}

function test() {
  doThing1();
  setTimeout(function() {
    addEventListener("interesting-event", goOn, false);
    doThing2();
  }, 10);
}

Which is the modern day equivalent of spreading gotos everywhere on your code (“oh, the horror!”), and makes it very hard to follow the intended logic, specially if you didn’t aptly name your functions with numbers to help you out.

To improve that, we can avoid nesting all of the calls and write a sequential function, and use the yield keyword to exit the function at one point and then continue running it from that point on. It’s like a set of sleep/wakeUp calls, and all the wakeUp control won’t be so mixed with our logic. So we would write as follows:

function wakeUp() {
  testGen.next();
}
function test() {
   doThing1();
   setTimeout(wakeUp, 10); yield;
   doThing2();
   addEventListener("interesting-event", wakeUp, false); yield;
   doThing3();
   setTimeout(wakeUp, 10); yield;
   finishUp();
}

Cleaner, isn’t it? Oh, and just a note, when you use this make sure you don’t end up subtly changing the logic of your calls, unlike “a friend of mine” who did this and broke the tests he had written.

• Bug 547997 – Perform hit target detection/correction on tap clicks
• Bug 547996 – Be able to tell when a click was generated by touch
• Bug 548005 – Browser zoom should stay centered when coordinates are available
• Bug 548012 – Browser zoom should be cancelable to be handled by webpage

And there are some other already existing bugs so there’s now the meta bug 548100 to keep track of them all.

On the DOM Events API front, on the previous week I unbitrotted the experimental patch to apply to current trunk. And at the moment I’m investigating the Windows 7 API to understand how to deal with the limitation that you can either get raw touch input or touch gestures data, but not both at the same time.

My next goals is to work on the bugs to improve the user experience and to keep moving with the events api making sure to coordinate this work with mobile’s interests too.

On one version, they increased the font-size of the whole page, similarly to how users with sight difficulties might increase the page zoom until they’re able to read the content. The second style featured the regular page but with mouse interactions disabled, so you can only navigate on the page using the arrows and the Tab key, as do users who are not able to handle the mouse cursor effectively. The third one featured a blacked out page where only one element is visible at a time, and again you navigate through the page using the Tab key. This experience relates to the interaction that screen readers’ users are used to.

These pages, beyond being examples of the different interactions that users might face on the web, are a great incentive for web designers to start testing what the experience for such users are on their own sites. Usability has fortunately become a popular trend recently, where more designers and web developers are aware of its importance. A recent Jakob Nielsen’s article stated that Anybody can do Usability [tests], where he says that with simple tasks and minimal testing, many usability problems can be detected (and hopefully improved) on a web page. Here’s to hope that the same trend follows to accessibility.

I asked Willie Walker, who I’ve met at JRSL Chile and works with Accessibility on Gnome, what he thought about this campaign, and he said it would be nice if there was some sort of tool that could help web developers to make these tests on their websites. This would indeed be cool. For the first example, you only need to hit Ctrl++ a few times and it’s done. For the other ones, you can work a few modifications to your webpage to get the desired effect, or… or you can have it all automatically done by these two bookmarklets that I prepared! Try it on your own page or your favorite page and see how well it fares. These bookmarklets are by no means perfect, but they should be a good starting point of inspiration. Some things to consider when doing these tests:

• Zoomed page: Hit Ctrl++ and see how well your layout stands. How many times can you zoom it without breaking everything? Try reducing the window size too.
• Mouse disabled: How less obvious does your links and navigation become without any mouse hover? Does your CSS menus break? How is the order of navigation on your links via the Tab key? Hint: you can have a good ordering of elements  via a well-thought mark-up, but you can also explicit change them using the tabindex attribute.
• Screen readers: How well is your content organized? Is the hierarchy too deep and confusing? Does the most important content on your page comes first?

Accessibility bookmarklets

Alix pointed me to the contribute page, and there’s this page from the mozilla.com website, but I also wanted to list the timely events going on, because they’re hard to keep track if you don’t follow all the news pretty closely, and my friend Marcio commented that he believes that these kind of events have a great engaging component, which I absolutely agree: the challenge of a deadline together with a specific goal is a great motivation to get started.

So here’s a list of what I know is currently going on, bonus added: the target audience, which is useful in helping the new members themselves to find their way in the community and their interests.

• Extend Firefox 3.5 Contest (developers)
  October 2nd
• Jetpack Contest (web developers)
  October 15th
• Fennec call for addons (developers)
  Exciting times ahead for Fennec
• Fall Design Challenge (power users, usability/ux students, infoviz lovers)
  Last week of September
• Firefox Goes Mobile (designers, artists)
  September 30th
• Mozilla Service week (everybody!)
  Do something today or tomorrow!
• Love the web? Show the love and celebrate it (everybody!)
  September 22nd

Some of these are close to their deadlines, but the list is not intended to be a final resource, just a beginning Listing all of these helps put on perspective how many cool and interesting things are going on.

There are other activities which are not time sensitive but are relatively new engaging points worth mentioning: Creating Ubiquity commands or new Personas, participating in the Creative Collective and the Test Pilot. (Or our great Nightly testers)

Hope this comes in handy the next time you want to help a new community member seeking advice on how to be part of the Mozilla world.