Posted on

Zippity – Using the Crowd to Collect Performance Data

If you’re serious about improving your application’s performance, you need to collect data. At Mozilla, we use Talos. The Mobile team has been using Talos to track performance on Maemo (using N810 and N900) devices for a few years and recently started tracking Android (using Tegras) as well. Here are a few of the metrics we track:

  • Startup (Ts)
  • Pageload (Tp)
  • HTML Rendering (Tdhtml)
  • Sunspider (Tss)

We build the application every time code is checked into the source repository. After each build, we run these performance tests and watch for changes. Regressions happen and need to be fixed as soon as possible.

One limitation of the current Talos system is the hardware variety. For Firefox Mobile, we run tests on two (2) types of hardware: N900 (Maemo) and NVidia Tegra (Android). One problem is that this doesn’t map to the real world very well. Don’t get me wrong – the current system minimizes measurement noise and we can see regressions fairly easily. However, without testing on a variety of devices, we just don’t get a good picture of how Firefox Mobile runs on your device.

Hello Zippity!

One idea that has talked about is to get the community involved in testing performance. If we could leverage many different people, using many different phones, we might get a better picture of how Firefox Mobile performs in the real world.

I created a simple add-on (Test Harness) that can install in any Firefox 4 for Mobile release and can run a series of tests. Then it posts the result to a public data server (Zippity) and anyone can view the results. When a test result is posted to Zippity, we capture a few bits of information:

  • Test Type: Pageload, startup – whatever the test might be tracking.
  • Test Manifest: What was the actual sequence performed? What series of pages did you load? This is important if we want to try to minimize noise in the results. If you want to only compare test runs that used the same input data, it can help reduce noise and make trends easier to see.
  • User Key: In case you want to only see results submitted by a particular user. The user key is completely optional. It can be used to help filter extraneous results.
  • Device Metadata: Information like device type, OS, OS version. Again, helpful for filtering.

I have been running Firefox Mobile nightly builds on several Android devices using a static set of webpages, hosted from a local webserver. Very controlled, to minimize noise and try to see small changes over time. You can see the results here.

Anyone can run the same set of live, real world pages using the add-on. Because of differences in devices, networking latency and changes to the actual website content – the results won’t be the same. That’s OK! The real world results are just as meaningful and provide just as much valuable insight.

Currently, the Test Harness add-on only sends pageload (Tp) data to Zippity, but plans are underway to extend the test types. If you want to play along, just install the Test Harness add-on. It is already configured to run the pre-defined pageset and submit the results to Zippity. Just push the button:

Zippity is still evolving and doesn’t have a lot of features. The source code should be in my Mercurial user repo soon. I’ll post more about Zippity and the Test Harness add-on as things improve.

Posted on

Comparing WebRunner and Adobe AIR

Okay, maybe my tongue-in-cheek, roasting of Ryan Stewart didn’t go over so well. But, at one point in the post/rant I mention working on a project called WebRunner that, IMO, is a competitor to Adobe AIR. I, unsurprisingly, feel that WebRunner does a better job complementing the Web and is a better fit for web applications in general. Both systems allow developers to run rich internet applications (defined as you wish) on the desktop. That’s in itself is both novel and interesting to many developers. However, I think WebRunner and AIR are suited to slightly different applications.

I see Adobe AIR as a desktop runtime for Flash applications. This is a nice boon to Flash developers who have long waited to escape the browser as a runtime. Although AIR supports running DHTML/AJAX/Ajax style web applications through its embedded WebKit renderer, this will likely be overshadowed by the Flash-based applications. Of course, this is just me talking, but most “applications” I have seen running in AIR are Flash-based.

WebRunner, on the other hand, is completely built around a browser – the same browser found in Firefox. Any existing web application should operate in WebRunner without any changes. Create an instant desktop version of your web application. Of course, its not really a desktop version – its just running in a Mozilla browser without any traditional browser chrome. WebRunner doesn’t change how you build web applications. It doesn’t impose a new stack of technologies, it works with the Web.

WebRunner does have the ability to add “features” to web applications such as:

  • Support for simple desktop integrations like displaying alert popups, registering as content handlers for local files, and access to the file system.
  • Greasemonkey-like application scripting to tweak the UI, access the simple desktop integrations or add offline features, not already built into the web application.

Its important to point out that these “features” are completely optional and can be added in a very mashup-friendly manner that most web developers and power users are already using.

There is definitely more to come from WebRunner.

Posted on

Firefox 3 – Offline App Demo

Firefox 3’s offline capabilities have been getting some attention lately: Chris Double’s post on porting Zimbra to use offline shows-off the potential. Robert O’Callahan and John Resig give details on the different pieces of the capabilities, namely: Offline Cache, Offline Events and DOMStorage.

A Simple Demo – Task Helper

I put together a simple offline application sample to help illustrate how the features can work. Call it – explanation through code. The application is a simple task list system. The current functionality includes:

  • Add tasks – Enter text in field and press ‘Add’
  • Complete tasks – Mark completed tasks and press ‘Complete’
  • Remove tasks – Mark tasks and press ‘Remove’
  • Data is stored as JSON and XHR is used to interact with server (PHP)

taskhelper.png

Nothing fancy. However, the application is “offline-aware”, meaning:

  • Application resources (HTML & JS) are tagged as offline cacheable
  • Before interacting with server, online status is checked. If online, use XHR. If offline, use DOMStorage
  • Online/Offline status is monitored using events. When the application switches form offline to online, data is resync’ed with the server

There is really nothing fancy about the offline stuff either, but putting it all together does make for a neat application. Using the latest Firefox 3 alpha, you should be able to:

  • Use the application while online.
  • Go offline using the “Work Offline” menu. The event log should show that the app went offline
  • Continue to use the application while offline
  • Go online using the “Work Offline” menu. The event log should show that the app is back online and has updated the server with any offline changes

If you were using a version of Firefox 3 with the offline cache patch (bug 367447) you would be able to a little more:

  • Use the application while online.
  • Go offline using the “Work Offline” menu. The event log should show that the app went offline
  • Continue to use the application while offline
  • Restart browser
  • Switch to offline (or have no network connection)
  • Enter URL to app (or use a bookmark)
  • Start using app with data from last session
  • Switching to online will update the server with offline data

Currently, the offline events only fire when offline is toggled via the “Work Offline” menu. Getting the events to fire by watching the network connection is the goal and would make this much more unobtrusive for the user.

Here is the source to the task helper application: taskhelper-src.zip

Posted on

XUL Validator – Beta

I put together a simple XUL validator based on the ideas in my last post. There is a primitive front end where you can put XUL into a textarea and validate it. The validator is completely client-side JavaScript with a simple JSON rules file. You can view the source and the rules from the validator “Tests” page.

Currently, the validator checks the structure and semantics of the XUL using the rules given in the last post, with caveats I’ll discuss in a moment. The validator only tests for a subset of the best practices checks. The current tests are listed on the validator “Tests” page. I’ll be adding the rest of the best practice checks soon.

Now, the caveats for the structure checks. There are very few “hard-fast” rules when it comes to XUL. Elements can appear nearly anywhere. Elements can contain any one-off attributes, even made-up ones, which can be useful for CSS selectors. Attribute events, such as “onclick”, can appear on parent elements because events can bubble. The XUL validator, tries to apply some logical warnings and checks, primarily based on the XUL Reference and Tutorial, as well as what I have seen in the Firefox/Thunderbird source code itself.

The current logic is definitely open for discussion and revision! Feedback appreciated.

Update: I made some fixes to the code/rules based on feedback. I also added a small set of options so user can turn off types of checks.

Posted on

UI Frameworks Are Pigs

Why are UI frameworks pigs? Because they don’t love you back.

You know the story: You meet this new UI framework. It seems so fresh and exciting. You start spending more time together, just doing small stuff. Things are so easy, not forced or boring like with your previous frameworks. So what if it acts a little immature. Before you know it, you’re writing specialized controls from scratch, embedding large amounts of business logic and enjoying every minute of it. You’re head over heels. Next thing you know, you’re crying yourself to sleep and listening to Barry Manilow.

Come on buddy! Snap out of it. UI frameworks don’t love you back. They don’t care about you. You need to watch out for yourself. Protect yourself. Isolate your code.

If your not ready to drop your current UI framework and switch to something else right now because it could take person-years to port, you have problems. If you have team members that love your current UI framework too much to want to change, you have problems.

They don’t love you back.

The experiences above are not about me, but I have this friend…

Posted on

OOD – Less Is More

My primary professional development language has been C++ for as long as I remember. It still is and I have no strong desire to move to something else. That also means that, many years ago, I embraced the Object Oriented Design (OOD) methodology.

I have derived Cat and Dog from Animal many times. At first, it took a while to see the object hierarchies, but it became easier. I loved object hierarchies. The deeper, the better.

Then, I started COM programming and designing using interfaces (abstract base classes) seemed to free my mind of the implementation details. I was happy. I was using interfaces for non-COM code in no time.

Then, I started using templates. First, with the help of the STL. What a concept! Later, with the help of Modern C++ Design (Andrei Alexandrescu) and the Boost libraries. Implementing design patterns using templates was amazing. Templates allowed me to do things I thought only inheritance could. I am glad I was wrong. Suddenly, my object hierarchies are not as deep.

Then, I started to learn XML and Services. Not really programming, I know, but the concept of storing all that data in a document that could be queried so easily made me question all those class getters and setters. Even some kinds of inheritance seemed overkill. Why can’t classes be more behavorial and less stateful? Let XML be the state and just pass it around.

In the end, I’m still using objects. Just less of them. I guess the data is becoming more important than the code.