It’s now well understood that, when designing for a touchscreen, there are certain minimum usable sizes for touchable targets. While the amount you can display on a screen is increasing with higher resolutions, human finger sizes aren’t changing, and fingertips are much larger than a mouse pointer. As a result, most UI recommendations for touch-target sizes on mobile devices range from around 7mm to 9mm.
It’s relatively easy to take these minimum sizes into account when designing your own interface. But what about when you’re displaying something that someone else has designed, and that wasn’t built with fingers in mind? This is the situation mobile browsers encounter in most web pages: links, fields, and buttons are often much smaller than you’d want them to be in order to tap on them, but making them bigger would interfere with the designer’s intended layout.
An approach that a number of touch-oriented OSes take is to make a small target’s tap-sensitive area larger, invisibly, than the visible target itself. This approach has been cleverly referred to as using “iceberg buttons” because the visible part of the target is much smaller than what’s lurking below. In fact, the iPhone does this with their keyboard as well, dynamically changing the invisible button target size based on what letters it predicts are most likely to come next.
Given how central link-tapping is to a browser, and how frustrating it is to tap the wrong link or not be able to tap at all, we decided to build our own approach.
Introducing SmartTap
In Firefox Mobile 1.1, we’ve added a smart-tapping scheme with the goal of allowing for accurate and easy tapping on links, form widgets and other focusable targets in web content. The main concepts of the approach are:
- Using a region, not just a point, to define the tap location
- Creating a list of focusable element candidates in the region of the tap
- Weight the elements by z-order
- Weight the elements by distance from the actual touch point
- Weight the links by number of visits
The result of the algorithm should be the element you were most likely trying to tap. Initial results show that tapping elements in Firefox Mobile 1.1 is much easy than previous versions. From a user’s perspective, taps just seem to work as they should.
Implementation Details
The code to support SmartTap was added to both the Mozilla platform and the Firefox front-end. It’s very flexible. The platform already exposes an elementFromPoint API to chrome and web content. The new API, nsIDOMWindowUtils.nodesFromRect, is very similar, but is only available to chrome content. For a given region, established by top/right/bottom/left, and a point, the method will return a list of possible DOM nodes.
The returned list is sorted in z-order. The Firefox front-end code then applies additional heuristics to the node list to find the most likely candidate. The code filters the nodes by “focusable” elements, weights by the distance of each node from touch point and, finally, weights visited links higher than other elements.
The region passed to nodesFromRect can be controlled via preferences (browser.ui.touch.top, browser.ui.touch.right, browser.ui.touch.bottom, browser.ui.touch.left). The weighting used for the visited links can also be adjusted using the browser.ui.touch.weight.visited preference.
Firefox Mobile uses a region that is offset more above the touch point. This has the affect of favoring elements above the touch point – which is based on our observations that people tend to “tap” below elements. Here’s a simple illustration:
click for larger image
The red dot is the actual touch point. The red box is the region passed to nodesFromRect. The title link will end up being “clicked” even though the author’s name text was actually “tapped.”
Another bit of intelligence in this system is based on the same insight that drives the Firefox awesomebar: that people tend to visit the same pages over and over again. Links are given higher weightings if they’ve been visited before, so visited links are more likely to “win” if a tap target is ambiguous because multiple small links are very close together. In practice, from the user’s perspective, tapping on the intended link just seems to work more often.
Thanks to Madhava Enros, Vivien Nicolas and Felipe Gomes for contributing to this post.
