I am a little late with the Q3 check-up – sorry!
This review of Android performance measurements covers the period July 1 – September 30: the third quarter of 2015.
– tcheck2 noisy – bug 1213032
– most test results are fairly steady over the quarter
This section tracks Perfomatic graphs from https://wiki.mozilla.org/Buildbot/Talos for background on Talos.for mozilla-central builds of Firefox for Android, for Talos tests run on Android 4.0 Opt. The test names shown are those used on treeherder. See
We intend to retire the remaining Android Talos tests, migrating these tests to autophone in the near future.
Measure of “checkerboarding” during simulation of real user interaction with page. Lower values are better.
20 (start of period) – 4 (end of period)
There is significant noise in recent results – bug 1213032 – but overall there is much less checkerboarding.
An svg-only number that measures SVG rendering performance. About half of the tests are animations or iterations of rendering. This ASAP test (tsvgx) iterates in unlimited frame-rate mode thus reflecting the maximum rendering throughput of each test. The reported value is the page load time, or, for animations/iterations – overall duration the sequence/animation took to complete. Lower values are better.
680 (start of period) – 730 (end of period)
Minor regression but fairly consistent results over the period.
Generic page load test. Lower values are better.
670 (start of period) – 670 (end of period)
Very steady results over the period.
Throbber Start / Throbber Stop
These graphs are taken from http://phonedash.mozilla.org. Browser startup performance is measured on real phones (a variety of popular devices).
Slight improvement in time to throbber start.
Slight regression in time to throbber stop, especially for nexus-s-4.
Android tests are no longer run on Eideticker.
These graphs are taken from the mozbench dashboard at http://ouija.allizom.org/grafana/index.html#/dashboard/file/mozbench.json which includes some comparisons involving Firefox for Android. More info at https://wiki.mozilla.org/Auto-tools/Projects/Mozbench.