Tags

,

I am a little late with the Q3 check-up – sorry!

This review of Android performance measurements covers the period July 1 – September 30: the third quarter of 2015.

Highlights:

– tcheck2 noisy – bug 1213032

– most test results are fairly steady over the quarter

Talos

This section tracks Perfomatic graphs from graphs.mozilla.org for mozilla-central builds of Firefox for Android, for Talos tests run on Android 4.0 Opt. The test names shown are those used on treeherder. See https://wiki.mozilla.org/Buildbot/Talos for background on Talos.

We intend to retire the remaining Android Talos tests, migrating these tests to autophone in the near future.

tcheck2

Measure of “checkerboarding” during simulation of real user interaction with page. Lower values are better.

check2

20 (start of period) – 4 (end of period)

There is significant noise in recent results – bug 1213032 – but overall there is much less checkerboarding.

tsvgx

An svg-only number that measures SVG rendering performance. About half of the tests are animations or iterations of rendering. This ASAP test (tsvgx) iterates in unlimited frame-rate mode thus reflecting the maximum rendering throughput of each test. The reported value is the page load time, or, for animations/iterations – overall duration the sequence/animation took to complete. Lower values are better.

svg

680 (start of period) – 730 (end of period)

Minor regression but fairly consistent results over the period.

tp4m

Generic page load test. Lower values are better.

tp4

670 (start of period) – 670 (end of period)

Very steady results over the period.

Throbber Start / Throbber Stop

These graphs are taken from http://phonedash.mozilla.org.  Browser startup performance is measured on real phones (a variety of popular devices).

throbstart

Slight improvement in time to throbber start.

throbstop

Slight regression in time to throbber stop, especially for nexus-s-4.

Eideticker

Android tests are no longer run on Eideticker.

mozbench

These graphs are taken from the mozbench dashboard at http://ouija.allizom.org/grafana/index.html#/dashboard/file/mozbench.json which includes some comparisons involving Firefox for Android. More info at https://wiki.mozilla.org/Auto-tools/Projects/Mozbench.

bench1

bench2

bench3

bench4

Advertisements