With almost every build of Firefox for Android, we run many performance tests and take numerous performance measurements. Big performance regressions – or big improvements – are easily noticed, but smaller changes and trends over time are harder to track. We have lots of data, we even have graphs of the data, but still it seems there is too much to take in — and too much to keep up with over time.
Here is my attempt to summarize performance measurements for Firefox for Android, for August 2012. If there is interest, I will put together a similar summary each month.
Talos
This section shows Perfomatic graphs from graphs.mozilla.org for mozilla-central builds of Native Fennec (Android opt). The test names shown are those used on tbpl. See https://wiki.mozilla.org/Buildbot/Talos for background on Talos.
Regrettably, some of these graphs are nearly impossible to read because the scale of the y-axis is completely inappropriate; we’re working on that and hope to see this problem resolved in time for next month’s summary.
tdhtml_nochrome
1236.29 (start of month) – 914.647 (end of month).
tcheckerboard
69.965 (start of month) – 221.62 (end of month)
tcheck2
2757.46 (start of month) – 1435.16 (end of month)
tcheck3
2576.33 (start of month) – 1860.86 (end of month)
trobopan
34897.2 (start of month) – 37372.2 (end of month)
tprovider
361.5 (start of month) – 366.25 (end of month)
tsvg_nochrome
7123.45 (start of month) – 7042.73 (end of month)
tp4m_nochrome
792.175 (start of month) – 739.475 (end of month)
tp4m_main_rss_nochrome
84065800 (start of month) – 82167700 (end of month)
tp4m_shutdown_nochrome
25208800 (start of month) – 25205600 (end of month)
ts
3045.16 (start of month) – 2982.05 (end of month)
ts_shutdown
25204300 (start of month) – 25203800 (end of month)
Throbber Start / Throbber Stop
These graphs are taken from http://mrcote.info/phonedash/#/. Browser startup performance is measured on real phones.
“Time to throbber start” measures the time from process launch to the start of the throbber animation.
“Time to throbber stop” measures the time from process launch to the end of the throbber animation.
Eideticker
These graphs are taken from http://wrla.ch/eideticker/dashboard/#/. Eideticker is a performance harness that measures user perceived performance of web browsers by video capturing them in action and subsequently running image analysis on the raw result. More info at: https://wiki.mozilla.org/Project_Eideticker
(There are more Eideticker graphs, but that’s all the screenshotting I can stand!)
