Android 2.3 Opt tests on tbpl

Tags

,

Today, we started running some Android 2.3 Opt tests on tbpl:

Image

“Android 2.3 Opt” tests run on emulators running Android 2.3. The emulator is simply the Android arm emulator, taken from the Android SDK (version 18). The emulator runs a special build of Gingerbread (2.3.7), patched and built specifically to support our Android tests. The emulator is running on an aws ec2 host. Android 2.3 Opt runs one emulator at a time on a host (unlike the Android x86 emulator tests, which run up to 4 emulators concurrently on one ix host).

Android 2.3 Opt tests generally run slower than tests run on devices. We have found that tests will run faster on faster hosts; for instance, if we run the emulator on an aws m3.large instance (more memory, more cpu), mochitests run in about 1/3 of the time that they do currently, on m1.medium instances.

Reftests – plain reftests, js reftests, and crashtests – run particularly slowly. In fact, they take so long that we cannot run them to completion with a reasonable number of test chunks. We are investigating more and also considering the simple solution: running on different hosts.

We have no plans to run Talos tests on Android 2.3 Opt; we think there is limited value in running performance tests on emulators.

Android 2.3 Opt tests are supported on try — “try: -b o -p android …” You can also request that a slave be loaned to you for debugging more intense problems: https://wiki.mozilla.org/ReleaseEngineering/How_To/Request_a_slave. In my experience, these methods – try and slave loans – are more effective at reproducing test results than running an emulator locally: The host seems to affect the emulator’s behavior in significant and unpredictable ways.

Once the Android 2.3 Opt tests are running reliably, we hope to stop the corresponding tests on Android 2.2 Opt, reducing the burden on our old and limited population of Tegra boards.

As with any new test platform, we had to disable some tests to get a clean run suitable for tbpl. These are tracked in bug 979921.

There are also a few unresolved issues causing infrequent problems in active tests. These are tracked in bug 967704.

Firefox for Android Performance Measures – March check-up

Tags

,

My monthly review of Firefox for Android performance measurements. March highlights:

- 3 throbber start/stop regressions

- Eideticker not reporting results for the last couple of weeks.

Talos

This section tracks Perfomatic graphs from graphs.mozilla.org for mozilla-central builds of Native Fennec (Android 2.2 opt). The test names shown are those used on tbpl. See https://wiki.mozilla.org/Buildbot/Talos for background on Talos.

tcanvasmark

This test runs the third-party CanvasMark benchmark suite, which measures the browser’s ability to render a variety of canvas animations at a smooth framerate as the scenes grow more complex. Results are a score “based on the length of time the browser was able to maintain the test scene at greater than 30 FPS, multiplied by a weighting for the complexity of each test type”. Higher values are better.

7200 (start of period) – 6300 (end of period).

Regression of March 5 – bug 980423 (disable skia-gl).

tcheck2

Measure of “checkerboarding” during simulation of real user interaction with page. Lower values are better.

24 (start of period) – 24 (end of period)

trobopan

Panning performance test. Value is square of frame delays (ms greater than 25 ms) encountered while panning. Lower values are better.

110000 (start of period) – 110000 (end of period)

tprovider

Performance of history and bookmarks’ provider. Reports time (ms) to perform a group of database operations. Lower values are better.

375 (start of period) – 425 (end of period).

Regression of March 29 – bug 990101. (test modified)

tsvgx

An svg-only number that measures SVG rendering performance. About half of the tests are animations or iterations of rendering. This ASAP test (tsvgx) iterates in unlimited frame-rate mode thus reflecting the maximum rendering throughput of each test. The reported value is the page load time, or, for animations/iterations – overall duration the sequence/animation took to complete. Lower values are better.

7600 (start of period) – 7300 (end of period).

tp4m

Generic page load test. Lower values are better.

710 (start of period) – 750 (end of period).

No specific regression identified.

ts_paint

Startup performance test. Lower values are better.

3600 (start of period) – 3600 (end of period).

Throbber Start / Throbber Stop

These graphs are taken from http://phonedash.mozilla.org.  Browser startup performance is measured on real phones (a variety of popular devices).

3 regressions were reported this month: bug 980757, bug 982864, bug 986416.

:bc continued his work on noise reduction in March. Changes in the test setup have likely affected the phonedash graphs this month. We’ll check back at the end of April.

Eideticker

These graphs are taken from http://eideticker.mozilla.org. Eideticker is a performance harness that measures user perceived performance of web browsers by video capturing them in action and subsequently running image analysis on the raw result.

More info at: https://wiki.mozilla.org/Project_Eideticker

Eideticker results for the last couple of weeks are not available. We’ll check back at the end of April.

Android 4.0 Debug tests on tbpl

Tags

,

Today, we started running some Android 4.0 Debug mochitests and js-reftests on tbpl.

Screenshot from 2014-03-31 12:13:10b

“Android 4.0 Debug” tests run on Pandaboards running Android 4.0, just like the existing “Android 4.0 Opt” tests which have been running for some time. Unlike the Opt tests, the Debug tests run debug builds, with more log messages than Opt and notably, assertions. The “complete logcats” can be very useful for these jobs — see Complete logcats for Android tests.

Other test suites – the remaining mochitests chunks, robocop, reftests, etc – run on Android 4.0 Debug only on the Cedar tree at this time. They mostly work, but have failures that make them too unreliable to run on trunk trees. Would you like to see more Android 4.0 Debug tests running? A few test failures are all that is blocking us from running the remainder of our test suites. See Bug 940068 for the list of known failures.

 

Firefox for Android Performance Measures – February check-up

Tags

,

My monthly review of Firefox for Android performance measurements.

February highlights:

- Regressions in tcanvasmark, tcheck2, and tsvgx; improvement in ts-paint.

- Improvements in some eideticker startup measurements.

Talos

This section tracks Perfomatic graphs from graphs.mozilla.org for mozilla-central builds of Native Fennec (Android 2.2 opt). The test names shown are those used on tbpl. See https://wiki.mozilla.org/Buildbot/Talos for background on Talos.

tcanvasmark

This test runs the third-party CanvasMark benchmark suite, which measures the browser’s ability to render a variety of canvas animations at a smooth framerate as the scenes grow more complex. Results are a score “based on the length of time the browser was able to maintain the test scene at greater than 30 FPS, multiplied by a weighting for the complexity of each test type”. Higher values are better.

7800 (start of period) – 7200 (end of period).

Regression on Feb 19 – bug 978958.

tcheck2

Measure of “checkerboarding” during simulation of real user interaction with page. Lower values are better.

tcheck2

2.7 (start of period) – 24 (end of period)

Regression of Feb 25: bug 976563.

trobopan

Panning performance test. Value is square of frame delays (ms greater than 25 ms) encountered while panning. Lower values are better.

110000 (start of period) – 110000 (end of period)

tprovider

Performance of history and bookmarks’ provider. Reports time (ms) to perform a group of database operations. Lower values are better.

375 (start of period) – 375 (end of period).

tsvgx

An svg-only number that measures SVG rendering performance. About half of the tests are animations or iterations of rendering. This ASAP test (tsvgx) iterates in unlimited frame-rate mode thus reflecting the maximum rendering throughput of each test. The reported value is the page load time, or, for animations/iterations – overall duration the sequence/animation took to complete. Lower values are better.

7500 (start of period) – 7600 (end of period).

This test both improved and regressed slightly over the month, for a slight overall regression. Bug 978878.

tp4m

Generic page load test. Lower values are better.

700 (start of period) – 710 (end of period).

No specific regression identified.

ts_paint

Startup performance test. Lower values are better.

4300 (start of period) – 3600 (end of period).

Throbber Start / Throbber Stop

These graphs are taken from http://phonedash.mozilla.org.  Browser startup performance is measured on real phones (a variety of popular devices).

“Time to throbber start” measures the time from process launch to the start of the throbber animation. Smaller values are better.

throbberstart

“Time to throbber stop” measures the time from process launch to the end of the throbber animation. Smaller values are better.

throbberstop

:bc has been working on reducing noise in these results — notice the improvement. And there is more to come!

Eideticker

These graphs are taken from http://eideticker.mozilla.org. Eideticker is a performance harness that measures user perceived performance of web browsers by video capturing them in action and subsequently running image analysis on the raw result.

More info at: https://wiki.mozilla.org/Project_Eideticker

There is some improvement from last month’s startup measurements:

eide1

eide2

eide3

awsy

See https://www.areweslimyet.com/mobile/ for content and background information.

I did not notice any big changes this month.

Complete logcats for Android tests

Tags

,

“Logcats” – those Android logs you see when you execute “adb logcat” – are an essential part of debugging Firefox for Android. For a long time, we have included logcats in our Android test logs on tbpl: After a test run, we run logcat on the device, collect the output and dump it to the test log. Sometimes those logcats are very useful; other times, they are too little, too late. A typical problem is that a failure occurs early in a test run, but does not cause the test to fail immediately; by the time the test ends, the fixed-size logcat buffer has filled up and overwritten the earlier, important messages. How frustrating!

Beginning today, Android 4.0 and Android 4.2 x86 test jobs offer “complete logcats”: logcat is run for the duration of the test job, the output is collected continuously, and dumped to a file. At the end of the test job, the file is uploaded to an aws server, and a link is displayed in tbpl. Here’s a sample of a tbpl summary:

Image

Notice the (blobuploader) line? Open that link (yes, it’s long and awkward — there’s a bug for that!) and you have a complete logcat showing what was happening on the device for the duration of the test job.

We have not changed the “old” logcat features in test logs: We still run logcat at the end of most jobs and dump the output to the test log. That might be more convenient in some cases.

Are you wondering what “blobuploader” means? Curious about how the aws upload works? That’s the “blobber” project at work. See http://atlee.ca/posts/blobber-is-live.html and https://air.mozilla.org/intern-presentation-tabara/.

Unfortunately, the Android 2.2 (Tegra) test jobs use an older infrastructure which makes it difficult to implement blobber and complete logcats. There are no logcats-via-blobber for Android 2.2 — it’s only available for Android 4.0 and the newer Android emulator tests.

Happy test debugging!

Firefox for Android Performance Measures – January check-up

Tags

, ,

My monthly review of Firefox for Android performance measurements.

January highlights:

- only minor Talos regressions

- Eideticker startup regressions

- inconsistent improvement in many awsy measures

Talos

This section tracks Perfomatic graphs from graphs.mozilla.org for mozilla-central builds of Native Fennec (Android 2.2 opt). The test names shown are those used on tbpl. See https://wiki.mozilla.org/Buildbot/Talos for background on Talos.

tcanvasmark

This test runs the third-party CanvasMark benchmark suite, which measures the browser’s ability to render a variety of canvas animations at a smooth framerate as the scenes grow more complex. Results are a score “based on the length of time the browser was able to maintain the test scene at greater than 30 FPS, multiplied by a weighting for the complexity of each test type”. Higher values are better.

7800 (start of period) – 7800 (end of period).

tcheck2

Measure of “checkerboarding” during simulation of real user interaction with page. Lower values are better.

rck2

2.5 (start of period) – 2.7 (end of period)

Jan 16 regression – bug 961869.

trobopan

Panning performance test. Value is square of frame delays (ms greater than 25 ms) encountered while panning. Lower values are better.

110000 (start of period) – 110000 (end of period)

tprovider

Performance of history and bookmarks’ provider. Reports time (ms) to perform a group of database operations. Lower values are better.

375 (start of period) – 375 (end of period).

tsvgx

An svg-only number that measures SVG rendering performance. About half of the tests are animations or iterations of rendering. This ASAP test (tsvgx) iterates in unlimited frame-rate mode thus reflecting the maximum rendering throughput of each test. The reported value is the page load time, or, for animations/iterations – overall duration the sequence/animation took to complete. Lower values are better.

svg

7200 (start of period) – 7500 (end of period).

Regression of Jan 7 – bug 958129.

tp4m

Generic page load test. Lower values are better.

700 (start of period) – 700 (end of period).

ts_paint

Startup performance test. Lower values are better.

4300 (start of period) – 4300 (end of period).

Throbber Start / Throbber Stop

These graphs are taken from http://phonedash.mozilla.org.  Browser startup performance is measured on real phones (a variety of popular devices).

“Time to throbber start” measures the time from process launch to the start of the throbber animation. Smaller values are better.

throbber_start

There is so much data here, it is hard to see what is happening – bug 967052. I filtered out many of the devices to get this:

throbber_start-2

I think existing, long-running devices are showing no regressions, and some of the new devices are exhibiting a lot of noise — a problem that :bc is working to correct.

“Time to throbber stop” measures the time from process launch to the end of the throbber animation. Smaller values are better.

throbber_stop

A similar story here, I think.

But there was a regression for some devices on Jan 24 – bug 964323.

Eideticker

These graphs are taken from http://eideticker.mozilla.org. Eideticker is a performance harness that measures user perceived performance of web browsers by video capturing them in action and subsequently running image analysis on the raw result.

More info at: https://wiki.mozilla.org/Project_Eideticker

Let’s look at our startup numbers this month:

eide1 eide2 eide3 eide4 eide5 eide6 eide7 eide8

Regressions noted in bugs 964307 and 966580.

awsy

See https://www.areweslimyet.com/mobile/ for content and background information.

awsy1

awsy2

awsy3

There seems to be an improvement in several of the measurements, but it is inconsistent — it varies from one test run to the next. I wonder what that’s about.

Android x86 tests (S4) on tbpl

Tags

, ,

Today, we started running Robocop and xpcshell tests in an Android x86 emulator environment on tbpl.

Firefox for Android has been running on Android x86 for over a year now [1] and we have had Android x86 builds on tbpl for nearly as long [2], but our attempts to run test suites on Android x86 [3] have been more problematic. (See [4] to get an appreciation of the complications.)

This is the first of our Android test jobs to run in an emulator (Android 2.2 test jobs run on Tegra boards and Android 4.0 jobs run on Panda boards). The Android x86 tests run in Android x86 emulators running Android 4.2. The emulators run on Linux 64-bit in-house machines, and we run up to 4 emulators at a time on each Linux machine.

The new tests are labelled “Android 4.2 x86 Opt” on tbpl and look like this:

Screen shot 2014-01-21 at 12.58.45 PM

Each “set” contains up to 4 test jobs, reflecting the set of emulators that are run in parallel on each Linux box. For now, only set S4 is run on trunk trees; S4 contains xpcshell tests and robocop tests, broken up into 3 chunks, robocop-1, robocop-2, and robocop-3:

Image

Other test suites – mochitests, reftests, etc – run on Android 4.2 x86 Opt only on the Cedar and Ash trees at this time. They mostly work, but have intermittent infrastructure failures that make them too unreliable to run on trunk trees  (bug 927602 is the main issue).

Image

If you need to debug a test failure on this platform, try server support is available, or you can borrow a releng machine and run the mozharness job yourself.

[1] http://starkravingfinkle.org/blog/2012/11/firefox-for-android-running-on-android-x86/

[2] http://oduinn.com/blog/2012/12/20/android-x86-builds-now-on-tbpl-mozilla-org/

[3] http://armenzg.blogspot.ca/2013/09/running-hidden-firefox-for-android-42.html

[4] https://bugzilla.mozilla.org/showdependencytree.cgi?id=891959&hide_resolved=0

Firefox for Android Performance Measures – 2013 in review

Tags

, ,

Let’s review our performance measures for 2013.

Highlights:

- significant regressions in “time to throbber start/stop” and Eideticker startup tests

- most Talos measurements stable, or regressions addressed

- slight, gradual improvements to frame rate and responsiveness

Talos

This section tracks Perfomatic graphs from graphs.mozilla.org for mozilla-central builds of Native Fennec (Android 2.2 opt). The test names shown are those used on tbpl. See https://wiki.mozilla.org/Buildbot/Talos for background on Talos.

tcanvasmark

This test runs the third-party CanvasMark benchmark suite, which measures the browser’s ability to render a variety of canvas animations at a smooth framerate as the scenes grow more complex. Results are a score “based on the length of time the browser was able to maintain the test scene at greater than 30 FPS, multiplied by a weighting for the complexity of each test type”. Higher values are better.

tcanvas

7800 (start of period) – 7700 (end of period).

This test was introduced in September and has been fairly stable ever since. There does however seem to be a slight, gradual regression over this period.

tcheck2

Measure of “checkerboarding” during simulation of real user interaction with page. Lower values are better.

tcheck2

4.4 (start of period) – 2.8 (end of period)

This test saw lots of regressions and improvements over 2013, ending on a stable high note.

trobopan

Panning performance test. Value is square of frame delays (ms greater than 25 ms) encountered while panning. Lower values are better.

tpan

14000 (start of period) – 110000 (end of period)

The nature of this test measurement makes it one of the most variable Talos tests. We overcame the worst of the Sept/Oct regression, but still ended the year worse off than we started.

tprovider

Performance of history and bookmarks’ provider. Reports time (ms) to perform a group of database operations. Lower values are better.

troboprovider

375 (start of period) – 375 (end of period).

This test has hardly ever reported significant change — is it a useful test?

tsvgx

An svg-only number that measures SVG rendering performance. About half of the tests are animations or iterations of rendering. This ASAP test (tsvgx) iterates in unlimited frame-rate mode thus reflecting the maximum rendering throughput of each test. The reported value is the page load time, or, for animations/iterations – overall duration the sequence/animation took to complete. Lower values are better.

tsvg

1200 (start of period) – 7200 (end of period).

Introduced in September; the regression of Nov 27 is tracked in bug 944429.

tp4m

Generic page load test. Lower values are better.

tp4m

700 (start of period) – 700 (end of period).

This version of tp4m was introduced in September; no significant changes here.

ts_paint

Startup performance test. Lower values are better.

tspaint

4300 (start of period) – 4300 (end of period)

Introduced in September; there are no significant regressions here, but there is a lot of variability, possibly related to the frequent test failures — see bug 781107.

Throbber Start / Throbber Stop

These graphs are taken from http://phonedash.mozilla.org.  Browser startup performance is measured on real phones (a variety of popular devices).

“Time to throbber start” measures the time from process launch to the start of the throbber animation. Smaller values are better.

throbberstart1

There is so much data here, it is hard to see what is happening, but a troubling upward trend over the year is evident.

“Time to throbber stop” measures the time from process launch to the end of the throbber animation. Smaller values are better.

throbberstop1

Again, there is a lot of data here. Here’s another graph that hides the data for all but a few devices:

throbberstop2

Evidently we have lost a lot of ground over the year, with an increase in “time to throbber stop” of nearly 80% for some devices.

Eideticker

These graphs are taken from http://eideticker.mozilla.org. Eideticker is a performance harness that measures user perceived performance of web browsers by video capturing them in action and subsequently running image analysis on the raw result.

More info at: https://wiki.mozilla.org/Project_Eideticker

Eideticker confirms that startup time has regressed over the year:

eide1

eide2

Most checkerboarding and frame rate measurements have been steady, or show slight improvement:

eide5

Responsiveness — a new measurement added this year — is similarly steady with slight improvement:

eide3

eide4

awsy

See https://www.areweslimyet.com/mobile/ for content and background information.

awsy1

awsy2

awsy3

There is an upward trend here for many of the measurements, but what I find striking is how often we have managed to keep these numbers stable while adding new features.

Firefox for Android Performance Measures – November check-up

Tags

, ,

My monthly review of Firefox for Android performance measurements.

November highlights:

- significant improvement in tcheck2

- significant regression in tsvgx (SVG-ASAP) – bug 944429

- more devices added to phonedash

Talos

This section tracks Perfomatic graphs from graphs.mozilla.org for mozilla-central builds of Native Fennec (Android 2.2 opt). The test names shown are those used on tbpl. See https://wiki.mozilla.org/Buildbot/Talos for background on Talos.

tcanvasmark

This test runs the third-party CanvasMark benchmark suite, which measures the browser’s ability to render a variety of canvas animations at a smooth framerate as the scenes grow more complex. Results are a score “based on the length of time the browser was able to maintain the test scene at greater than 30 FPS, multiplied by a weighting for the complexity of each test type”. Higher values are better.

7800 (start of period) – 7800 (end of period).

tcheck2

Measure of “checkerboarding” during simulation of real user interaction with page. Lower values are better.

tcheck2

30 (start of period) – 2.5 (end of period)

The regressions of October were more than reversed.

trobopan

Panning performance test. Value is square of frame delays (ms greater than 25 ms) encountered while panning. Lower values are better.

140000 (start of period) – 140000 (end of period)

tprovider

Performance of history and bookmarks’ provider. Reports time (ms) to perform a group of database operations. Lower values are better.

375 (start of period) – 375 (end of period).

tsvgx

An svg-only number that measures SVG rendering performance. About half of the tests are animations or iterations of rendering. This ASAP test (tsvgx) iterates in unlimited frame-rate mode thus reflecting the maximum rendering throughput of each test. The reported value is the page load time, or, for animations/iterations – overall duration the sequence/animation took to complete. Lower values are better.

tsvg

1400 (start of period) – 7500 (end of period).

Regression of Nov 27 – bug 944429

 

tp4m

Generic page load test. Lower values are better.

700 (start of period) – 740 (end of period)

Slight regression of Nov 27 – bug 944429

 

ts_paint

Startup performance test. Lower values are better.

4300 (start of period) – 4300 (end of period)

Throbber Start / Throbber Stop

These graphs are taken from http://phonedash.mozilla.org.  Browser startup performance is measured on real phones (a variety of popular devices).

throbber-start

“Time to throbber start” measures the time from process launch to the start of the throbber animation. Smaller values are better.

Time to throbber start was generally stable during this period.

throbber-stop

“Time to throbber stop” measures the time from process launch to the end of the throbber animation. Smaller values are better.

Time to throbber stop was generally stable during this period.

Note that many new devices were added this month.

Eideticker

These graphs are taken from http://eideticker.mozilla.org. Eideticker is a performance harness that measures user perceived performance of web browsers by video capturing them in action and subsequently running image analysis on the raw result.

More info at: https://wiki.mozilla.org/Project_Eideticker

Many of the eideticker graphs were generally stable this month.

eide

awsy

See https://www.areweslimyet.com/mobile/ for content and background information.

awsy

awsy graphs were generally stable this month.

 

Firefox for Android Performance Measures – September/October check-up

Tags

, ,

I didn’t get a chance to write up a performance check-up post at the end of September, so this entry reviews September and October.

Highlights:

 - New Talos tcanvasmark test added in September

 - Talos tsvgx replaced the previous SVG test

 - Talos Tp4m and Ts tests updated

 - New “responsiveness” measures added to Eideticker tests

Talos

This section tracks Perfomatic graphs from graphs.mozilla.org for mozilla-central builds of Native Fennec (Android 2.2 opt). The test names shown are those used on tbpl. See https://wiki.mozilla.org/Buildbot/Talos for background on Talos.

tcanvasmark

This newly added (September 2013) test runs the third-party CanvasMark benchmark suite, which measures the browser’s ability to render a variety of canvas animations at a smooth framerate as the scenes grow more complex. Results are a score “based on the length of time the browser was able to maintain the test scene at greater than 30 FPS, multiplied by a weighting for the complexity of each test type”. Higher values are better.

Image

7800 (start of period) – 7800 (end of period).

tcheck2

Measure of “checkerboarding” during simulation of real user interaction with page. Lower values are better.

Image

High variability and regressions in August/September — see bug 913683.

15 (Sept 1) – 30 (Oct 30)

trobopan

Panning performance test. Value is square of frame delays (ms greater than 25 ms) encountered while panning. Lower values are better.

Image

16000000 (Sept 1) – 140000 (Oct 30)

The very significant regression from August (bug 908823) was fixed in September.

Image

A closer look at October reveals another, smaller, regression.

Regression of October 23 – see bug 877203.

tprovider

Performance of history and bookmarks’ provider. Reports time (ms) to perform a group of database operations. Lower values are better.

375 (Sept 1) – 375 (Oct 30).

tsvgx

This test was introduced in September, replacing the previous SVG test (see Replacing tscroll,tsvg with tscrollx,tsvgx).

An svg-only number that measures SVG rendering performance. About half of the tests are animations or iterations of rendering. This ASAP test (tsvgx) iterates in unlimited frame-rate mode thus reflecting the maximum rendering throughput of each test. The reported value is the page load time, or, for animations/iterations – overall duration the sequence/animation took to complete.

Image

1200 (Sept 19) – 1350 (Oct 30)

Regression of Oct 17 – see bug 923193.

tp4m

This test was updated in September, replacing tp4m_nochrome.

Generic page load test. Lower values are better.

Image

700 (Sept 19) – 700 (Oct 30).

tp4m_main_rss

136000000 (Sept 19) – 136000000 ( Oct 30).

ts_paint

Another test updated in September.

Startup performance test. Lower values are better.

Image

4300 (Sept 19) – 4300 (Oct 30).

Throbber Start / Throbber Stop

These graphs are taken from http://phonedash.mozilla.org.  Browser startup performance is measured on real phones (a variety of popular devices).

Image

“Time to throbber start” measures the time from process launch to the start of the throbber animation. Smaller values are better.

Time to throbber start was pretty stable during this period.

Image

“Time to throbber stop” measures the time from process launch to the end of the throbber animation. Smaller values are better.

Time to throbber stop was pretty stable during this period.

Eideticker

These graphs are taken from http://eideticker.mozilla.org. Eideticker is a performance harness that measures user perceived performance of web browsers by video capturing them in action and subsequently running image analysis on the raw result.

More info at: https://wiki.mozilla.org/Project_Eideticker

Eideticker now includes a “responsiveness” measure for several of the existing tests. See http://wrla.ch/blog/2013/10/first-eideticker-responsiveness-tests/ for background information.

Image

Most of the other measures seem fairly stable for September/October. Here are some of the more active graphs:

Image

Image

awsy

See https://www.areweslimyet.com/mobile/ for content and background information.

Image

Image

Image

Follow

Get every new post delivered to your Inbox.