Tags

, ,

Beginning today, “Android 4.3 API11+ opt” unit tests are running on treeherder on all trunk trees. These tests run against our standard Firefox for Android API11+ arm builds, but run in an Android arm emulator running Android 4.3. Just like the existing Android 2.3 tests, the emulator for 4.3 runs on an aws instance.

The emulator environment has these characteristics:

  • Android 4.3 (AOSP 4.3.1_r1, JLS36I); standard 2.6.29 kernel
  • 1 GB of memory (not quite fully utilized because the kernel does not support CONFIG_HIGHMEM)
  • 720×1280, 320 dpi screen
  • 128 MB VM heap
  • 600 MB /data and 600 MB /sdcard partitions
  • front and back emulated cameras; all emulated sensors
  • standard crashreporter, logcat, anr, and tombstone support

This Android 4.3 emulator environment is very much like the existing “Android 2.3 API9 opt” environment. Broadly, tests seem to run in about the same amount of time on 4.3 as on 2.3 and we see some of the same failures on 4.3 as on 2.3. One significant difference between the 4.3 and 2.3 environments is the “device manager” used to communicate between the test harnesses and the device. On Android 2.3, sutagent is installed on the device and a custom tcp protocol is used to push/pull files, start processes, etc; on Android 4.3, sutagent is not used at all (it doesn’t play well with SELinux security) and adb is used instead.

Android 4.3 API11+ opt tests are available on try and run as a consequence of:

try: -b o -p android-api-11 -u …

As Android 4.3 API11+ opt tests have been introduced, the corresponding “Android 4.0 API11+ opt” test jobs have been disabled. Android 4.0 tests were running on our aging Pandaboards; running in the emulator on aws is more scalable, cost-effective, and future-safe.

Android 4.0 API11+ debug tests continue to run, but we plan to migrate those to the 4.3 emulator soon.

A few Android 4.0 API11+ opt Talos tests continue to run. We are evaluating whether those can be replaced by similar Autophone tests.

As with the introduction of any new test platform, some tests failed on Android 4.3 and had to be disabled or marked as failing on 4.3. Corresponding bugs have been opened for these tests and you can find them by looking for bugs with “[test disabled on android 4.3]” on the whiteboard:

https://bugzilla.mozilla.org/buglist.cgi?list_id=12205056&resolution=—&resolution=FIXED&resolution=INVALID&resolution=WONTFIX&resolution=DUPLICATE&resolution=WORKSFORME&resolution=INCOMPLETE&resolution=SUPPORT&resolution=EXPIRED&resolution=MOVED&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=disabled%20on%204.3&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=RESOLVED&bug_status=VERIFIED&bug_status=CLOSED

Great thanks to everyone who has contributed to the 4.3 effort, but especially :kmoir for all the Release Engineering work to get everything running smoothly in continuous integration.

Looking for more details about the new environment? Have a look at:

Bug 1062365 Investigate Android 4.3/4.4 emulator test setup

Bug 1133833 Android 4.3 emulator tests

…or ask me!

Advertisements