Robocop supports UI testing of Firefox for Android using Robotium. With the changes in bug 854549, Robocop now automatically generates a screenshot when a test fails, so you can see what was happening at the moment the test failed.
I am hoping that this will help diagnose intermittent test failures, especially those that seem to only happen during automated testing.
With these changes, robocop takes a screenshot whenever a test assertion fails or a test throws an exception. It does that with Robotium’s Solo.takeScreenshot(), which simply generates a jpg. Unfortunately, takeScreenshot is confused by our content view: Instead of capturing the displayed web page, it shows a big black rectangle. That’s going to be a big disappointment when debugging some failures, and I hope we can find a solution to this limitation, but at least we can see chrome — and a lot of tests fail on the awesome screen or when interacting with menus or other Java UI elements.
The screenshot jpg is written to /mnt/sdcard/Robotium-Screenshots on the device. To allow this to work on tbpl, the robocop framework pulls the screenshot, base64 encodes it, and dumps it to the log. When debugging robocop failures on tbpl, look for “SCREENSHOT” in the log. You should see something like:
SCREENSHOT: data:image/jpg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAEB ...
Copy/paste the base64 text to a file and decode it:
base64 -d myfile > screenshot.jpg
And now you can view your screenshot:
This is still a little rough. One of these days I hope we can:
- capture content…maybe use a gecko screenshot?
- display the screenshot directly off of tbpl, similar to the reftest analyzer (see bug 747440)