Threading and Android integration testing

Synopsis
Recently I was testing some single-threaded (non user-interface) Android code that was using a BroadcastReceiver to monitor network state. This post describes how your test can run into thread issues even though your code is not multithreaded.

Introduction
The Android testing framework allows you to test different components of your application on an actual device or an emulator. This framework is great for integration- and functional testing, i.e. testing a combination of modules/classes and interaction with the device and Android components. (The Android testing framework is however not very suitable for unit testing - see my previous post).

When running integration tests with the Android testing framework it is good to keep in mind that by default, tests are not run on the main application thread (also known as the UI Thread). Some Android components such as those that interact with the GUI (i.e. Views) however need run on the main thread. The Activity instrumentation test provides nice support for testing GUI components  on the main thread as described in the activity testing tutorial. There are however other components that also run on the main thread and if you do not consider this in your integration tests you might be up for a surprise.

Multithreaded surprise
A good example of a component that runs on the main application thread is the onReceive method of a BroadcastReceiver.  As stated in the documentation, "This method is always called within the main thread of its process, unless you explicitly asked for it to be scheduled on a different thread ...".

Recently I was testing a module that is doing some networking related stuff. This module does not spawn off any local threads but it registers a BroadcastReceiver for monitoring connectivity and the state of the wifi interface. When I was testing my code, I occasionally saw some weird sequences of action which were being caused by different threads running in parallel on my module. The reason was that events that arrive on the BroadcastReceiver were being run on the main application thread while the methods invoked from the test were running on a different thread.

Since I do not want to make my code thread safe just for testing purposes I needed to figure out how to run the tests on the main thread.  Also, since my code is part of a library and not tied to a particular Android Activity it was not reasonable to use the Activity test case mentioned above.

Test example
The test case below shows the test threading issue in practice with a simple example.  All it does is register a BroadcastReceiver that monitors changes in wifi state.

When running this test (with wifi turned on) I see the following logcat output

D/TestStuff(11890): Test is running on thread: Instr: android.test.InstrumentationTestRunner
D/TestStuff(11890): onReceive is running on thread: main

Obviously the testStuff method and the onReceive method are running in different threads.  So how can we make the test to run on the main thread, just like the broadcast receiver?

The solution is to use the @UiThreadTest annotation which causes the test to run on the main thread.  This annotation does however only have the desired effect if the test case is subclassed from InstrumentationTestCase.

After running the new test we see the following logcat output.

D/TestStuff(12042): Test is running on thread: main
D/TestStuff(12042): onReceive is running on thread: main

Now both the test and the broadcast receiver are running on the same thread.

Android Test Driven Development

The unit test is the heart of Test Driven Development (TDD). For the development cycle of TDD to be effective, it is crucial that your unit tests run fast and that the test-deploy-program cycle runs smoothly and does not break your flow when you are "in the zone".

The Android Testing framework that is integrated with the Android Development environment is however not well suited for TDD. Why? Whenever you want to run a test in this environment, the test and the code being tested needs to be deployed on the Android Emulator (or an actual device) where they can be run in an actual Android Runtime Environment (i.e. the Dalvik VM). This deployment is slow (takes several seconds) making the test driven development cycle heavy and cumbersome - which is exactly what it is not supposed to be. To put it short: relying on the Android emulator or an actual device for unit testing is a pain in the butt (the lights in my office dim when I start the emulator).

Dependency inversion and mock objects

If you follow the SOLID principles (which you should (listen to your mother!)) you know that your code should not depend on the actual Android implementations but instead you should wrap and mask these with appropriate interfaces. On runtime you use the android implementations for your interfaces and when unit testing you inject a mock implementation. When mocking the Android methods you can use frameworks like mockito to help you out and eventually you might have your core application code as a pure java library. Now you can unit test this code with the normal JVM and at runtime you provide interface implementations from a separate Android library. This is how I was doing Android TDD until recently.

The downside with this is that you end up with a lot of test-specific code - mocking out all Android dependencies quickly results in a lot of boilerplate code.  So my concern is this:  How can we do TDD with Android and follow the dependency inversion principle but reduce the level of test-specific code?

Introducing Robolectric

Recently I started using Robolectric for all my Android unit tests. This has significantly boosted my work-flow when programming and reduced the level of mocking required in my projects.

Robolectric intercepts the loading of the Android classes and instead provides shadow objects for the corresponding Android classes. This means that you can run your unit tests on a regular Java VM in your IDE test environment. Robolectric integrates very nicely with maven and getting it to work with eclipse is relatively straight forward if you follow the steps on their website.

What about the Android Testing framework then?

Although I avoid using the Android Testing Framework in my TDD work cycle, it is still extremely useful for integration testing.  Integration tests are important tests but they are usually run less frequently and most programmers (well...at least me) don't include them in the daily TDD cycle. In particular the Android Test Framework is needed when you want to run integration tests using an actual device database, network connection, wifi, Bluetooth, NFC etc.