AppSPIN: reconfiguration-based responsiveness testing and diagnosing for Android Apps

App responsiveness is the most intuitive interpretation of App performance from the users’ perspective. Traditional performance profilers only focus on one type of program activity (e.g., CPU profiling). In contrast, the cause of slow responsiveness can be diverse or even due to the joint effect of multiple kinds. Also, various test configurations, such as device hardware and wireless connectivity, can dramatically impact particular program activities and indirectly affect App responsiveness. Conventional mobile testing lacks mechanisms to reveal configuration-sensitive bugs. In this paper, we propose AppSPIN, a tool to diagnose App responsiveness bugs and systematically explore configuration-sensitive bugs automatically. AppSPIN instruments the App to collect program events and UI responsiveness. The instrumented App is exercised with automated monkey testers, and AppSPIN correlates excessive and lengthy program events with poor responsiveness detected at runtime. The diagnosis process also synthesizes the major resource bottleneck for the App under test. After one test run, AppSPIN automatically alters the test configuration with most bottlenecked resources to further explore responsiveness bugs that occur only with particular test configurations. Our experiments with 30 real-world Apps show that AppSPIN can detect 123 unique responsiveness bugs and successfully diagnose the cause for 87% cases with an average of 15-minute test time and negligible overhead. Also, with altered test configurations, AppSPIN uncovers a considerable number of new bugs within four extra test runs.