Robotium is the most popular at the moment. The main pitfall is that it'll only interact with the android components itself (i.e. not any flash or web parts of the app). Also it's not the best in terms of getting using the logs for debugging since you can't take screenshots using it. But the best that I know of and I've had a few people in my team investigating it. Others we've looked at - FoneMonkey do an Android version of the iOS automated test tool but we've not tried that. Calculon's a newer one, but I get the impression it's not in the same league yet.
I'd be interested to hear other peoples views as well. It's not easy to get a lot of comparative reviews of these sorts of tools.
Excellent, thanks for the reply. I'll look forward to giving them a try.
Amy - You were probably referring to testing android apps ... but I would also be interested in methods for testing web page interfaces on android and iXxxx devices. Are there any emulators on the market that would emulate the various touchscreen interfaces?
Sorry for hijacking your thread! I thought it would be a good addition to this discussion.
Jeff - I was meaning apps but since you mention it I could also be interested in website testing on Android so I'm hoping you get a response :)
Amy - While doing some research this weekend, I ran across this site. The content and the comments at the end look promising.
Check this one out... First is an article about it, then the link to the site...
Bearing in mind that the article is written by the developer! ;0)
Thanks, a multi-platform tool is certainly an interesting option.
I know we tried using the Android SDK plugin for Eclipse & executing some tests from our Java platform but there was limited value for effort it took getting it set up & maintaining the test suite.
Consequently, we're using html contract tests, which have their own inherent problems!
Thanks for posting the question - some useful responses for us in here.
What do you mean by 'html contract tests'?
Unfortunately they're nothing specific to Android - we use them for our desktop & mobile sites.
So our pages are generated using String Templates as we have several different website templates the user can choose from.
These templates have different components in different places - this results in the web pages being rendered with a different html structure to another template.
We know what the html structure will be of each template so we have an automated test for each template which checks that the when the page is rendered for a template, it matches the expected html we know about (the expected html just lives in a text document)
The idea being, if a Developer accidentally changes some code which impacts the rendering of the web page, we'll know about it. Examples include adding/removing a component or moving the component.
These tests live in the Integration layer, so no need for a web server, making them relatively cheap to run
They're quite brittle, for obvious reasons, but they do give some assurances through automation the websites aren't completely smashed - there may still be some cosmetic problems, but at least the site should be legible.
BTW - this is my understanding of how they work. Might be a better answer out there!
I have a cool demo of TouchTest on my blog: http://www.fredberinger.com/mobile-test-automation-the-soasta-way-w... This is for iOS but it's coming for Android as well (and whoever emerge as the third contestant in the mobile battle ... Windows Mobile?)
We believe it's the right way to automate for mobile ... No simulator/emulator, no OCR, no jailbreaking or tethering .. Test inside the app!