Google recently released the Arc Welder extension for Chrome, which allows an Android app to be run on the desktop. The aim of Arc Welder is to help make testing Android apps easier, but conveniently it also makes scraping Android apps easier too.
To run an app in Arc Welder we need to first download the app’s APK file. Google Play does not make this straightforward so I would suggest using an alternative such as androiddrawer.com. I often use the Exchange Rates app when travelling and was curious where they source their currency data. The APK is available for download here, and then when loaded in Arc Welder looks like this:
(Note that if you get an error saying WebGL is not supported then you need to force Chrome to support WebGL by enabling the Override software rendering list flag: chrome://flags/#ignore-gpu-blacklist)
Now that the Exchange Rates app is running on my computer I used Wireshark to track network traffic, which showed the following request was made:
This means that this app simply uses the Yahoo Finance API for their currency data, available at: http://finance.yahoo.com/webservice/v1/symbols/allcurrencies/quote?format=json. So to scrape the data from the Exchange rate app merely requires calling this API.
(If you want to learn more about how to use Wireshark check out their extensive documentation.)
I chose a non-controversial app for this blog post that uses a public API, but this same technique can be applied to any Android app that loads its content from a backend server, which ought to be most apps with data of interest.