Mobile DevOps using VSTS, Xamarin Test Cloud and HockeyApp for Xamarin

In this post I will show how effectively we can use VSTS to provide build automation, release management for Xamarin Apps and use Xamarin Test Cloud and HockeyApp for application testing.

VSTS stands for Visual Studio Team Services, is a cloud service to leverage certain features like defining new projects, team structure, build management, release automation and others that can be used in each phase of the software development lifecycle. Every software lifecycle undergo several phases that includes planning, designing, development, testing and then deployment on production and VSTS provides several components that can be used to accelerate and automate tasks with the complete team management and reporting.

VSTS is an online version of TFS that can be accessed from and it provides a free account supported up to 5 members. With VSTS we can define team projects, select version control (Git or TFS), define team members, manage work items, define build definitions, release management and so on.

So let’s take a simple example of hosting our basic Xamarin app on VSTS and define build definitions and policies to build application on code check in and then pass through some steps to test on cloud and publish on HockeyApp

Creating Team Project

Login with your account on {accountname} and create a new Team Project by hitting New Project button as shown below. You can name it any, in my case I named it as MobileDevOps and selected Git as the version control. Once this project is created you can clone this repository in Visual Studio or use Git commands to push your Xamarin solution on cloud.

Once your project is checked in on VSTS you can view the files from Code tab as shown below


So now we can play with defining a build and enable continuous integration.

Setting up a Build Definition

We can create a new build definition by going to the Build & Release tab and click on Builds. This opens up a page from where we can create a new definition. When you hit new definition, it will ask to select any template from the existing ones or select empty to create your own.

We will select the Xamarin.Android as in this post we will cover the configuration related to Android. Go through the wizard and it generates the basic template containing few steps


I have modified and few steps and they were deprecated and below snapshot shows the final set of steps executed when the new build is queued.


First step talk about restoring NuGet packages and restore all the packages defined in the package.json file in your solution. The configuration for NuGet restore would be as follows


Second step is about building the Xamarin project. Project reference the path of our Xamarin Android project, Output directory is where the .apk file will be created. Create App Package needs to be checked. From MSBuild we can select the version and architecture for which we need to create a package for. From JDK Options we can select the specific JDK version


Third step is to copy the .keystore file to the binaries directory. This is required if we want to distribute our app to the HockeyApp store. We can create .keystore file by running a keytool command from the path where Java is installed.

Here is the command

C:\Program Files (x86)\Java\jdk1.8.0_112\bin>keytool -genkey -v -keystore “D:\Projects\Xamarin\myappkey.keystore” -sigalg SHA1withDSA -keyalg DSA -keysize 1024

Below snapshot shows the steps when command is executed and ask for the Password.


We also need to note down the KeyAlias to specify in HockeyApp site and that can be obtained by running following command.

C:\Program Files (x86)\Java\jdk1.8.0_112\bin>keytool -keystore “D:\Projects\Xamarin\myappkey.keystore” -list -v


So, once the keystore is generated you have to check-in that file on your source code repository on the root folder where .sln file resides and then configure the Copy files to build directory as shown below.


On fourth step, we have to sign our package with the keystore we have generated and specify the same Key Password provided during the generation of keystore and for Alias use the command above to obtain the alias name.


Next, for Build solution step which is the fifth step in our definition. We will specify the Test project path and MSBuild arguments to place the test binaries under test-assembly folder. To learn how to create unit testing project for Xamarin apps, check following link,_testing,_and_metrics/touch.unit/


Sixth step is running the tests on cloud and here is the configuration for Xamarin Test Cloud


Team API key can be obtained from Login to this website and go to Account Settings. Click on Teams & Apps and click on Show API Key


Then, you have to define a new test run for that particular Team and select Android


Select devices for which you want the tests to run


Complete the wizard and note the device ID as shown on the last page wizard, and then specify the device Id, API key and your email to which your account is registered in Xamarin Test Cloud on configuration page in VSTS.


Seventh step is about publishing results in an xml file executed by Xamarin Test Cloud.


And then finally on eighth step we will copy the tested package in our drop artifact


Please note: Xamarin Test Cloud needs a permission to access and execute test cases. So make sure that your Xamarin AndroidManifest.xml file must have following entry

<uses-permission android:name=”android.permission.INTERNET” />

To verify, lets run a build by hitting Queue new build and see if the build is succeeded.

Create Release Definition to distribute our App to HockeyApp users

To create a release definition, we will go to Build & releases > Releases and create a new definition.  Provide any name and then create a new Environment and add task HockeyApp as shown below


If HockeyApp task is not showing in your task list, then add the extension for VSTS from the VS Marketplace

And here is the HockeyApp configuraiton


So this create our release definition but it will not work until we associate our VSTS with HockeyApp. So we have to go to the website  and then create an API token with full access. This API token can be created by going to the Account Settings> API Token as shown below.


Note the API Token and then create a new Service Endpoint from VSTS by going to the Services tab and choose HockeyApp as follows


Specify the connection name and the API token retrieved from HockeyApp website


once this is done we can install the hockey app on our android device from as this is not available on store. We have to enable the “unknown sources” in our android device settings so HockeyApp app can be installed.

Once installed, you can sign in with your hockey account and download the app which will be pushed when the release definition will run.

Please note that the Continuous Integration (CI) can be enabled for particular build definition from Triggers tab and this runs everytime when the user check in any code. Moreover we can also enable the Continuous Deployment (CD) from release definition triggers tab which deploy the app on HockeyApp once the build is succeeded.


Hope this helps!



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s