Roland Oldengarm - Independent IT Contractor

Living in the coolest little capital Wellington, New Zealand!

How to automatically test your non-Angular Cordova or Ionic App

Part of building any application is testing. I am focusing only on tests executed by the developer, so the first stage in any development cycle. The first option is to do this manually. While this works for most usecases, it has some major disadvantages:

  • It is time consuming.
  • You can’t run all the manual tests after each change.

When a developer does a manual test, he only confirms that the change did what it had to do. To assure it did not break anything else, in most cases he will do a quick regression test, but that won’t cover all possible regression issues.

We are building a Cordova / Ionicapp, with some complicated, time consuming use cases like logging in, logging out, adding items to a list, etc. Any change we make may cause a regression. We do have some unit tests to cover certain services and controllers, but that does not cover everything.
So, we have decided to use end-to-end automated tests to be sure that before each release, the major part of the app is still working as expected. Any bugs detected with these tests can be fixed by a developer straightaway. If it would be detected by our testers, or even worse by the client in UAT, this would make it much more expensive to fix.

I have spent some time to figure out what the best way of doing this is. I couldn’t find a lot of working documentation and examples to set up automated user testing with a Cordova / Ionic mobile application.

Which tools are required for automated Cordova / Ionic testing?

There are multiple options, but after some analysis I’ve picked the following tools to create my automated tests:

  • Appium: This is essentially Selenium for mobile apps. It provides a service that allows your scripts to communicate with the connected device or emulator
  • Protractor: This is the defacto test framework for Angular applications. The good thing is: Protractor also works with non-Angular applications! Our application contains certain non-Angular parts, and Protractor can handle this as I will show you later.

Other tools you will need are:

  • Node.JS: These tools run on Node.js, so you will also need Node.js and NPM installed. Most likely you already have this on your machine.
  • Python: Python is required for certain Node.JS packages, including Appium and Protractor.
  • The Android SDK: I am going to show you how to use an Android Emulator or Android device to run the tests. Obviously you’ll need the Android SDK installed to be able to deploy an Android app.
  • An editor: I am using Microsoft Visual Studio to build and deploy Cordova / Ionic apps, but it will work on any environment. I haven’t tested it, but it should also work on Linux or Mac OSX. You could use Visual Studio Code to edit the files, and the command line to deploy.

The big advantage of automated testing is that bugs, once detected, won’t come back, ever.

Install the prerequisites

First of all, make sure that NPM and Node.js are up-to-date. It may work on older versions, but I’ve updated to the latest version which currently is 4.3.0. Then, run the following commands to install Appium, Protractor, and some dependecies:


npm install -g appium
npm install -g protractor
npm install -g protractor-fail-fast
npm install protractor --save-dev
npm install wd --save-dev
npm install wd-bridge --save-dev

Appium and Protractor need to be installed globally, the other packages can be installed in your project. Not 100% sure, but protractor must also be installed locally, for the webdriver and webdriver-bridge to work. Protractor-fail-fast should work as a local package, but I couldn’t get it to work, so installed that plugin globally.

  • Protractor-fail-fast is a package that will make sure that your test run stop as soon as an error is encountered. This is quite handy, as a test run can take quite a while.
  • WD and WD-Bridge: Web driver is the technology that allows you to communicate with the browser, in our case the WebView that is powering the Cordova app. WD-bridge allows Appium to communicate with the Web driver.

Download Chromedriver

For Android development, Chromedriver is used to be able to communicate with the device. Chromedriver.exe is installed as part of Appium, but the version is not the latest one. I got some weird errors, caused by the outdated Chromedriver. After installing the latest version (currently 2.21) the issue was fixed. Download it from here and extract it to /chromedriver/chromedriver.exe

Set up environment variables

  • Make sure java.exe is included in your path (C:\Program Files (x86)\Java\jdk1.7.0_55\bin on Windows in my case)
  • Set ANDROID_HOME environment variable to the root folder of the Android SDK (C:\Program Files (x86)\Android\android-sdk in my case)

Set up protractor-config.js

Add a new file to the root of your project and name it protractor-config.js. Copy and paste the contents from below:

var failFast = require('protractor-fail-fast');

exports.config = {
 plugins: [{
 package: 'protractor-fail-fast'
 }],
 seleniumAddress: 'http://localhost:4723/wd/hub',
 specs: ['./tests/e2e/*.js'],

 // Reference: https://github.com/appium/sample-code/blob/master/sample-code/examples/node/helpers/caps.js
 capabilities: {
 platformName: 'Android',
 platformVersion: '4.4.4',
 deviceName: 'Android',
 browserName: "",
 autoWebview: true,
// Appium will install a certain Chromedriver version, but that failed in my case. Download Chromedriver from http://chromedriver.storage.googleapis.com/index.html?path=2.21/ and extract it to /chromedriver/chromedriver.exe
// chromedriverExecutable: '/chromedriver/chromedriver.exe',
 //CHANGE THIS TO YOUR ABSOLUTE PATH
 app: '/path/to/your/package.apk',
 },
 baseUrl: 'http://10.0.2.2:8000',

 // configuring wd in onPrepare
 // wdBridge helps to bridge wd driver with other selenium clients
 // See https://github.com/sebv/wd-bridge/blob/master/README.md
 onPrepare: function () {
 jasmine.getEnv().addReporter(failFast.init());
 var wd = require('wd'),
 protractor = require('protractor'),
 wdBridge = require('wd-bridge')(protractor, wd);
 wdBridge.initFromProtractor(exports.config);
 },
 afterLaunch: function () {
 failFast.clean(); 
 }
 
};

That’s it! Some things to notice:

  • Change the app property and set it with the path to your APK. Appium will deploy this APK to the phone before the tests.
  • specs: Configure it with the path to your test scripts. More about that later.
  • onPrepare: This essentially connects protractor with the webdriver, and it enables failFast to stop tests as soon as one fails.
  • capabilities: This is passed through to the ChromeDriver, and if you miss something here, the ChromeDriver will throw an error. E.g. if you emit platformName, it won’t work.

How to create a basic test script?

You can use multiple languages to create the test script, for example in C# or in Java. As I’m familiar with Jasmine tests, I’ve picked that one. A test script looks like this:

describe('Testing the app', function () {
  beforeEach(() => {
    // this line disables the wait for Angular. This is required if your app is not using Angular
    browser.ignoreSynchronization = true;
    // increase default timeout of a test, as a test may take a couple of minutes
    jasmine.DEFAULT_TIMEOUT_INTERVAL = 300000;
  });
  afterAll(() => {
    // end your session with Appium. If you don't do this, Appium will throw an error the next time you try to run it as the old session has not been closed
    browser.driver.quit();
  });
  it('01. should have a header', function () {

    var header = element(by.id('header'));
    expect(header.getAttribute('class')).toContain('toolbar');

   });

The comments in the code should be self-explanatory. Important to note is that even though Protractor is for Angular applications, it will work for non-Angular applications as well!

Protractor is for Angular applications, it will work for non-Angular applications too!

Our app is made of a non-Angular part, and a part driven by Ionic. So this was a requiremenet for us.

Let’s try if it all works. Open two command prompts, or use the awesome console for Windows named ConEmu which allows you to create multiple tabs. In one window, typ ” appium –session-override &” to start the Appium server. I’ve set the flag session-override; when something goes wrong, the afterAll() in your test scripts won’t release the session, resulting in the Appium error A new session could not be created. (Original error: Requested a new session but one was in progress).

In the other window, go to your project folder and type:

protractor protractor-config.js

This will start all of the test scripts, and show the results in the output. Each successful test will show a dot (.), for a failed test an F will be displayed:

> protractor protractor-config.js 
Using the selenium server at http://localhost:4723/wd/hub 
[launcher] Running 1 instances of WebDriver 
Started 
.
Ran 1 of 1 specs

Create test scripts for all your scenarios

The next thing is to create test scripts for your scenarios. We have created test scripts for the basic functionality of our app. And, if a bug was detected by a tester, we sometimes created a test script to reproduce it, and to be sure it doesn’t come back in the future.

Protractor allows you to do things like clicking a button, sending text to a text box, tapping and sliding, waiting until an element is visible, etc. etc. Check out the API here.

Some things I’ve learnt and I want to share:

  • Timing is very important: Your script will in most cases run too quickly. Don’t just add browser.sleep() because that’s just random. Add browser.wait() to make sure your script waits until your device is ready for the next step.
  • Use browser.driver, and not just browser. According to the documentation, browser.driver must be used in non-Angular applications, and browser can be used in Angular applications. I’ve found that browser.driver is slightly more reliable.
  • I haven’t found a way to work with native pop-ups: If your Cordova app uses the notification plugin, you probably can’t access them in your scripts.
  • Use a real device: An emulator is much slower, even on my very fast laptop. I’ve tested both Google’s emulators, and the Visual Studio emulators, and both are quite slow.
  • If you demo to e.g. a client record a video: A test run may take quite some time, and by recording a video you can skip through. And your test will always succeed 🙂 Open Broadcaster Software is an excellent piece of software to do this for you.

Any questions? 

I have spent quite some time to get all of this working. Please flick me an email to me@rolandoldengarm.com if you have any questions, I’m happy to help out.

1 Comment

  1. GovindaNarayanaBhat

    June 22, 2017 at 9:30 pm

    can we integrate with cucumber? if yes, any pointers on the same?

Leave a Reply

Your email address will not be published.

*