Create XRAY JSON Report And Upload it via API

When you use Jira as a testing team it is highly likely that you also use XRAY for testmanagement. So when you run UI-tests with AskUI you want to integrate the test run reports into xray automatically on each run.

With the latest version of askui-reporters we added the AskUIXRayStepReporter that can create XRAY-proprietory-json reports, which offers step-level-reporting with screenshots!

In this post we will show you how to setup this reporter and get the reports into your Jira/XRAY via REST-API.

Prerequisites

• AskUI installed and configured on your system (Windows, Linux, macOS)

Installation and Configuration of the AskUIXRayStepReporter

The AskUIXRayStepReporter already comes bundled with the latest AskUI version. But by default our Allure Reporter is active.

In the following steps we will show you how to activate and configure the AskUIXRayStepReporter instead.

Enable and Configure the AskUIXRayStepReporter in `helper/askui-helper.ts`

First you have to modify your helper/askui-helper.ts file to use the AskUIXRayStepReporter.

Afterwards it should look like this.

import { UiControlClient, UiController } from 'askui';
import { AskUIXRayStepReporter } from '@askui/askui-reporters';

// Client is necessary to use the askui API
let aui: UiControlClient;

jest.setTimeout(60 * 1000 * 60);

let xRayReporter: AskUIXRayStepReporter;

beforeAll(async () => {
  xRayReporter = new AskUIXRayStepReporter({
    // Uncomment the next line if you want screenshots also when steps pass
    // withScreenshots: 'always',
  });

  aui = await UiControlClient.build({
    reporter: xRayReporter,
  });

  await aui.connect();
});

beforeEach(async () => {
  xRayReporter.createNewTestEntry(global.testName);
});

afterEach(async () => {
  xRayReporter.finishTestEntry(global.testStatus);
});

afterAll(async () => {
  await xRayReporter.writeReport();

  aui.disconnect();
});

export { aui };

Enable the `jest-xray-environment` in `jest.config.ts`

Our Reporter has to use a special jest-environment, so it can include the name of each test from the it-block in the report.json. You configure the jest-environment in jest.config.ts like this.

import type { Config } from '@jest/types';

const config: Config.InitialOptions = {
  preset: 'ts-jest',
  testEnvironment: '@askui/askui-jest-xray-environment',
  setupFilesAfterEnv: ['./helpers/askui-helper.ts'],
  sandboxInjectedGlobals: [
    'Math',
  ],
  reporters: [ "default" ]
};

// eslint-disable-next-line import/no-default-export
export default config;

An Example Report

Here is an example report.json from our example repository. As you can see the testKey property is the description from the it-block. Also in this case we configured the xRayReporter in helper/askui-helper.ts to alway take screenshots instead of only when a step failed.

xRayReporter = new AskUIXRayStepReporter({
  withScreenshots: 'always',
});

Full report:

{
  "tests": [
    {
      "testKey": "TEST1",
      "start": "2024-01-10T15:17:05.998Z",
      "steps": [
        {
          "status": "PASS",
          "evidences": [
            {
              "data": "iVBORw0KGgoAAAANSUhEUgAAC ...",
              "filename": "before.png",
              "contentType": "image/png"
            },
            {
              "data": "AAAAesCAYAAACEMRk8ABJYYklEQVR ...",
              "filename": "after.png",
              "contentType": "image/png"
            }
          ]
        }
      ],
      "finish": "2024-01-10T15:17:10.852Z",
      "status": "FAIL"
    }
  ]
}

Upload over XRAY REST-API

Getting a XRAY-compatible JSON is nice, but useless without pushing it to XRAY 😉. The recommended way is to use XRAYs-REST-API.

In this Gitlab-CI-example the first job run-askui-job runs your tests inside a Gitlab-Pipeline and makes the report availaible under xray-report/report.json.

The next job transfer-report-to-xray-job requests a token from the Jira/XRAY-Server and then uploads the report via API.

Do not forget to set the Gitlab-Variables for your Jira/XRAY-Instance: CLIENT_ID, CLIENT_SECRET, JIRA_SERVER_URL, PROJECT_KEY

Conclusion

With our AskUIXRayStepReporter you can create XRAY-compatible JSON with stepreporting and screenshots on fail-only/always. For a complete example including all the setup files, please check our XRAY reporter example repository on Github.