Integrating Video Recording and Custom Reporters into AskUI Runs

Debugging UI-Automations can be a real pain especially when you run them inside a pipeline with no chance to observe what is actually happening during execution.

Wouldn't it be nice to get a report of failed automation steps that include screenshot before and after an execution and also a video?

We've recently added video recording support to our UI controller, enabling you to record a video for each step you execute and add it to a report of your AskUI runs.

We also provide an interface you can implement so you can integrate it into your favorite reporting tool like for example Allure.

Implementing A Reporter

To help you get started, we've created an example repository for Allure and also a repository where we collect reporter implementations.

Implementing your own reporter involves two main steps:

1. Implementing the reporter interface
2. Configuring the reporter within the askui-helper.ts.
   

Depending on your chosen reporter framework, you may not need to implement every function in the interface.

import { Step } from './step';
import { ReporterConfig } from './reporter-config';

/**
 * Can be used to report on a step run.
 *
 * The reporter is an interface (instead of a class) 
 * so that the it may be implemented by a class
 * that already extends another class, e.g., the 
 * reporter of a test framework.
 */
export interface Reporter {
    config?: ReporterConfig;
    onStepBegin?(step: Step): Promise<void>;
    onStepRetry?(step: Step): Promise<void>;
    onStepEnd?(step: Step): Promise<void>;
}

For instance, our Allure reporter only includes the config and onStepEnd() functions.

import { Reporter, Step, StepStatus } from "askui";
import { Status } from "jest-allure-circus";

function convertPngDataUrlToBuffer(pngDataUrl: string): Buffer {
  const base64Data = pngDataUrl.replace(/^data:image\/png;base64,/, "");
  const buffer = Buffer.from(base64Data, "base64");
  return buffer;
}

function mapAskuiToAllureStepStatus(status: StepStatus): Status {
  switch (status) {
    case "passed":
      return Status.PASSED;
    case "failed":
      return Status.FAILED;
    case "erroneous":
      return Status.BROKEN;
    default:
      return Status.SKIPPED;
  }
}

export class askuiAllureStepReporter implements Reporter {
  config = {
    withScreenshots: 'always' as const,
  };

  async onStepEnd(step: Step): Promise<void> {
    const status = mapAskuiToAllureStepStatus(step.status);
    const attachments = [];
    if (step.lastRun?.begin?.screenshot !== undefined) {
      attachments.push({
        name: "Before Screenshot",
        type: "image/png",
        content: convertPngDataUrlToBuffer(step.lastRun?.begin?.screenshot),
      });
    }
    if (step.lastRun?.end?.screenshot !== undefined) {
      attachments.push({
        name: "After Screenshot",
        type: "image/png",
        content: convertPngDataUrlToBuffer(step.lastRun?.end?.screenshot),
      });
    }
    allure.logStep(
      step.instruction.valueHumanReadable,
      status,
      attachments,
    );
  }
}

Optionally, you can provide mappings for the steps of __Jest__ to your reporter framework, tailoring the integration to your preferences.

To integrate video recording into your reporting tool, configure your reporter in the askui-helper.ts file within the UIControlClient. Additionally, extract the recorded video from the UIController and add it to your reporter.

Here's an example of how that is done:

...
  aui = await UiControlClient.build({
    inferenceServerUrl:
      process.env["ASKUI_INFERENCE_SERVER_URL"] ??
      "https://inference-dev.askui.com",
    reporter: new askuiAllureStepReporter(),
  });

  await aui.connect();
});

beforeEach(async () => {
  await aui.startVideoRecording();
});

afterEach(async () => {
  await aui.stopVideoRecording();
  const video = await aui.readVideoRecording();
  allure.createAttachment(
    "Video",
    convertBase64StringToBuffer(video),
    ContentType.WEBM
  );
});

Allure Example

Our Allure example repository also publishes its report to a website: Check it out here!

‍

Links for Already Implemented Reporters

We've created a community repository on our GitHub space where you can find and contribute implemented reporters. If you need inspiration or want to share your custom reporter with the community this is the right place to start!

All reporters and Instructions to implement your own

Conclusion

Incorporating screenshots and videos into your executed steps reports can significantly enhance your reports, as demonstrated in the provided example.

If a specific reporter is not implemented already, you can implement it on your own and integrate it with the instructions and examples given in our repositories.

If you need help, please join our Community where the AskUI staff and the community are ready to help you out!