Using CDRouter with GitLab CI/CD

GitLab is a popular DevOps tool used to manage the workflow versioning, building and testing software. In a CPE firmware build environment, CDRouter can be combined with GitLab’s CI/CD features to provide testing verification of each firmware build.

GitLab CI/CD Test Results

GitLab CI/CD has built in concepts for testsuites and testcases. It natively recognizes the Junit XML test result format. While not all CDRouter test concepts directly map to GitLab CI/CD, it is possible to translate CDRouter test results into Junit XML test results that GitLab can understand.

Example Work Flow

The GitLab CI/CD pipeline is divided into multiple stages, and within each stage, individual jobs. Each of these is defined in a YAML script, usually called .gitlab-ci.yml and stored in the root directory of a repository. You can read more about CI/CD configuration in gitlab’s documentation site; this document will focus on defining a CDRouter test job in the test stage of the pipeline. The build stage needs to install and deploy the current firmware version onto a CPE that is setup to be tested against CDRouter. The steps to set this up are implementation-specific and outside the scope of this document. For more details about setting up a CPE for testing, see the CDRouter Quick Start Guide.

In the following example, the .gitlab-ci.yml file defines a Python script that uses the CDRouter Web API to start one or more test packages. The script waits for the packages to complete, prints the results and creates a Junit file that GitLab can parse to display the results.

Each time a GitLab pipeline is run CDRouter will run one or more packages against the latest firmware build, and include the test results in the pipeline report. This brings immediate visibility to any firmware change that affects a CDRouter test.

A Simple Example

The following shows an example test job in a .gitlab-ci.yml file:

cdrouter-test:
  tags:
    - cdrouter-gitlab-runner
  stage: test
  script:
    - gitlab.py <CDRouter-URL> <token> nightly $CI_COMMIT_BRANCH,$CI_PIPELINE_ID
  artifacts:
    when: always
    paths:
      - out/*.zip
    reports:
      junit: out/results_*.xml
  • cdrouter-test is the name of the job.
  • tags specifies which GitLab Runner this job can run on.
  • stage indicates that this job is part of the “test” stage.
  • script defines what will be run when this job starts.
    • <CDRouter-URL> is the URL to the CDRouter system. This must be reachable from the GitLab Runner.
    • <token> refers to the API token for the CDRouter System.
    • nightly is the example tag that specifies which package will be run. All packages that are tagged “nightly” will be started by this script.
    • $CI_COMMIT_BRANCH,$CI_PIPELINE_ID is an example of pipeline-specific variables that can be used as tags for the test results. In this example, CDRouter test results that are generated as a result of this script will be tagged with the branch name and the unique pipeline ID. This can help organize test results. More predefined variables can be found here.
  • artifacts defines what will be saved by GitLab with each pipeline run.
    • when must always be set to always in order for test results to be shown even on failing jobs.
    • paths tells GitLab to save all zip files that are present in the directory out/. This can be helpful if not all members of the development team have access to the CDRouter System.
    • reports must be defined to take advantage of the built in individual test case reporting within GitLab CI/CD.

This example uses a script written in python using the cdrouter.py module available on GitHub. The gitlab.py script is available as an example in the module.

GitLab Runner

GitLab CI/CD uses GitLab Runners to run the scripts defined in the .gitlab-ci.yml file. There is an option to use shared GitLab Runners or install the GitLab Runner software on systems that are locally managed. This example requires there to be a GitLab Runner available to the project that has network access to the CDRouter System. The GitLab Runner must also have python and the cdrouter.py package installed. Alternatively, the GitLab Runner software can be installed directly on the CDRouter System.

Using the Web API

CDRouter’s Web API provides full control over executing CDRouter test packages and managing results. QA Cafe has also developed a python module with the API to make interactions with CDRouter even easier.

In most cases, scripts used to drive CDRouter testing will call out to a CDRouter system to start one or more test packages. The script can wait until the tests are complete and then gather the final results. Scripts may also want to export CDRouter test results for a backup or perform other result processing.

Web API Tokens

When the CDRouter user model is enabled, the web API requires the use of an API token. Each user is assigned an API token automatically. It may be beneficial to create a user specifically for Jenkins automation. Please see the Authorization section of web API for more information.

Help Us Improve

The cdrouter.py python module is hosted on GitHub. We welcome push requests and feature suggestions to allow CDRouter to work better with GitLab and other CI tools.