Using CDRouter with GitLab CI/CD
GitLab is a popular DevOps tool used to manage the workflow versioning, building and testing software. In a CPE firmware build environment, CDRouter can be combined with GitLab’s CI/CD features to provide testing verification of each firmware build.
GitLab CI/CD has built in concepts for testsuites and testcases. It natively recognizes the Junit XML test result format. While not all CDRouter test concepts directly map to GitLab CI/CD, it is possible to translate CDRouter test results into Junit XML test results that GitLab can understand.
The GitLab CI/CD pipeline is divided into multiple stages, and within each
stage, individual jobs. Each of these is defined in a YAML script, usually
.gitlab-ci.yml and stored in the root directory of a repository. You
can read more about CI/CD configuration in gitlab’s documentation
site; this document will focus on defining a CDRouter
test job in the test stage of the pipeline. The build stage needs to install and
deploy the current firmware version onto a CPE that is setup to be tested
against CDRouter. The steps to set this up are implementation-specific and
outside the scope of this document. For more details about setting up a CPE for
testing, see the CDRouter Quick Start
In the following example, the
.gitlab-ci.yml file defines a Python script that
uses the CDRouter Web API to start one or more test packages. The script waits
for the packages to complete, prints the results and creates a Junit file that
GitLab can parse to display the results.
Each time a GitLab pipeline is run CDRouter will run one or more packages against the latest firmware build, and include the test results in the pipeline report. This brings immediate visibility to any firmware change that affects a CDRouter test.
The following shows an example test job in a
cdrouter-test: tags: - cdrouter-gitlab-runner stage: test script: - gitlab.py <CDRouter-URL> <token> nightly $CI_COMMIT_BRANCH,$CI_PIPELINE_ID artifacts: when: always paths: - out/*.zip reports: junit: out/results_*.xml
cdrouter-testis the name of the job.
tagsspecifies which GitLab Runner this job can run on.
stageindicates that this job is part of the “test” stage.
scriptdefines what will be run when this job starts.
<CDRouter-URL>is the URL to the CDRouter system. This must be reachable from the GitLab Runner.
<token>refers to the API token for the CDRouter System.
nightlyis the example tag that specifies which package will be run. All packages that are tagged “nightly” will be started by this script.
$CI_COMMIT_BRANCH,$CI_PIPELINE_IDis an example of pipeline-specific variables that can be used as tags for the test results. In this example, CDRouter test results that are generated as a result of this script will be tagged with the branch name and the unique pipeline ID. This can help organize test results. More predefined variables can be found here.
artifactsdefines what will be saved by GitLab with each pipeline run.
whenmust always be set to
alwaysin order for test results to be shown even on failing jobs.
pathstells GitLab to save all zip files that are present in the directory
out/. This can be helpful if not all members of the development team have access to the CDRouter System.
reportsmust be defined to take advantage of the built in individual test case reporting within GitLab CI/CD.
GitLab CI/CD uses GitLab Runners to run the scripts defined in the
.gitlab-ci.yml file. There is an option to use shared GitLab Runners or
install the GitLab Runner software on systems that are locally managed. This
example requires there to be a GitLab Runner available to the project that has
network access to the CDRouter System. The GitLab Runner must also have python
and the cdrouter.py package installed. Alternatively, the GitLab Runner software
can be installed directly on the CDRouter System.
CDRouter’s Web API provides full control over executing CDRouter test packages and managing results. QA Cafe has also developed a python module with the API to make interactions with CDRouter even easier.
In most cases, scripts used to drive CDRouter testing will call out to a CDRouter system to start one or more test packages. The script can wait until the tests are complete and then gather the final results. Scripts may also want to export CDRouter test results for a backup or perform other result processing.
When the CDRouter user model is enabled, the web API requires the use of an API token. Each user is assigned an API token automatically. It may be beneficial to create a user specifically for Jenkins automation. Please see the Authorization section of web API for more information.
The cdrouter.py python module is hosted on GitHub. We welcome push requests and feature suggestions to allow CDRouter to work better with GitLab and other CI tools.