Multiple report files

Combining parallel test runners or multiple test suites

Single report file

By default TestRecall looks for a single report file, in a standard setup:

curl -o reporter https://get.testrecall.com/r/latest/reporter.linux-amd64
chmod +x reporter
trap './reporter' EXIT

npm run test # => output report.xml

TestRecall checks for files that look like junit xml (e.g. report*.xml, junit*.xml). This behavior can be overriden by specifically passing the file name and path:

./reporter -file /tmp/test-results/my-report-file.xml

Multiple report files

TestRecall uses a -multi flag for multiple test reports that need to be grouped together, with the values:

  • before
  • partial
  • after

Before

Before results are uploaded call ./reporter -multi before to let TestRecall know there will be multiple reports grouped together and to wait until they are all completed.

During tests

Upload partial reports using ./reporter -multi partial. This also accepts the -file flag for filenames.

After tests

After results are uploaded call ./reporter -multi after to tell TestRecall to group the partial results together into the complete report.

multiple_tests_success

Handling failures

If a parallel or multiple test suite fails, the final report will not be completed until after has been called. For example if after is called in the final stage of a CI pipeline that depends on the previous stages succeeding - then failed tests reports will not be uploaded.

multiple_tests_failed

In this example the test suite for report_2.xml failed, stopping the CI pipeline. To make sure tests are still reported, we need to use the after flag instead of partial, to signal that we have completed running tests.

Example of a TravisCI pipeline:

script:
  - bundle install
  - npm run test # output junit report.xml file

before_script:
  - curl -o reporter https://get.testrecall.com/r/latest/reporter.linux-amd64
  - chmod +x reporter

after_success:
  - ./reporter -multi partial

after_failure:
  - ./reporter -multi after

stages:
  - setup
  - test
  - upload

jobs:
  include:
    - stage: setup
      script: ./reporter -multi before
    - stage: upload
      script: ./reporter -multi after