Format test reports
Results on the Tests tab are parsed from test reports specified in the Report Paths setting in Run and Test steps. Test reports must be in JUnit XML format to appear on the Tests tab, because Harness parses test reports that are in JUnit XML format only.
For optimal rendering in the Harness UI, there is a limit of 8,000 characters per field. If a field in your XML file contains more than 8,000 characters, the output might render incorrectly on the Tests tab.
JUnit XML format resources
Use these resources to learn about JUnit XML formatting.
- Common JUnit XML format and examples from Testmo
- Examples of JUnit XML format mapping from IBM
- Apache Ant JUnit schema
Tools with built-in JUnit XML output
Here are some Harness YAML examples for test tools that produce JUnit XML output by default.
C, C++
CTest Output JUnit
You can use the --output-junit command with CTest.
- step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
mkdir build
cmake -S . -B build
ctest --test-dir build --output-junit out.xml
reports:
type: JUnit
spec:
paths:
- /harness/build/out.xml
Java - Gradle
Gradle
Run step:
- step:
type: Run
...
spec:
shell: Sh
command: |-
test --tests
...
reports:
type: JUnit
spec:
paths:
- "/harness/results.xml"
type: JUnit
Test step:
- step:
type: Test
name: Intelligent Tests
identifier: test
spec:
command: test --tests
shell: Sh
...
reports:
- "/harness/results.xml"
PHP
PHPUnit
- step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
mkdir -p /harness/phpunit
phpunit --log-junit /harness/phpunit/junit.xml tests
reports:
type: JUnit
spec:
paths:
- /harness/phpunit/junit.xml
Python
For Python, use pytest or unittest. You can also use pytest to run unittest.
Pytest in a Run step
This example runs pytest in a Run step.
- step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
. venv/bin/activate
mkdir /harness/test-results
pytest --junitxml=harness/test-results/junit.xml
reports:
type: JUnit
spec:
paths:
- /harness/test-results/junit.xml
If you use test splitting with pytest in a Run step, you must set junit_family=xunit1 in your code repo's pytest.ini file or include -o junit_family="xunit1" in the step's command.
Pytest with Test Intelligence (Test step)
This example runs pytest with Test Intelligence.
- step:
type: Test
name: Intelligent Tests
identifier: test
spec:
command: |-
python3 -m venv .venv
. .venv/bin/activate
python3 -m pip install -r requirements/test.txt
python3 -m pip install -e .
pytest --junitxml=out_report.xml
shell: Python
...
reports:
- out_report.xml*
Ruby - Cucumber
Cucumber
- step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
bundle check || bundle install
mkdir -p /harness/cucumber
bundle exec cucumber --format junit --out /harness/cucumber/junit.xml
reports:
type: JUnit
spec:
paths:
- /harness/cucumber/junit.xml
JUnit converters, formatters, and plugins
If your test tool doesn't automatically produce test results in JUnit XML format, there are JUnit converters, formatters, and plugins available for all major languages. Some examples of conversion tools and corresponding Harness YAML are provided below.
C# - .NET Core, NUnit
Example: NUnit to JUnit
- step:
type: Run
identifier: test
name: Test
spec:
shell: Powershell
command: |-
cd dotnet-agent/TestProject1
wget -UseBasicParsing https://dot.net/v1/dotnet-install.ps1 -o dotnet-install.ps1
.\dotnet-install.ps1
dotnet build
wget https://raw.githubusercontent.com/nunit/nunit-transforms/master/nunit3-junit/nunit3-junit.xslt -o nunit3-junit.xslt
"C:/Program Files (x86)/NUnit.org/nunit-console/nunit3-console.exe" dotnet-agent/TestProject1/bin/Debug/net48/TestProject1.dll --result="UnitTestResults.xml;transform=nunit3-junit.xslt"
reports:
type: JUnit
spec:
paths:
- UnitTestResults.xml
Clojure
Go
Go Junit Report
You can use the go-junit-report tool.
- step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
go install github.com/jstemmer/go-junit-report/v2@latest
go test -v ./... | tee report.out
cat report.out | $HOME/go/bin/go-junit-report -set-exit-code > report.xml
reports:
type: JUnit
spec:
paths:
- report.xml
Java - Maven
Maven Surefire Plugin
This example uses the Maven Surefire Plugin.
Run step:
- step:
type: Run
...
spec:
shell: Sh
command: |-
test -Dmaven.test.failure.ignore=true -DfailIfNoTests=false
...
reports:
type: JUnit
spec:
paths:
- "target/surefire-reports/*.xml"
type: JUnit
Test step:
- step:
type: Test
name: Intelligent Tests
identifier: test
spec:
command: test -Dmaven.test.failure.ignore=true -DfailIfNoTests=false
shell: Sh
...
reports:
- "target/surefire-reports/*.xml"
JavaScript
ESLint
- step:
type: Run
name: Run ESLint Tests
identifier: run_eslint_tests
spec:
shell: Sh
command: |
mkdir -p /harness/reports
eslint ./src/ --format junit --output-file /harness/reports/eslint.xml
reports:
type: JUnit
spec:
paths:
- "/harness/reports/eslint.xml"
Jest
- step:
type: Run
name: Run Jest Tests
identifier: run_jest_tests
spec:
shell: Sh
command: |
yarn add --dev jest-junit
jest --ci --runInBand --reporters=default --reporters=jest-junit
envVariables:
JEST_JUNIT_OUTPUT_DIR: "/harness/reports"
reports:
type: JUnit
spec:
paths:
- "/harness/reports/*.xml"
Karma
- step:
type: Run
name: Run Karma Tests
identifier: run_karma_tests
spec:
shell: Sh
command: |
npm install
mkdir /harness/junit
karma start ./karma.conf.js
envVariables:
JUNIT_REPORT_PATH: /harness/junit/
JUNIT_REPORT_NAME: test-results.xml
reports:
type: JUnit
spec:
paths:
- "/harness/junit/test-results.xml"
Mocha
- step:
type: Run
name: Run Mocha Tests
identifier: run_mocha_tests
spec:
shell: Sh
command: |
npm install
mkdir /harness/junit
mocha test --reporter mocha-junit-reporter --reporter-options mochaFile=./path_to_your/file.xml
reports:
type: JUnit
spec:
paths:
- "/harness/junit/test-results.xml"
Ruby - Minitest, RSpec
Minitest
Add the Minitest Junit Formatter to your Gemfile.
- step:
type: Run
name: Run Ruby Tests
identifier: run_ruby_tests
spec:
shell: Sh
command: |
bundle check || bundle install
bundle exec rake test --junit
reports:
type: JUnit
spec:
paths:
- "/harness/report.xml"
RSpec
Add the RSpec JUnit formatter to your Gemfile.
- step:
type: Run
name: Run RSpec Tests
identifier: run_rspec_tests
spec:
shell: Sh
command: |
bundle check --path=vendor/bundle || bundle install --path=vendor/bundle --jobs=4 --retry=3
mkdir /harness/rspec
bundle exec rspec --format progress --format RspecJunitFormatter -o /harness/rspec/rspec.xml
reports:
type: JUnit
spec:
paths:
- "/harness/rspec/rspec.xml"
Code coverage reports and test report artifacts
For information about code coverage reports and publishing report URLs to the Artifacts tab, go to Code Coverage.
Troubleshoot test reports
Go to the CI Knowledge Base for questions and issues related to test reports in Harness CI, including: