pytest

This is a reference page. See Getting started, Sending data to Launchable, and Subsetting your test runs for more comprehensive usage guidelines.
Recording test results
When you run tests, create a JUnit XML test report using the --junit-xml option, e.g.:
1
pytest --junit-xml=test-results/results.xml
Copied!
If you are using pytest 6 or later, please specify junit_family=legacy as the report format. pytest has changed its default test report format from xunit1 to xunit2 since version 6. See Deprecations and Removals — pytest documentation. The xunit2 format does not output the file name in the report, and the file name is required to use Launchable.
Then, after running tests, point the CLI to your test report file(s) to collect test results and train the model:
1
launchable record tests --build <BUILD NAME> pytest ./test-results/
Copied!
You might need to take extra steps to make sure that launchable record tests always runs even if the build fails. See Always record tests.
Subsetting your test runs
The high level flow for subsetting is:
1.
Get the full list of test paths and pass that to launchable subset with an optimization target for the subset
2.
launchable subset will get a subset from the Launchable platform and output that list to a text file
3.
Pass the text file into your test runner to run only those tests
To retrieve a subset of tests, first list all the tests you would normally run and pass that to launchable subset:
1
 pytest --collect-only  -q | launchable subset \
2
  --build <BUILD NAME> \
3
  --confidence <TARGET> \
4
  pytest > launchable-subset.txt
Copied!
The --build should use the same <BUILD NAME> value that you used before in launchable record build.
The --confidence option should be a percentage; we suggest 90% to start. You can also use --time or --target; see Subsetting your test runs for more info.
This creates a file called launchable-subset.txt that you can pass into your command to run tests:
1
pytest --junit-xml=test-results/subset.xml $(cat launchable-subset.txt)
Copied!

NUnit

Robot

Last modified 10d ago

Copy link

Contents

Recording test results

Subsetting your test runs