Links

Gradle

This page outlines how the Launchable CLI interfaces with Gradle.
This is a reference page. See Getting started, Sending data to Launchable, and Subsetting your test runs for more comprehensive usage guidelines.
If you are using TestNG with Gradle, refer to another page for more robust integration.

Recording test results

After running tests, point the CLI to your test report files to collect test results and train the model:
launchable record tests --build <BUILD NAME> gradle ./build/test-results/test/
  • By default, Gradle's report files are saved to build/test-results/test/, but that might be different depending on how your Gradle project is configured.
  • You can specify multiple directories if you do multi-project build.
  • For a large project, a dedicated Gradle task to list up all report directories might be convenient. See the upstream documentation for more details and insights.
  • Alternatively, you can specify a glob pattern for directories or individual test report files (this pattern might already be specified in your pipeline script for easy copy-pasting), e.g. gradle **/build/**/TEST-*.xml.
You might need to take extra steps to make sure that launchable record tests always runs even if the build fails. See Always record tests.

Subsetting your test runs

For instructions on how to implement Zero Input Subsetting, scroll down to Using Zero Input Subsetting.
The high level flow for subsetting is:
  1. 1.
    Get the full list of test directories/tests/test paths and pass that to launchable subset with an optimization target for the subset
  2. 2.
    launchable subset will get a subset from the Launchable platform and output that list to a text file
  3. 3.
    Pass the text file into your test runner to run only those tests
To retrieve a subset of tests, first list all the directories containing tests you would normally run (e.g. project1/src/test/java) and pass that to launchable subset:
launchable subset \
--build <BUILD NAME> \
--confidence <TARGET> \
gradle \
project1/src/test/java \
project2/src/test/java > launchable-subset.txt
  • The --build should use the same <BUILD NAME> value that you used before in launchable record build.
  • The --confidence option should be a percentage; we suggest 90% to start. You can also use --time or --target; see Choosing a subset optimization target for more info.
This creates a file called launchable-subset.txt. For Gradle, this file is formatted like:
--tests MyTestClass1 --tests MyTestClass2 ...
You can pass this into your command to run only the subset of tests:
gradle test $(cat launchable-subset.txt)
# equivalent to gradle test --tests MyTestClass1 --tests MyTestClass2 ...
Note: The Gradle plugin for Android requires a different command, because the built-in test task does not support the --tests option. Use testDebugUnitTest or testReleaseUnitTest instead:
./gradlew testDebugUnitTest $(cat launchable-subset.txt)
# or
./gradlew testReleaseUnitTest $(cat launchable-subset.txt)

Using Zero Input Subsetting

To use Zero Input Subsetting, follow these instructions.
First, you need to add a snippet to your Gradle config to enable test exclusion via the Gradle command line:
test {
if (project.hasProperty('excludeTests')) {
exclude project.property('excludeTests').split(',')
}
}
Then, to retrieve a list of non-prioritized tests (per Zero Input Subsetting), run:
launchable subset \
--build <BUILD NAME> \
--confidence <TARGET> \ # or another optimization target
--get-tests-from-previous-sessions \
--output-exclusion-rules \
gradle . > launchable-exclusion-list.txt
  • The --build should use the same <BUILD NAME> value that you used before in launchable record build.
  • The --confidence option should be a percentage; we suggest 90% to start. You can also use --time or --target; see Choosing a subset optimization target for more info.
This creates a file called launchable-exclusion-list.txt. For Gradle, this file is formatted like:
-PexcludeTests=com/example/FooTest.class,com/example/BarTest.class
You can pass this into your command to exclude the non-prioritized tests. This will make sure only prioritized and new tests are run:
gradle test $(cat launchable-exclusion-list.txt)
# equivalent to gradle test -PexcludeTests=com/example/FooTest.class,com/example/BarTest.class
Note: The Gradle plugin for Android requires a different command, because the built-in test task does not support the --tests option. Use testDebugUnitTest or testReleaseUnitTest instead:
./gradlew testDebugUnitTest $(cat launchable-exclusion-list.txt)
# or
./gradlew testReleaseUnitTest $(cat launchable-exclusion-list.txt)

Command is too large for the exclusion list

If the exclusion list is very large, it may not be able to specify the list from the command directly. In that case, you can change the Gradle config to read from launchable-exclusion-list.txt.
Change the Gradle config as follows:
test {
if (project.hasProperty('excludeTestsTxt')) {
exclude new File(project.property('excludeTestsTxt')).text.replaceFirst('-PexcludeTests=', '').trim().split(',')
}
}
Then, specify the exclusion tests file from the command.
gradle test -PexcludeTestsTxt=$PWD/launchable-exclusion-list.txt

Bare output

By default, the CLI outputs a list of classes each with a prefix (--tests or -PexcludeTests= depending on the above). However, in some cases, a raw list of class names is preferable. Use the --bare option to enable this formatting. Note that this option goes after the gradle string:
launchable subset \
--build <BUILD NAME> \
--confidence <TARGET> \ # or another optimization target
gradle \
--bare \
. > subset.txt

Example integration to your CI/CD

GitHub Actions

You can easily integrate to your GitHub Actions pipeline.
name: gradle-test-example
on:
push:
branches: [main]
env:
LAUNCHABLE_TOKEN: ${{ secrets.LAUNCHABLE_TOKEN }}
LAUNCHABLE_DEBUG: 1
LAUNCHABLE_REPORT_ERROR: 1
jobs:
tests:
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./
steps:
- uses: actions/[email protected]
- uses: actions/setup-[email protected]
# You need JDK 1.8.
- name: Set up JDK 1.8
uses: actions/setup-[email protected]
with:
java-version: 1.8
- name: Run test
run: |
# Install launchable CLI.
python -m pip install --upgrade pip
pip install wheel setuptools_scm
pip install launchable
# Verify launchable command.
launchable verify
# Record build name.
launchable record build --name ${{ github.sha }} --source src=.
# Subset tests up to 80% of whole tests.
launchable subset --target 80% --build ${{ github.sha }} gradle src/test/java > subset.txt
# Run subset test and export the result to report.xml.
gradle test $(cat subset.txt)
# Record test result.
launchable record tests --build ${{ github.sha }} gradle ./build/test-results/test