Observing subset behavior
Sometimes teams want to observe the potential impact and behavior of running subsets in a real environment before they start using them. In other words, they want to measure subsets' real world efficacy against the simulation shown in Choosing a subset optimization target.
You can do this using observation mode, which is a special usage mode of
launchable subset \
... [other options]
When observation mode is enabled for a test session, the output of each
launchable subsetcommand made against that test session will always include all tests, but the recorded results will be presented separately so you can compare running the subset against running the full suite.
For example, let's imagine you have a test suite with 100 tests that each take 1 second to run (100 seconds in total):
- By default, if you requested a subset of this test suite with a 30% duration optimization target, the subset output would include 30 tests.
- However, with observation mode enabled, if you requested a subset of this test suite with a 30% duration optimization target, the subset output would include all 100 tests. When you record results, Launchable will recognize this "full session" as a subset observation session.
Because you marked the session as an observation session, Launchable can analyze what would have happened if you had actually run a subset of tests, such as whether the subset would have caught a failing session, and how much time you could have saved by running only the subset of tests: