GD-981: Add context menu to "Run until failure"

[DevSavvySerge]

Feature Type

Test runner enhancement

Problem/Use Case

When facing a flaky test, it would be nice to have the ability to let the test runner run until it come to a failure case, allowing to get insight on the "unhappy path".

Proposed Solution

Have an additional contextual "Run until failure" button in the test explorer.

When clicked this button would run the selected test(s) until it ran with a failure.

Ideally, a maximum number of attempts could be set by user in the configuration panel.
A bit alike, the very opposite of this feature which is the rerun of flaky tests until success (tests should not be left nondeterministic).

Alternatives Considered

Run the button "Run test" over and over again, which, due to how tests' trees automatically collapses now and then, gets tedious quick.

Impact and Benefits

Helps in finding out which cases cause a test to fail.

Implementation Notes

If the feature is adopted and a maximum number of attempts can be set, displaying the current attempt number during the run is nice to have.

Examples and References

Not a new idea, exist in Visual Studio for instance:

Contribution

• I'm willing to contribute code for this feature
• I'm willing to help with testing/feedback
• I'm willing to help with documentation
• I would just like to request this feature

[MikeSchulze]

Thanks for providing this proposal 👍

Run the button "Run test" over and over again, which, due to how tests' trees automatically collapses now and then, gets tedious quick.

You can change the settings of auto collapse test tree