Document and Design the Verification Process For Issues

[shawn-hurley]

Is there an existing issue for this?

• I have searched the existing issues

Konveyor version

All

Description

We need to design and document the process and the automation that will help us track and handle the verification of issues in the konveyor projects.

Today, we are leaving issues open and waiting for a verified label to be applied. This is a bit confusing though becasue the issue has a fix already in place, and typically you would have the issue closed at that point.

Additional Information

No response

[fabianvf]

I agree. I think it would probably make a lot more sense to add a label-based verification on top of the issue lifecycle. This would mean that as issues are fixed (as in, relevant PR merged), they can be closed as normal, but if it's a bug or something that definitely needs to be verified, we just ensure the issue has a needs-verification label. We could create separate views in the Planning project that allow us to see all issues (closed or open) tagged with needs-verification so that it's easy to see what's waiting for further testing. Then, when testing is completed, it can either have the needs-verification tag replaced with verified, or the issue can be reopened and live through the normal lifecycle again. Basically

1. Issue opened.
2. If it will require thorough testing, a needs-verification label is added as part of triage
3. The issue is worked as normal, a PR is opened + merged
4. The issue is closed
5. It shows up as a closed issue requiring verification on the QE triage board in the Planning project
6. It is tested
   a. If the issue is resolved, needs-verification is removed and verified is added
   b. If the issue is not resolved, it is reopened and sent back to the dev for further investigation

I think this gets us the benefits of a normal lifecycle process that is intuitive for contributors, with an easy to follow separate testing process that allows QE to direct its resources well.

[shawn-hurley]

I am a plus-one on this approach.

It seems that the needs-verification flag is added during triage by a team member. I think that, as an extra step in the triage process, we should also have a no-verification label, which is set during triage. This would ensure that we can review issues that have seemingly been triaged but have not been vetted to determine if they require verification.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity for 60 days.
It will remain open for visibility and reporting purposes.
Please comment if this issue is still relevant.