Skip to content

Adding a new "validation" step to our issue handling workflow #130

Description

@iritkatriel

One of the suggestions coming out of the Language Summit discussion about "The Issue and PR Backlog" was to add a formal way for core developers (and perhaps also triagers who are subject matter experts) to "accept" or "validate" a reported issue. The meaning of validation is that the expert agrees that the issue should drive some kind of change in Python (a bugfix, a new feature or a documentation update).

This classification would help in the following ways:

  1. Contributors will be able to search for issues that are worth working on, because there is a core dev who is likely to review a PR and help push it forward.

  2. The triagers and core devs who are working through the backlog and closing invalid issues will be able to skip validated issues, and work on the ones that have not been classified as valid (or closed as invalid).

  3. It will give the OP and others more clarity about the status of an open issue. For example, when we ping on an issue like we did with the "stale" label, it will be possible to direct our pings to the right people. A valid issue is the responsibility of the core devs, while a yet to be validated one could perhaps benefit from more advocacy by the OP.

There was a fair amount of discussion in this thread and I believe there is general agreement on adding this new label, but we didn't quite reach consensus on a few details. There seems to be little interest among the core devs to discuss these last details further, so I would like to request that the SC try and make a final decision on them.

The details are:

  1. whether the label should be positive (added to the issue once it's validated) or negative (removed when the issue is validated).
  2. The name for the label.
  3. Applying the label to the existing open issues.

My proposal is to add a "valid" label (and to remove the existing "invalid" label, which people are finding too harsh and is actually not very useful for us, see the discussion on the issue). The valid label would be added to old and new issues in the same way, following analysis and a decision to validate the issue.

The alternative proposal is to add a label automatically to every newly opened issue, which is removed to signal validation. Names suggested for this label include "unacknowledged", "new", "unclassified" and a few others. I believe it would be necessary to add this label to all existing open issues so that they can be classified.

My preference is for the positive label "valid". I believe that negative-sounding labels added to an issue as soon as it is opened can be intimidating for new contributors. My proposal also avoids the shock of adding a new label to all open issues at once. (In the far future, when we have cleaned up the bug tracker to the point that almost all open issues have a valid label, we can reconsider).

However, the alternative proposal can work as well. You may also find a third option.

Thank you for your help.

Activity

  1. encukou commented on Jun 28, 2022

    @encukou
    Member

    I've added this to our agenda.

  2. ezio-melotti commented on Jun 28, 2022

    @ezio-melotti
    Member

    I believe it would be necessary to add this label to all existing open issues so that they can be classified.

    My plan isn't to add the label to all existing open issues, but to:

    • add it automatically to all new issues when they are created
    • add it to old issues that have no comments (or just comments from the OP)
    • leave old open issues alone, since most of them are already valid

    Old open issues can be revisited at any time and:

    • if they are invalid they can be closed
    • if they are valid they can be moved forward
    • if they need attention, the label can be added

    If we automatically ping issues like we did with the "stale" label, we can ping core-devs and they can (re)evaluate the issue in the way I just described.

    I believe that negative-sounding labels added to an issue as soon as it is opened can be intimidating for new contributors.

    I agree, and the list of previously-proposed names includes names like new, awaiting triage, needs triage, waiting-triager which are not negative-sounding. As suggested in a later message1, these can be tweaked to needs approval, awaiting approval, or pending approval (or something else, I'm open to other suggestions)2.

    Footnotes

    1. this message also summarizes the points of agreement and contention as of four days ago. ↩

    2. I haven't settled on a name yet because it depends on the semantics. new works well if the label only applies to new issues that are then either approved (and the label removed) or closed. If we want to apply the label to older issues while we discuss/decide whether to approve them or close them, then needs approval might be a better choice. For this case we could also use pending instead, and close it after a while unless we reach agreement on the validity of the issue. ↩

  3. encukou commented on Jul 12, 2022

    @encukou
    Member

    Let's go with the triaged label, see the thread.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions