Handling old issues

@brainwane

Type: other

Ping @brainwane @gloomy-ghost @Hainish @J0WI @wonderchook but this discussion is open to anyone.

I want to have a discussion on what we should do about the large number of old (year+) issues.

Old issues are trickier to deal with than old PRs:

A PR can be evaluated by itself no matter how old it is, but old issues can refer to websites that have changed, a UI that has changed, the old XUL/XPCOM Firefox stuff, etc.
Our oldest issues are significantly older: we have many issues from 2014 and 2015, but our oldest PRs only go back to 2016.
Issue submitters vs PR submitters tend to submit fewer issues per person and be less familiar with HTTPS Everywhere, so the quality of the issues can be lower (not always though), and the same number of issues can generate more and longer conversations from maintainers. By the second point I mean that if you have 100 random open issues, you'll probably need to have close to 100 conversations because each issue is likely to come from a different person, whereas 100 random open PRs come from maybe 10 people, so you might get away with fewer or at least shorter conversations with each person.
Issues might be architectural, policy-based, forward-looking, or otherwise require (scarce) EFF input instead of (slightly less scarce) volunteer maintainer input.

So, what should we do?

One thing that would help is if EFF staff could go through their own open issues, especially ancient issues, and close or update them as appropriate. I don't want to put too much attention on these EFF issues -- there aren't a ton of them -- but there are more than a few.

@terrorist96

Another thing to do is to improve the workflow around the low effort "please add example.com, thanks!" type issues. These are good to have but they can add a lot of clutter to the issues page when domains are reported one at a time.

The closest thing to a collection of these issues we have is issue #3069. @terrorist96 has done a great job managing this list for years but nevertheless this situation has some problems. The list is managed by one person who specifically asks the maintainers not to edit the list, so if @terrorist96 disappears, the list just stops. @terrorist96 isn't formally associated with the project so there is a (general, not specific) security concern, for example they could randomly delete the entire list, or delete certain entries. Revisions to the list aren't tracked. They also disagree with including domains that redirect themselves, even though these domains are considered valid for inclusion by the project generally.

So I propose we do this:

Move the list in issue Sites that should have https forced #3069 into the GitHub wiki with restricted editing
Close issue Sites that should have https forced #3069 with sincere thanks to @terrorist96 for their work
Update the issue format bot's behavior. It would be great if the bot could add new domains to the wiki on its own. Failing that, have it attach a label to "new ruleset" issues so they can be found easily.
If needed, someone with wiki access periodically adds domains from issues into the wiki article and then closes those issues.
If a contributor makes a PR for a domain listed in the wiki, they ping the maintainers in the PR so someone can remove that domain from the wiki.

Thoughts?

Click to see the current checklist from the top comment of issue 3069

Outstanding:

Pending merge:

Completed:

Dead sites:

Also, the wiki should be mentioned by the issue format bot when contributors make new issues ("Be advised that new domains requested here will be transferred into the GitHub wiki at [URL here] and this issue will be closed").

Contributors should also be gently discouraged from creating PRs directly for domains that are still issues, instead of in the wiki, so as to avoid circumventing the issue-to-wiki process.

I don't think the wiki is a good idea. It vastly increases the workload of maintainers. It also makes impossible to add any useful information (for example, that only some paths can be rewritten).

Instead, each domain should be submitted in a separate issue and contributors should be incited to use the GitHub auto-closing functionality.

To prevent issues from cluttering the issue list, we should improve the bot. All rulesets issue should be tagged as such. I also noticed that most issue reporters don't follow the template or the bot instructions, which means that it should be revised (but that's another topic).

@Bisaloo

I probably shouldn't have added the discussion about domain tracking and the wiki. It's been discussed at length before (#6307, #6322). For example in #6322 (comment) over six months ago I estimate without proof that manually tracking issues is "15 minutes a month of manual work" which also applies to @Bisaloo 's comment about "vastly increases the workload of maintainers".

For whatever reason there seems to be no interest other than from me to use the wiki for this and I don't want to have that discussion again. For this discussion I'm more interested in ideas for resolving other types of issues.

Move the list in issue #3069 into the GitHub wiki with restricted editing

I don't think the wiki is a good idea. It vastly increases the workload of maintainers. It also makes impossible to add any useful information (for example, that only some paths can be rewritten).

I think moving the list to the wiki is a great improvement. To avoid additional maintenance overhead a bot can be used to update the wiki (the wiki is just another git repo, so this should work).
The current functionality (what is it?) of the bot is not very useful for me. I hope it will be more useful in future with tasks like this.

If needed, someone with wiki access periodically adds domains from issues into the wiki article and then closes those issues.

Currently every GitHub user should have access to the wiki. Let's see if it works.

If a contributor makes a PR for a domain listed in the wiki, they ping the maintainers in the PR so someone can remove that domain from the wiki.

Again, I hope a bot can do this. New contributors may not aware of the list in the wiki.

Edit:
</offtopic> I'm fine with closing PRs and moving them to the wiki. I currently see inactive PRs as a backlog and I'm happy if someone pickup the work again. But we can't just close issues without checking if they are obsolete. I would also avoid closing PRs or issues without any feedback. Otherwise we have issues or PRs that nobody want's to review and after some time they are just getting closed. That's not nice.

@J0WI

@J0WI

A big advantage to me of moving the domain list to the wiki is that it would be restricted-access to collaborators and above and maybe a few others. I want to avoid Wikipedia-style public edit wars over the list, or to have to watch it to make sure people aren't making malicious or simply bad edits.

As you know we have way too many issues/PRs and too few people reviewing them, so someone is not going to be happy.

Issues are a lot worse than PRs to resolve. They are often poorly written, or require a login or some complicated interaction with a site, or that you use some other extension, or whatever. You know what I mean and there are hundreds of examples open right now. And, to fix something that is actually broken will require a new PR.

Some sort of automated handling of old issues is essential. At a minimum we should have a bot go through and ask the issue contributor to confirm they are still active and that the problem still exists. If they don't respond soon, we close it. It's not ideal but that's probably the nicest way to start.

It needs to be a bot and not a person because otherwise the person who pings them can be put on the spot to help them out. I'm finding this as I ping people on old PRs (#13859). We should also only ping small groups of people at a time in case there is some overwhelming response.

@jeremyn

@jeremyn I think #3069 belongs in the wiki, it makes the most sense in dealing with clutter. As to @Bisaloo's concerns, if there are idiosyncrasies with a proposed ruleset, a table in the wiki can link to that issue for additional information. As long as the row is still present in the list of proposed rulesets, we can additionally close that issue as well. This would also be useful for de-duplication of proposed rulesets - a references column could be made which links out to multiple issues.

I do think that the complexity of such a bot increases - we're not just talking about parsing an issue and adding tags, we're now tasking the bot with editing a live document. That's bound to be more difficult than what we currently have in place. As a temporary measure, I propose we:

Take all issues that have been tagged with top-X and have an issue type of new ruleset and add the labels new-ruleset and ruleset to them.
Take all issues that have been tagged with top-X and have an issue type of ruleset issue and add the labels ruleset-bug and ruleset to them.*
Take any issues labelled with either new-ruleset, ruleset-bug, or ruleset-enhancement, and add the label ruleset to them.
Modify the issue-format-bot to add a ruleset enhancement type.
Modify the issue-format-bot to rename the ruleset issue type to ruleset bug
Ensure that any new issues of type:
6.1. new ruleset have the new-ruleset and ruleset labels applied, as well as the Alexa label.
6.2. ruleset bug have the ruleset-bug and ruleset labels applied, as well as the Alexa label.
6.3. ruleset enhancement have the ruleset-enhancement and ruleset labels applied, as well as the Alexa label.

This will unify the issue types from the template with the issue labels, and should make filtering easier, since ruleset would be a top-level selector for ruleset issues, with ruleset-bug, ruleset-enhancement, and new-ruleset being second-tier selectors.

I'm happy to get started on this work, pending feedback.

Forgot to explain the asterisk:

Not all issues of type ruleset issue will be bugs. Some will be enhancements. But currently, the template doesn't give issuers the choice to specify. Therefore, if some issues are miscategorized as ruleset-bug rather than ruleset-enhancement, this is probably acceptable. This being said, we should be clear what we mean by both of these labels. The way I see it, ruleset-bug is a more urgent categorization, and should ideally be reserved for only rulesets causing active problems on sites. ruleset-enhancement should be used to indicate inadequate coverage, for example if certain subdomains or paths are excluded due to legacy reasons, but they need not be anymore. In any case, we should explain the distinction in the issue template so issuers have enough knowledge to make the right choice.

I don't want my list to be moved. I started the list with sites I came across myself, so I could keep them all in one place. Slowly, others added other sites to the issue so I added them to my list as well if it fit my criteria. The issues section is the perfect place for it; it consist of things that should be added to HTTPSE. A wiki is for information, reading material, etc.

@Hainish

@Hainish

If we're going to use the wiki, I think we should do it manually for a while before bringing the bot into it.

There are only about 20 open issues with the top-X label so your items (1) and (2) won't really have much impact.

I disagree with having all these new-ruleset/ruleset-bug/ruleset-enhancement labels. Just ruleset is enough, or add ruleset/modify ruleset. Deciding between ruleset-bug and ruleset-enhancement is a hair-splitting argument I don't want to have with people, especially with the obvious connotation that ruleset-enhancement is lower priority. Imagine waiting for a year for some activity on your issue and then see that someone has downgraded your issue. Ugh.

Who uses the labels? People who feel like working on issues. I guess most people who feel like tackling some problems want to work on either code problems, or ruleset problems. Once they've decided which of these two categories they're interested in, they'll go through the relevant list of open issues themselves and figure out what to do. I doubt they will care much which are -bug and which are -enhancement.

@terrorist96

@terrorist96

Project issues aren't the best place to keep private lists. List issues are fine if they are limited in scope, for example issue #9842 "Audit default_off rulesets" is appropriate. But an open-ended issue about "Sites that I'm interested in" is not great.

There's also a need for an official list of "Sites that people want covered", and your list will compete with that, which is also not great. I recognize that some of that is my fault, since I've advocated using #3069 as a dumping ground for these domains for a while, since it's the closest thing we've had to an official list.

It's not a private list, it's public, available to anyone who wants to make new rulesets. Maybe you can pin it to the top of the issues list for more visibility. There won't be any competition between my list and any official list as long as no official list is made. Or go ahead and copy my list if you really want, but I would like to retain control over the list I've compiled over the years.

@terrorist96

@terrorist96 Either the list in #3069 is purely private to you, in which case it shouldn't be a GitHub issue, or the list is important to the project, in which case it should be under the control of people officially attached to the project, for the reasons I gave above in #13890 (comment).

Who says it has to be either or? Having it private to me is of no use. What am I gonna do with a private list if it's not available for others to see and make rules from it? Having it under the control of the maintainers of this repo is akin to taking my work. I said I'm fine if you wanna copy it; it's not a copyrighted list. Feel free to "fork" it. But what you're proposing is taking away my ownership of it, which I'm not ok with. By your logic "if it's important to the project, it should be under EFF's official control", then all issues posted in this repo should be under EFF's control.

To address your concerns:
If I disappear, you can still have your copy of the list.
If I delete certain entries, again, you can still have your copy of it.

Having the list in a repository seems reasonable. Maybe it should be in a new repository, not this one, to keep the number of commits down, and to limit the damage that an (unintentionally) misbehaving bot with commit access could do.

I'm closing this issue with these observations:

There is some interest is revisiting/improving how we process simple requests to add coverage for some domains, but not any clear consensus. Perhaps interested people can open specific issues or pull requests along those lines, for updating the bot or wiki or similar.

There has been little discussion here about solving the substantive problems in the current 500+ issue backlog.

I'd just like to add that the issue backlog is not only a burden from a technical point of view (it is hard to find the actual/current issues among the noise). This also has some real impact on HTTPS Everywhere image to potential users.

Of course, we are not trying to sell anything and technical relevance should always prevail over marketing. But the end goal of HTTPS Everywhere and (I assuming) most of its contributors is to make the web a safer place. If we can increase HTTPS adoption, we should strive for it. Increasing our userbase is one way to do so.

I've seen people linking to our bug tracker claiming this was a proof that HTTPS Everywhere broke way too many sites to be pratictal. While I don't deny we sometimes make mistakes, the huge outdated issue backlog blow this problem way out of proportions.

@jeremyn

Some ideas to fix this situation:

Rulesets maintainers and regular contributors should encourage new contributors to use GitHub auto-closing feature.
I agree with @jeremyn we need to review the current backlog and weed out the obviously outdated issues (eg. issues with preloaded domains, possibly issues related to HTTPS Everywhere observatory or old UI like enable sorting of "Enabled" column #703). Let me know if this is something I can help with.

@jeremyn

I agree with @jeremyn we need to review the current backlog and weed out the obviously outdated issues (eg. issues with preloaded domains, possibly issues related to HTTPS Everywhere observatory or old UI like #703). Let me know if this is something I can help with.

One possible community-oriented thing we can do to relieve the backlog of issues is to convene an HTTPS Everywhere hack-a-thon, reaching out to the technical community around EFF and gather those interested in participating in a little house-cleaning. I'm envisioning volunteers commenting on issues, soliciting feedback from those who originally opened them if there are still open questions, tagging ruleset maintainers where appropriate. This would drive engagement with the project and perhaps promote a longer-term useful participation.

For the hack-a-thon idea, if the EFF is going to arrange a burst of effort over a couple days, it should also organize its own maintainers for the event too. I don't want to discover one day that I've been pinged on a couple hundred issues.

@Hainish

Hey @Hainish, sorry I missed your comment above. In principal I am happy to work more on the bot, though I've been without a working laptop recently so it'd have to wait until that was repaired. I'm hoping to get it fixed soon.

I just looked and apparently there are already labels matching the bot's Type information which I hadn't noticed previously. I should probably update the bot to use those labels. This might help with this problem:

I've seen people linking to our bug tracker claiming this was a proof that HTTPS Everywhere broke way too many sites to be pratictal.

As then it would be more obvious that most of the issues are about new rulesets (since they'd be labeled as such).

I'm envisioning volunteers commenting on issues, soliciting feedback from those who originally opened them if there are still open questions, tagging ruleset maintainers where appropriate.

I wonder if we should make it so the bot allows people to do this on their own. I.e. you could say "@https-everywhere-bot type: new ruleset, domain: eff.org" and the bot would edit the issue description to add the metadata, then label the issue appropriately. That way maintainers wouldn't get a million pings. People could deliberately mess with the bot to add bad metadata but that seems kinda unlikely. especially because a human would notice right away? Plus we could turn it off again right after the hackathon or whatever

@strugee

@strugee, if you want, you could open a new issue to gather suggestions for the bot. I have some ideas but I am not sure if they are possible with the GitHub API.

@Hainish, a hackathon sounds exciting but I'd like to see the two following points solved beforehand:

The main issue here isn't a lack of contributors fixing bugs IMO (although more is of course better). We mainly need a clear policy on when it is acceptable to close issues. For example, should the issues related to the old UI I mentioned (enable sorting of "Enabled" column #703) be closed?
If we try to bring a large number of new contributors, I would be more comfortable if we have a code of conduct. In other words, I think we should wait until Add a code of conduct #10724 is closed to organize a such event.

I am not sure where to post this but another argument to clean up our tracker and standardize issue format: duckduckgo privacy essentials apparently uses it to determine which rulesets are broken:

https://github.com/duckduckgo/duckduckgo-privacy-extension/blob/4175b1874606c7ecda8c3116f84de8ab41ce892c/scripts/httpsWhitelist.js

@Hainish

On a related note: is is acceptable to ping rulesets maintainers or @Hainish if we find an issue that has been fixed server side?

@Bisaloo

@Bisaloo I'm not sure what you are asking.

Can we ping you to close an issue after we checked it is fixed?

@Bisaloo

@Bisaloo You mean issues that aren't fixed by a PR, that is, you want to ping us for issues that seem to have "fixed themselves" or that you can't reproduce?

The ideal case for any issue is for the issue reporter to close the issue themselves. So if an issue seems fixed, you can add a note to the issue saying you think it's fixed and then try to convince the reporter to close it.

If that doesn't go anywhere then I personally am not interested in getting a ping for it. Whoever looks at the issue notes later on will see what you wrote, and what the reporter wrote, and can decide what to do with it then, no ping needed.

@Bisaloo

@Bisaloo I'm happy to be pinged with issues that I've resolved, but haven't yet been closed (as you've been doing).

Navigation Menu

Repository navigation

Handling old issues #13890

18 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Handling old issues #13890

Description

Activity

jeremyn commented on Dec 6, 2017

jeremyn commented on Dec 6, 2017

Bisaloo commented on Dec 6, 2017

jeremyn commented on Dec 6, 2017

J0WI commented on Dec 8, 2017

jeremyn commented on Dec 9, 2017

Hainish commented on Dec 12, 2017

Hainish commented on Dec 12, 2017

terrorist96 commented on Dec 12, 2017

jeremyn commented on Dec 12, 2017

jeremyn commented on Dec 12, 2017

terrorist96 commented on Dec 12, 2017

jeremyn commented on Dec 13, 2017

terrorist96 commented on Dec 13, 2017

18 remaining items

jeremyn commented on Jan 31, 2018

jeremyn commented on Feb 8, 2018

Bisaloo commented on Feb 8, 2018

Bisaloo commented on Feb 8, 2018

Hainish commented on Feb 8, 2018

jeremyn commented on Feb 8, 2018

strugee commented on Feb 9, 2018

Bisaloo commented on Feb 11, 2018

Bisaloo commented on Feb 12, 2018

Bisaloo commented on Feb 18, 2018

jeremyn commented on Feb 18, 2018

Bisaloo commented on Feb 18, 2018

jeremyn commented on Feb 18, 2018

Hainish commented on Feb 20, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions