Better network errors when "block unencrypted requests" is enabled

Currently if you visit an HTTP site with HTTPS Everywhere blocking all unencrypted connections, you get an "unable to connect" error. It'd be nice if the extension did something more useful, like displaying a page that said "Unencrypted request blocked" or something.

I don't know the code where this happens, but from observation, "Block all unencrypted requests" seems to just rewrite unencrypted HTTP URLs to HTTPS. So it's not that the add-on is blocking the request, it's that server can't process it. You can see this in your "localhost" issue #8242 .

So this isn't really something HTTPS Everywhere can do, or if it is, it would require some weird detective work. I think this issue should be closed. What do you think?

@jeremyn

@jeremyn yeah, that seems to be what's happening. But I don't think it'll require a lot of detective work - after all, the extension knows when it's rewritten a URL. If it rewrites HTTP -> HTTPS and then notices a network error immediately after, it seems safe to assume it was due to the rewrite.

The add-on doesn't think about things in that way. It's not watching or tracking connections. It normally just watches outgoing requests, sees if the URL matches a regex in one of the rulesets, and if so it rewrites the URL however the matching ruleset wants. For "Block all unencrypted requests" mode I assume it just rewrites any leftover unencrypted HTTP requests to HTTPS.

Think about the localhost example again. If your computer isn't running any local server and you get an error at https://localhost, is "Unencrypted request blocked" appropriate? No, because it suggests that the plain HTTP request would have worked.

To implement your idea, the add-on would have to try the plain HTTP connection first and see what happens. This is bad for privacy and would require some sort of new tracking and evalution logic in the add-on that doesn't exist right now.

@jeremyn

@jeremyn maybe the error should be adjusted to "Upgrading unencrypted connection failed" or something like that. That being said I'd argue that "Unencrypted request blocked" doesn't imply HTTP would have worked - there would have been an outbound request no matter what had HTTPS Everywhere not "blocked" it. Whether or not the request would've succeeded is orthogonal.

If we added some text below "Unencrypted request blocked" that said something to the effect of, "it's possible that disabling 'block all unencrypted requests' and trying again will fix this issue," would you be OK with that?

A question is, what information do you want to user to learn from any of these messages? They already know the URL was rewritten because they can check the address bar or dev tools. They already know the HTTPS site doesn't work because it's reporting an error like "Invalid certificate" or "Unable to connect". I think what you are trying to tell the user is, "This request would have returned a valid response if HTTPS Everywhere hadn't rewritten the request from HTTP to HTTPS using the 'block everything' rule". Is that right?

They already know the URL was rewritten because they can check the address bar or dev tools.

Users won't think to do this. I'm a web developer and even I only barely thought to check the URL bar - getting an error on localhost when I knew my server was running was extremely confusing. Regular users will have no idea what was going on.

I think a large part of the difference in thinking here stems from the fact that AFAICT you're assuming that users will be able to figure out that HTTPS Everywhere's "blocking" mode actually just unconditionally rewrites everything to HTTPS. I don't think that's a valid assumption.

"This request would have returned a valid response if HTTPS Everywhere hadn't rewritten the request from HTTP to HTTPS using the 'block everything' rule".

Roughly, yes. Mostly I just want it to be clear to the user that the failure was caused by HTTPS Everywhere. I'm pushing for "Unencrypted request blocked" because I think that's the easiest to understand - even though it's not technically true in the sense that HTTP requests aren't being blocked, they're being rewritten, it matches the text of the option that the user enabled much better and is thus clearer. Or in other words, it's easier to understand even if it's not 100% accurate.

Right, so in order to know

"This request would have returned a valid response if HTTPS Everywhere hadn't rewritten the request from HTTP to HTTPS using the 'block everything' rule".

We need to gather this info:

Did the plain HTTP request work? yes/no
Did we rewrite the URL because of the fallback rule? yes/no
Did the HTTPS request work? yes/no

The error message we're discussing then displays if the answers are "yes", "yes", and "no". Is that right? If so, we have the privacy and tracking problems I discussed in #8239 (comment).

Another way of thinking about this is that:

Mostly I just want it to be clear to the user that the failure was caused by HTTPS Everywhere.

is not directly true. The request failed for any of the various reasons requests fail. The user would have gotten the same failure if they typed that HTTPS URL into the address bar themselves. "Blaming" the failure on HTTPS Everywhere requires the analysis and information gathering above.

@jeremyn

@jeremyn hey, so sorry. This fell off my radar for some reason.

I think you're missing the point I'm making. You're getting hung up on making sure that the error message is technically accurate, but what I am proposing is that we lie about knowing that an HTTP request would've worked, because it will be the truth 95% of the time. Thus, the first question above ("did the plain HTTP request work") is irrelevant.

The user would have gotten the same failure if they typed that HTTPS URL into the address bar themselves.

But they didn't. You're taking a very small and focused view of the world here. That's fine, but I'm stepping back and looking at the broader picture: if the user types http://example.com, the request will work. If the user types http://example.com and HTTPS Everywhere upgrades it, and https://example.com fails, HTTPS Everywhere has caused the error. It's cause and effect. The request would've worked unless HTTPS Everywhere's "blocking" mode was enabled, period.

Now, that doesn't mean that it's HTTPS Everywhere's fault. But it was caused by HTTPS Everywhere.

Do you see the distinction I'm making? Without the information gathering you've proposed, the error message will be accurate in 95% of cases, and I don't really think it's a big deal if it's not completely correct in the other 5%. It's a big usability win in the majority case versus a very minor inaccuracy in the minority.

I'm not sure what you are arguing for here. We are not going to rewrite a browser's error page to some error page we made up. I'm not even sure that that's technically possible, my guess/hope is that browsers don't let add-ons lie about page errors. At "best" we might add a suggestion in the extension UI that they could try disabling the rule or add-on. I'm against that too. Novice users should not be encouraged to disable parts of the add-on, and technical users will already know that they can.

We are not going to rewrite a browser's error page to some error page we made up.

This was what I was suggesting, yes. However if you're really against changing browser error pages (which I bet you can do in Firefox; I'd forgotten HTTPS Everywhere worked in Chrome too and assume it wouldn't work there) the same effect could be achieved by having a little informational popup from the icon that said "unencrypted request blocked" or something.

Novice users should not be encouraged to disable parts of the add-on, and technical users will already know that they can.

This argument does not apply when the relevant part of the addon comes disabled out-of-the-box.

To sum up, what you want is:

If a user has enabled "Block all unencrypted requests", and
The page fails to get a response because of "Unable to connect", "Secure connection failed", maybe a time-out, or any other connection-type failure, and
The URL was rewritten using the fallback "Block all unencrypted requests" rule, then
HTTPS Everywhere should generate (at minimum) a UI pop-up saying something like "Unencrypted request blocked".

Is that right? If so then I would say no. If you really disagree with that, then we can ask someone else.

@jeremyn

@jeremyn yes, that's correct. (I assume when you say "the URL" you mean the URL in the address bar, not a subresource URL.)

Why do you say no? It's a huge usability win IMO and I don't see any downsides.

@Hainish

I don't think the popup would be helpful and in fact, as we've discussed, it could be misleading. I don't know who its target audience is.

@Hainish Can you take a quick look at this feature request and let us know what you think? You can just look at #8239 (comment). I don't think we want this but I could be mistaken about its value.

I think this adds some complexity to the code without a whole lot of utility. Having the red icon I think should be enough to alert a user that the 'Block all unencrypted requests' is enabled. I might be inclined to include it if a PR is made, depending on how it's implemented, but I don't think it's an outstanding issue.

Hey, sorry for the delay (again). The red icon provides good information but isn't sufficiently noticeable to actually help - remember, users don't notice passive indicators (see for example The Emperor's New Security Indicators which, while about a different icon, demonstrates the same problem).

I also still stand by my point that if we phrase any warning carefully, we can make it not (egregiously) misleading while still matching the text that already exists in the icon menu ("block all unencrypted requests").

@strugee

If a popup says "HTTPS Everywhere has blocked this site", these users might disable the add-on and
never think to re-enable it. I think it would be better for them to think "this site just doesn't work" and
either work around it or talk to someone more technical who can help them.

This sounds like an argument in favor of misleading the user, which I think is a bad idea. BAHR will definitely cause a certain amount of breakage, and we have to empower the users to make a decision that's right for them.

We shouldn't hide or obfuscate the underlying error ("Invalid certificate" or "Connection refused" or "Secure connection failed" or "Time out"), for security reasons if nothing else.

But these aren't the underlying error. The underlying error is "you clicked an HTTP link, and BAHR is enabled, but there's no HTTPS available."

If we implemented @strugee's idea of "try before you redirect," and showed Chrome's built-in "blocked by extension" if that failed, would you still feel like that was hiding an error that should be shown?

I think we need to avoid words like "misleading". I've argued that if a request fails because a server has (say) an invalid certificate, then saying that the request was "blocked by HTTPS Everywhere" is literally false. The other position is that having HTTPS Everywhere configured in a certain way led directly to the failure and to not point that out is false by omission. Both positions have merit so it's unhelpful to call one misleading and the other not. (I've used the word "lie" in this discussion so I'm guilty of doing this myself.)

Another difficult word is "empowering". Information that a technical user finds useful, a nontechnical user can find confusing or intimidating. So the information is empowering to some users and the opposite to others. This is relevant because we're discussing whether to provide more or less information.

So instead of using these value-laden words, let's talk about outcomes. What problem are we solving? Who would benefit from this solution? Can the proposed solution cause other problems to other users? Is there any data, or even a collection of anecdotes beyond the three people in this discussion, to support a decision one way or the other?

What makes this specific issue different from many others is that we're talking about providing information to users that can only encourage them to use a less secure configuration. This means we need to be extra careful and especially sensitive to the needs of nontechnical users who generally do not participate in discussions on GitHub.

@jsha

To answer this specifically from @jsha:

If we implemented @strugee's idea of "try before you redirect," and showed Chrome's built-in "blocked by extension" if that failed, would you still feel like that was hiding an error that should be shown?

If HTTPS Everywhere is not actually blocking/dropping the request, then saying that it is blocking the request is not literally true. (Given my previous comment about value-laden words, I'm hesitant to use the word "hiding".)

@Bisaloo

@Bisaloo notified me that my issue #16046 was a duplicate of this one (thanks!). I'm re-stating my text here in the hopes that addresses some of the concerns raised (especially for an everyday use-case).

What it doesn't address is the request for data that demonstrates this is a worthy feature. Anecdotally, this case (an HTTP-only domain) occurs infrequently enough for me that I'm always surprised when it happens and waste time figuring out if the link I followed is valid, if I typed a domain correctly, etc, before realizing that it's an HTTP-only site. But I'm not sure how to provide real-world data to demonstrate that this is worth addressing (or not).

Is there any data indicating how many users use or prefer BAUR?

User story: I am an everyday user with "Block all unencrypted requests" ticked. I want to be notified when a valid domain has no HTTPS available, so I understand why it isn't loading.

Steps to reproduce:

Start browser with HTTPS Everywhere installed and "Block all unencrypted requests" ticked
Enter a valid domain in the address bar (where the domain is HTTP-only, no HTTPS version is available)
An example domain (at the time of writing) is wikiba.se

What should happen:

Notification that no HTTPS version is available

What happens instead:

After a very long time, browser reports that "The connection has timed out"
It appears as though the website is down, rather than it is up but not available over HTTPS

Suggested solution:

Display a simple message in the content window, similar to a browser message (not a pop-up or dialog box): "www.example.com is not accessible over HTTPS"

@kwill

@kwill In the above discussion, I argued against changing the current behavior. As far as I know, HTTPS Everywhere is in essentially the same state as it was during the earlier discussion and so my original points still stand. My concerns then were more about the fundamental nature of the problem posed, and whether it's appropriate for HTTPS Everywhere to try to solve it, or whether it could be solved without major changes to the extension.

As an example of the difficulty of this problem: how would HTTPS Everywhere distinguish between what you refer to as an "HTTP-only site", by which I guess you mean a site where the server for the domain does not listen on the HTTPS port or something similar, and a situation where an attacker in the middle of the connection is interfering with HTTPS traffic to that site? It realistically can't, so what else can HTTPS Everywhere do but let the request fail and let the user figure out what to do next?

I think "Block all unencrypted requests" should be considered an advanced, power-user type feature. Perhaps the answer to this issue is to more clearly label it that way, or otherwise emphasize that enabling it can cause unexpected problems with sites and the user should be prepared to do some occasional troubleshooting if they want to use it. Would that help, do you think?

@kwill

I think "Block all unencrypted requests" should be considered an advanced, power-user type feature

I think I might be considered a power user, since I used to be the primary maintainer of the extension. And I had exactly the same problem @kwill reports: When I used to use BAUR, I occasionally got timed out page loads, and I found these very confusing. I often wasted time trying to figure out the same things - did I mistype the domain name or something?

As a result, I never use BAUR anymore. I suspect that's a common result. @jeremyn you objected to showing a meaningful error message after a failed optimisic redirect. You were concerned that people who saw that error might turn off BAUR. I think that misses the broader picture: More people would use BAUR if it more reliably told you what was going on.

how would HTTPS Everywhere distinguish between what you refer to as an "HTTP-only site"

HTTPS Everywhere knows when it has performed an optimistic redirect. In those situations, it has no reason to believe that the site was available on HTTPS, because the original requested URL was HTTP. Perhaps it makes more sense to talk about displaying a meaningful error message after failed optimistic redirects?

@kwill

In @kwill 's wikiba.se example, when the https URL fails to load, the user has all available information: https://wikiba.se is in the address bar and "The connection timed out" is displayed. That's all HTTPS Everywhere knows about it, too. HTTPS Everywhere can't provide more information than that. It can speculate about the causes of the error or about how to avoid the error, and it can hide information, but it can't provide more information than the user already has available to them. So what should it do?

I think BAUR is unavoidably problematic, by the nature of what it does. Sites that 100% support HTTPS can opt into HSTS preloading. After that, HTTPS Everywhere's rulesets provide some additional coverage. URLs that aren't covered by HSTS preloading or an HTTPS Everywhere ruleset are the most likely to be broken with HTTPS, and turning on BAUR says you want to use HTTPS with those. So, problems should be expected.

Maybe what people find annoying is having a problem and then forgetting that they've turned BAUR on? Maybe we could provide a hint, if HTTPS Everywhere detects some problem, that they might have better luck if they disable BAUR and try again. On the other hand we should be very careful about advising users to make the extension less secure.

Also, I suspect many users are unaware that the way HTTPS Everywhere is intended to work is by using its extensive list of rulesets. Perhaps some users download the extension, then see an unchecked option "Block all unencrypted requests" and think, "Yes, of course I want that, that's why I downloaded an extension called 'HTTPS Everywhere'", and check it without realizing they are basically opting into problems as I described in my previous comment.

We could add some documentation around the BAUR checkbox, maybe a tooltip, or else move the checkbox out of the main UI and into "Preferences" > "Advanced Settings".

(We already have "Enable mixed content rulesets" under "Preferences" > "Advanced Settings", and BAUR is more or less -- but not exactly -- a stronger version of that.)

@jeremyn

Thanks for all your comments @jeremyn!

I agree that BAUR as an advanced option, but I'm cautious about hiding it deeper in the interface. That may make it even harder to realise, "Doh! I switched BAUR on!" Unless there is some kind of hint, as you suggested. I'm not sure what other hint mechanism to suggest.

Notes:

I think the hint message should be something more like "HTTPS is not available" or "Try HTTP" rather than "Disable BAUR".
When I knowingly encounter an HTTP-only domain I don't disable BAUR, instead I open it in my "insecure" browser profile. And maybe nag the webmaster to use Let's Encrypt :D
Would it be feasible to add a new type of ruleset for "HTTP-only" domains? For example, I'd add a wikiba.se rule, and this could trigger the warning right away. We couldn't cover every URL in existence, but would reach at least the popular ones. And build a shame list at the same time :D

The HTTPS Everywhere icon is red when you have BAUR mode enabled, so there is a sort of hint already visible. I don't mean to rule out other improvements though.

I'm not clear on the value of a message "HTTPS is not available" when the user is already looking at a "The connection timed out" error, or of the value in wrapping the time out message with the more generic message. If we agree that BAUR is an advanced feature, then hiding information is going in the wrong direction. Also I'm not sure wrapping a browser error is possible, for example see my earlier comment #8239 (comment).

I don't like advising users to "Try HTTP", which is to advise them to do something less secure. An important use case of the extension is on a system of a non-technical user which has been pre-configured by a more technical user. Think of a non-technical reporter or activist who has a device configured by their organization's IT department in some safe space and then takes it into a war zone. A message to "Try HTTP", if followed, could put that user in actual danger.

The HTTPS Everywhere icon is red when you have BAUR mode enabled, so there is a sort of hint already visible.

Maybe it belongs to a separate issue but it's worth noting that this visual hint does not exist for the android version of Firefox.

@jeremyn

Thanks for the input on this issue @jeremyn @Bisaloo @kwill @jsha.

I think we should add a warning for clarification when unencrypted resources are blocked. I've drafted some text in #16053 that I think gives ample warning to the dangers of network attacks when disabling this option, but lets users know that HTTPS Everywhere is doing the blocking. This I think strikes a nice balance between safety and usability.

Closed by #16053

Navigation Menu

Repository navigation

Better network errors when "block unencrypted requests" is enabled #8239

5 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Better network errors when "block unencrypted requests" is enabled #8239

Description

Activity

jeremyn commented on Jan 27, 2017

strugee commented on Jan 27, 2017

jeremyn commented on Jan 27, 2017

strugee commented on Jan 28, 2017

jeremyn commented on Jan 28, 2017

strugee commented on Jan 28, 2017

jeremyn commented on Jan 28, 2017

strugee commented on Feb 20, 2017

jeremyn commented on Feb 20, 2017

strugee commented on Feb 20, 2017

jeremyn commented on Feb 20, 2017

strugee commented on Feb 20, 2017

jeremyn commented on Feb 20, 2017

Hainish commented on Feb 21, 2017

strugee commented on Feb 27, 2017

5 remaining items

jsha commented on Mar 9, 2017

jeremyn commented on Mar 9, 2017

jeremyn commented on Mar 9, 2017

lofidevops commented on Jul 9, 2018

jeremyn commented on Jul 9, 2018

jsha commented on Jul 9, 2018

jeremyn commented on Jul 10, 2018

jeremyn commented on Jul 10, 2018

jeremyn commented on Jul 10, 2018

lofidevops commented on Jul 10, 2018

jeremyn commented on Jul 11, 2018

Bisaloo commented on Jul 11, 2018

Hainish commented on Jul 12, 2018

Hainish commented on Jul 19, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions