Non capitalized HTTP headers

[ninoseki]

Sorry for a dumb question.

I have to work with an API server which deals HTTP headers in case-sensitive manner.
(Yep I know that is a RFC violation)

Is there a good way to send non-capitalized HTTP headers?

[tarcieri]

I believe this is a dupe of #337, but I'll let @ixti confirm that.

This gem canonicalizes both request and response headers.

We've discussed disabling the canonicalization for requests, which I think is the most straightforward solution. We could also make it configurable on a request and response basis, e.g.:

HTTP.canonicalize_headers(false)
HTTP.canonicalize_headers(:request)
HTTP.canonicalize_headers(:response)
HTTP.canonicailze_headers(:all)

...or thereabouts.

[ixti]

Yeah it's a dupe. And I'm going to work on this soon. Finally have an idea in my head how to provide predictable API without intorducing public API changes. Will layout the idea as code soon so that discussion will be started :D But in few words, my idea is to change Headers class to keep header names as-is but using normalized names for headers lookup:

# imagine repsonse was sent with headers:
# foo_bar: 123
response.headers["Foo-Bar"] # => ["123"]
response.headers["foo_bar"] # => ["123"]
response.headers.keys # => ["foo_bar"]

When passing headers, that's pretty much the only public API change that will be needed:

1. Fail if given header name is neither String nor Symbol
2. Pre-normalize header name if it's given as Symbol

[tarcieri]

@ixti neat! SGTM 👍

[brasic]

Hi! Just came across this issue while checking to see if anyone had run into the problem we are currently dealing with. An API we need to communicate with requires a particular header be specified with an underscore in the header name. It's otherwise case-insensitive. HTTP.rb makes it impossible to pass a header in an underscore because of this line, which transforms it to -:

http/lib/http/headers.rb

Line 206 in e31d0a5

normalized = name.split(/[\-_]/).each(&:capitalize!).join("-")

I would argue that the behavior of HTTP::Headers#normalize_header violates RFC 7230, which allows the _ character to be part of a header name. We can monkeypatch this for now on our side but are there any thoughts on making the canonicalization algorithm spec-compliant? I don't know of any case-insensitivity scheme that considers _ and - to be equivalent. I'd be happy to open a PR to remove _ from the split regex but that would likely require a major version bump (looking at the specs, lots of code depends on _ canonicalizing as -).

[ixti]

@brasic yes - I'm working on refactoring HTTP::Headers completely so that it will llow to pass any RFC compliant header (with or without normalization).

[Hendrione-Moka]

any update for this? I still got the issue.

[ixti]

I had no time to work on this yet.

[joshuaflanagan]

Just ran into this again, dealing with an API that has case-sensitive headers. That may be "wrong" on their side, but it is what it is, and switching http libraries is much easier than getting an external API to change their behavior.
Do you have an in-process branch? I'd be happy to take a stab at fleshing it out. Would rather do that than switch libraries.

[tarcieri]

@joshuaflanagan I think if you implemented the change @ixti suggested here it'd be accepted, and shouldn't be too difficult: #524 (comment)