MONGOCRYPT-841 add signed binaries for Github release

[kevinAlbs]

Summary

Add an upload_release task to upload smaller signed tarballs.

Evergreen patch: https://spruce.corp.mongodb.com/version/69d8dca860bf2400076c5dc4. This patch was run by temporarily removing the patchable: false option.

Background & Motivation

This PR is intended to help migrate drivers from direct S3 URLs to GitHub releases (DRIVERS-3441).

• MONGOCRYPT-838 changed the S3 bucket, so URLs in 1.18.0 already require changing release driver scripts.
• New uploads are smaller (only library file and headers), include per-tarball signatures, and have self-descriptive names.
• pymongocrypt docs link to the unsigned libmongocrypt-all.tar.gz. This can instead refer to the GitHub releases page.

The set of platforms for upload_release tasks are added to match existing driver download scripts (referenced in "How binaries are obtained?" of Which drivers package libmongocrypt?).

The naming scheme follows some ideas of: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/, and follows this form:

libmongocrypt-<os>-<arch>(-<libc>-<libc.major>_<libc.minor>)(-nocrypto)-<version>.tar.gz

Examples:

libmongocrypt-linux-x86_64-musl_1_2-nocrypto-1.18.0.tar.gz
libmongocrypt-linux-s390x-glibc_2_17-nocrypto-1.18.0.tar.gz
libmongocrypt-windows-x86_64-1.18.0.tar.gz
libmongocrypt-macos-univeral-1.18.0.tar.gz

libmongocrypt on Linux has an OpenSSL dependency by default, unless building with nocrypto. Users must set crypto callbacks (e.g. call mongocrypt_setopt_crypto_hooks) to do operations requiring crypto to avoid an error.

Version numbers for glibc were determined by running objdump on the build_and_test_and_upload tasks.

[rcsanchez97]

So, I haven't reviewed the code itself yet, but I wanted to comment on the use of ldd --version to guide the naming the uploaded artifact. While this will, in fact, tell you the C library version running on the host question, it is the case that we don't always use the newest symbols from the library. This can often result in expressing an overly restrictive dependency. For instance, take the RHEL 8.2 build. The one that I grabbed at random produced an artifact named libmongocrypt-linux-arm64-glibc_2_28-nocrypto-56bad1f7ed7e6fd93e6495e105b4b5270b6b7aab.tar.gz, as RHEL 8.x releases do indeed ship glibc 2.28. However, when I looked at the actual symbols used by libmongocrypt.so in that file, it looks like this:

$ objdump -T ./lib64/libmongocrypt.so | grep GLIBC_ |sed -r -e 's/.*\(GLIBC_([0-9.]+)\).*/\1/' | sort -u
2.17

Which means that even when built on RHEL 8.x, the library only uses symbols from version 2.17 of glibc.

Glibc 2.17 is the same glibc version shipped in RHEL 7.x builds. So, in reality, it is not necessary to create both ...glibc_2_28... (for RHEL 8.x) and ...glibc_2_17... (for RHEL 7.x) artifacts, as both RHEL 8.x and 7.x could use the same artifact.

@kevinAlbs the main drawback of taking the approach I am suggesting here (i.e., to inspect the actual symbol references and determine the dependency/naming from that) is that it results in multiple platforms producing potentially identically named artifacts (as is the case with RHEL 8.x and 7.x). That means that the individual upload tasks would need to upload to a staging location and then a task which depends on the completion of all individual upload tasks would need to sort through the uploads and deduplicate. This has the additional drawback of potentially preventing the upload of any artifacts if even one individual upload task happens to fail.

[rcsanchez97]

I'm pretty sure that we don't the & here, as it will background the invocation. It probably happens that the invocation returns quickly enough that the & has no visible effect, but it still seems like it shouldn't be there.

[kevinAlbs]

The ldd check is now removed in favor of an objdump check.

[kevinAlbs]

Thank you very much for the tip.

I used objdump to fail the task if the glibc version used did not match the version specified in the file name. This confirmed (as you found) mismatches:

glibc version check failed: release_os_arch requires glibc 2.17 but library uses glibc 2.7

results in multiple platforms producing potentially identically named artifacts

That is actually OK. The S3 paths in the uploads are prefixed with the build variant. But regardless, redundant artifacts with the same architecture+glibc requirement are removed.

[rcsanchez97]

LGTM!