Repository navigation
Node.js Foundation Modules Team Meeting 2019-07-17 #349
Description
Activity
I also added the agenda label to the two modules feature PRs:
Also, I'd like to start chatting about resource loading / loader hooks since that seems like a big ominous object hanging over our heads so far. There's an intern here at Google (@a-lxe) that has been working on a project that could really benefit from a clean resource API, especially one that supports CJS as well.
Any objections to having him at this meeting and potentially give a quick overview about what he's planning in that area?
Reacted by Geoffrey Booth and Wesley Wigham@ljharb recommended I take the GitHub Package Registry agenda detailed here to this meeting. Happy to join.
(edit: adding in full agenda text)
👋I'd like to join this meeting and discuss some package namespace concepts the GitHub Package Registry team is considering. We would value your input on the direction we're taking.
Single source of truth
For simplicity and security GPR (the GitHub Package Registry) plans to provide a system for "upstreaming NPM" such that our users only need to add a single registry URL to their config.
npm login --registry=https://npm.pkg.github.com
Through this single URL users should have access to both NPM packages and those packages which reside on GPR. However we have been running into issues with namespace conflicts with the NPM registry given its global name resolution and separate namespace system.
Namespace Problems
GitHub offers namespaces through organizations (https://github.com/nodejs/), users (https://github.com/clarkbw), and repos (https://github.com/nodejs/TSC/ or https://github.com/clarkbw/jest-localstorage-mock). But our namespaces are often different from those namespaces on NPM and this creates the conflict when trying to use a single source; who is correct if both have an answer.
For example our system currently allows you to require a module with a namespace. Without upstream support this isn't a problem but when we combine both our system and NPM you don't know which system namespace
@octo-orgshould resolve to."dependencies": { "@octo-org/octo-app": "1.0.0" }
This namespace conflict is apparent with namespaces
google-cloudandGoogleCloudPlatformwhere the URLs are https://www.npmjs.com/org/google-cloud and https://github.com/GoogleCloudPlatform respectively. If a user specified@GoogleCloudPlatformand that was also registered on NPM it would be unclear what the user wanted and vice versa for@google-cloudand that namespace being on GitHub.URLs as namespace
Our long term plan is to use URLs as a namespace prefix such that
@octo-org/octo-appshould actually be written asgithub.com/octo-org/octo-app. We believe this makes namespace resolution obvious and in control of the developers while also securing users from squatting attacks and being more future proof.The problem we face when trying to use URLs is that the NPM client mostly assumes that a GitHub URL means direct
gitaccess to the repository as seen in the NPM docs. Annpm install github.com/octo-org/octo-appornpm install https://github.com/octo-org/octo-appeither fails or connects to thegitserver.Protocol as namespace
An alternate version of the URL as namespace is using a protocol instead such that GPR requests are prefixed with
github:. Annpm install github:octo-org/octo-appisn't ideal like URLs are but would work if the NPM client didn't already use thegithub:protocol for direct git server access.Hybrid solution
The GPR team would like to make progress on the longer term solution, we believe to be something like URL namespaces. At the same time we want to make forward progress on a solution that could work better in the next couple months.
We're looking at a number of different solutions for the near term that will allow our users to use GPR as expected while we work toward a more stable namespace solution like URLs. These solutions are purely temporary so we can get up and running without requiring changes to clients that take time to propagate out.
Namespace Prefix
One example temporary solution is to use a prefix on namespace like
@gh-so users would enter@google-cloud/storagefor the default NPM namespace and@gh-GoogleCloudPlatform/storagefor the GitHub namespace. We are not in 😍with this.
We're humbled to be able to take part in this conversation and appreciate your time, thoughts, and advice on these proposals.
Reacted by Jordan HarbandReacted by Jordan HarbandProtocol as namespace
Personally I think a protocol solution would make a lot of sense, but given that Github is just acting an npm registry-provider why have a specific protocol for it rather than just a generic
npm:protocol that can connect to any registry?e.g.
// Get package from the github registry "some-package": "npm://github.com/some-org/some-package", // Get package from the npm registry "some-package-2": "npm://registry.npm.org/some-package-2@2.0", // Get package from custom registry "some-package-3": "npm://registry.my-site.org/some-package-3"Sure it's more verbose in the
package.jsonbut it's both incredibly clear where packages come from and is very extensible to many registries (and it only affects installing notimport-ing).Note that anything like that (notation in package.json dependencies) would have to be something npm itself supported; so it's likely that our task is mostly going to be to come up with a united recommendation from us and the TSC, and leave it to npm to implement it.
Thanks for sharing your thoughts @clarkbw, sounds like a great direction.
I wanted to mention the PURL spec (https://github.com/package-url/purl-spec#some-purl-examples) just-in-case this is unknown for some, as I think it connects well with the suggested approach and maybe it's worth to consider aligning with that (and perhaps contributing to that effort :-))Reacted by Benjamin Gruenbaumhey all sorry for the delay, having zoom issues
hopefully have it set up asapReacted by Bryan Clark, Jordan Harband and Alex AubuchonReacted by snek, Jan Olaf Martin and Alex AubuchonWould you like me to start the meeting instead?
looks like someone did get this resolved, without pinging here?
Time
UTC Wed 17-Jul-2019 19:00 (07:00 PM):
Or in your local time:
Links
Agenda
Extracted from modules-agenda labelled issues and pull requests from the nodejs org prior to the meeting.
nodejs/modules
Invited
Observers/Guests
Notes
The agenda comes from issues labelled with
modules-agendaacross all of the repositories in the nodejs org. Please label any additional issues that should be on the agenda before the meeting starts.Joining the meeting