You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For any package published on Bower right now it is possible to assume that one would get their source files with the following glob: src/**/*.purs
In purescript/spago#288 we explored what it would take to lift that limitation (it's a good read to get some context on what follows here)
One of the main blockers for that was the assumption coming from Bower registry that "one package = one repo". Since here we're moving away from the git-based model and towards packaging sources in immutable archives, I thought we could take this occasion to allow publishing packages with sources in arbitrary paths.
This would allow things like publishing several package from the same monorepo.
There has been some discussion in #4 about this, and we're spinning off this issue to keep track of a concern expressed there:
The issue I'm worried about with sources is basically for setups which don't use Spago; I want to avoid making life difficult for people who are still using other things like psc-package or pulp/bower. Yes, it's probably not too much effort to switch a build script of just a few lines in most cases, especially with spago init, but it's harder if the assumption is built into more complex tooling, such as, say, editor plugins. This assumption is built into purs publish right now too. It becomes significantly more complicated if these build scripts or tooling can't/don't want to assume the presence of Spago on the machine in use, or don't want to add Dhall as a dependency.
I would be happy if we could have the curator require that the sources for the lib target for any published package are just ["src/**/*.purs"], at least to start with; I could see us relaxing that later once most of the ecosystem is using this registry.
This would allow things like publishing several package from the same monorepo.
This ought to be possible anyway with the new design even if we require source files to be in src, right? If you have multiple packages in a monorepo, I would expect that they would each be in separate directories with separate package manifest files, and the sources key in the package manifest file would be interpreted relative to the directory in which the manifest file lives.
There is now (as of #76) support for "multiple manifests in the same repo" via the subdir key in the Repo type.
However, I think the concern expressed of "we should limit to one package only until the ecosystem is using this" is still valid, and more specifically I'd like to have support for this in Spago before we allow the publishing of packages that specify a subdir. Makes sense?
I still don't understand what we gain by making source paths configurable by a package author, over having the rule be "the visible source files are always those which are located in the src folder relative to the package manifest file, and only those". That still permits managing a number of packages together in a monorepo, and as far as I'm aware that is the only concern which has been brought up in the context of this issue.
I think the subdir key in Repo is good; I agree that we should allow the author to put a package manifest file in an arbitrary subdirectory of a repo. I just think that package authors should not have freedom to put source files wherever they want beyond that; I think source files should always be placed in a src/ directory next to the package manifest file.
Oh I see, this makes sense. Since the "official package source" is going to be contained in a Target called lib (or whatever name we want to pick for that) should we just enforce that the sources for that Target are exactly ["src/**/*.purs"]?
Oh also, how should tagging work for libraries in monorepos? Right now we require tags to be of the form vX.Y.Z in package-sets, right? But if there are different libraries together in one repo, that won't work (unless the libraries' versions are all kept in sync).
@hdgarrood cool, I'll update the spec to use commit hashes instead of tags
I just realized this feature would enable us to possibly keep all the core libraries in a monorepo, making cross-library updates (such as in the case of whole-ecosystem-breakage) much easier, as we could compile all the libraries together in the same build - cc @JordanMartinez
@f-f While it might make it possible to build things together via a monorepo, the same can't be said for issue trackers, right? Meaning, there isn't a corresponding way to distinguish issues within the issue tracker to repo-specific issues (or at least in a good way).
All the monorepos I've seen use labels to classify tickets. In practice that seems to work well, and personally I'd be more concerned about notification spam, i.e. one might not want to be notified for all the things happening across all the core libraries.
In any case I'm not proposing anything, mine was just a note admiring how this feature could enable some interesting workflows that we don't have available right now 🙂
It is good to find those issues, which I didn't consider prior to that.
I believe a package is basically a container for just two things: package manifest file and src directory (which should be the standard restricted name), what else is needed for a PS package to be a package (huh there is also need to be some standard stuff like readme/licence)? Until it is not supposed to be something more complex and bloated where you can place whatever data you want, but I believe it is not the case for PureScript packages.
I just realized this feature would enable us to possibly keep all the core libraries in a monorepo, making cross-library updates (such as in the case of whole-ecosystem-breakage) much easier, as we could compile all the libraries together in the same build -
I'm glad that there is an understanding or at least a suspicion that a monorepo based model may be a good thing for maintaining PS libraries. That is what I was talking about in the proposal that assumes that some of the existing packages have to be decomposed and split apart. This wouldn't lead to a drastic increase in the number of packages as a whole because it really refers only to a limited number of the bloated packages, but certainly, if we consider the model package == repo, this would be almost ridiculous to have a separate repo for each package.
I also believe this presumption (that package == repo) as well leads to a desire to combine and merge packages, instead of decomposing them, and when you still have to decompose, it causes additional pain of setting up repos with the infrastructure for each, etc. With the monorepo approach, such things become much more obvious and easy (even though it requires some effort to setup).
Oh also, how should tagging work for libraries in monorepos? Right now we require tags to be of the form vX.Y.Z in package-sets, right? But if there are different libraries together in one repo, that won't work (unless the libraries' versions are all kept in sync).
A simple tagging strategy surely doesn't play well with monorepos. Though as you can have multiple tags it could be quite ok if it would be really needed. But for packages that are supposed to be published in the registry, there is no reason to be strongly coupled with the repository that holds the source.
and personally I'd be more concerned about notification spam, i.e. one might not want to be notified for all the things happening across all the core libraries.
I think in general for big repos people don't want to have all the notifications enabled (only the most cautious/responsible ones). If there is a need to draw the attention of a particular person it is achieved using the name mention, and more accurate automation and targeting can be accomplished with custom Github bots.
I think git (or perhaps GitHub) needs to have first-class support for mono-repos.
I don't think that something meaningful in this direction is going to happen soon, just because actually support for mono repo of git itself is good enough, all the other stuff is achieved by external tooling (as well as mentioned GitHub bots), which should not be complex, and usually still crafted for a particular use case.
Activity
hdgarrood commentedon Apr 26, 2020
This ought to be possible anyway with the new design even if we require source files to be in
src, right? If you have multiple packages in a monorepo, I would expect that they would each be in separate directories with separate package manifest files, and thesourceskey in the package manifest file would be interpreted relative to the directory in which the manifest file lives.f-f commentedon Sep 27, 2020
There is now (as of #76) support for "multiple manifests in the same repo" via the
subdirkey in theRepotype.However, I think the concern expressed of "we should limit to one package only until the ecosystem is using this" is still valid, and more specifically I'd like to have support for this in Spago before we allow the publishing of packages that specify a
subdir. Makes sense?hdgarrood commentedon Sep 27, 2020
I still don't understand what we gain by making source paths configurable by a package author, over having the rule be "the visible source files are always those which are located in the
srcfolder relative to the package manifest file, and only those". That still permits managing a number of packages together in a monorepo, and as far as I'm aware that is the only concern which has been brought up in the context of this issue.f-f commentedon Sep 27, 2020
How do we find out the locations of several package manifests?
hdgarrood commentedon Sep 27, 2020
I think the
subdirkey in Repo is good; I agree that we should allow the author to put a package manifest file in an arbitrary subdirectory of a repo. I just think that package authors should not have freedom to put source files wherever they want beyond that; I think source files should always be placed in asrc/directory next to the package manifest file.f-f commentedon Sep 27, 2020
Oh I see, this makes sense. Since the "official package source" is going to be contained in a
Targetcalledlib(or whatever name we want to pick for that) should we just enforce that thesourcesfor thatTargetare exactly["src/**/*.purs"]?hdgarrood commentedon Sep 27, 2020
Yeah, exactly.
f-f commentedon Sep 28, 2020
Agreed 👏
So I'd say the actionable points for this issue are:
sources: ["src/**/*.purs"]in itslibTargetsubdiris present to new packages until we are confident that downstream tooling can deal with itMakes sense?
hdgarrood commentedon Sep 28, 2020
Sounds good!
hdgarrood commentedon Sep 28, 2020
Oh also, how should tagging work for libraries in monorepos? Right now we require tags to be of the form
vX.Y.Zin package-sets, right? But if there are different libraries together in one repo, that won't work (unless the libraries' versions are all kept in sync).1 remaining item
hdgarrood commentedon Oct 24, 2020
That sounds good to me 👍
f-f commentedon Oct 26, 2020
@hdgarrood cool, I'll update the spec to use commit hashes instead of tags
I just realized this feature would enable us to possibly keep all the core libraries in a monorepo, making cross-library updates (such as in the case of whole-ecosystem-breakage) much easier, as we could compile all the libraries together in the same build - cc @JordanMartinez
JordanMartinez commentedon Oct 26, 2020
@f-f While it might make it possible to build things together via a monorepo, the same can't be said for issue trackers, right? Meaning, there isn't a corresponding way to distinguish issues within the issue tracker to repo-specific issues (or at least in a good way).
f-f commentedon Oct 26, 2020
All the monorepos I've seen use labels to classify tickets. In practice that seems to work well, and personally I'd be more concerned about notification spam, i.e. one might not want to be notified for all the things happening across all the core libraries.
In any case I'm not proposing anything, mine was just a note admiring how this feature could enable some interesting workflows that we don't have available right now 🙂
JordanMartinez commentedon Oct 27, 2020
I think
git(or perhaps GitHub) needs to have first-class support for mono-repos.Regardless, yeah, there's a lot of things this can enable, such as running all tests on packages in the package sets. It'll be interesting to explore.
f-f commentedon Dec 15, 2020
Update:
This means also updating the
versionfield here to be calledreff-f commentedon Dec 19, 2020
I updated the spec in #107, so the actionable points for this issue remain:
Fix #16
wclr commentedon May 8, 2021
It is good to find those issues, which I didn't consider prior to that.
I believe a package is basically a container for just two things: package manifest file and src directory (which should be the standard restricted name), what else is needed for a PS package to be a package (huh there is also need to be some standard stuff like readme/licence)? Until it is not supposed to be something more complex and bloated where you can place whatever data you want, but I believe it is not the case for PureScript packages.
I'm glad that there is an understanding or at least a suspicion that a monorepo based model may be a good thing for maintaining PS libraries. That is what I was talking about in the proposal that assumes that some of the existing packages have to be decomposed and split apart. This wouldn't lead to a drastic increase in the number of packages as a whole because it really refers only to a limited number of the bloated packages, but certainly, if we consider the model package == repo, this would be almost ridiculous to have a separate repo for each package.
I also believe this presumption (that package == repo) as well leads to a desire to combine and merge packages, instead of decomposing them, and when you still have to decompose, it causes additional pain of setting up repos with the infrastructure for each, etc. With the monorepo approach, such things become much more obvious and easy (even though it requires some effort to setup).
A simple tagging strategy surely doesn't play well with monorepos. Though as you can have multiple tags it could be quite ok if it would be really needed. But for packages that are supposed to be published in the registry, there is no reason to be strongly coupled with the repository that holds the source.
I think in general for big repos people don't want to have all the notifications enabled (only the most cautious/responsible ones). If there is a need to draw the attention of a particular person it is achieved using the name mention, and more accurate automation and targeting can be accomplished with custom Github bots.
I don't think that something meaningful in this direction is going to happen soon, just because actually support for mono repo of git itself is good enough, all the other stuff is achieved by external tooling (as well as mentioned GitHub bots), which should not be complex, and usually still crafted for a particular use case.