Repository navigation
Inconsistent default behavior of node and npm for module installing/loading #14904
Description
Activity
- changed the title
[-]Inconsistent behavior of node and npm for module installing/loading[/-][+]Inconsistent default behavior of node and npm for module installing/loading[/+]on Aug 17, 2017 Btw. this doc should be correct and put additional info for Windows, where Node use not
$HOMEbut$USERPROFILEenv (per https://github.com/nodejs/node/blob/master/lib/module.js#L618).Pull requests are, as always, welcome.
The current Node.js recommendation is to install your modules locally to your applications under a node_modules folder. This wasn't always the case, and things like
NODE_PATHand the global folders described in https://nodejs.org/api/modules.html#modules_loading_from_the_global_folders are historic and kept for compatibility (It was even recently suggested that they should be removed).npm has a concept of global installations, but these are not the same as the historic global folders above. Instead npm's concept of a global installation is for installing command line tools and not for installing modules that you would require (https://docs.npmjs.com/getting-started/installing-npm-packages-globally). Whether installing locally or globally, the last time I looked npm was hardcoded to install to a node_modules folder. I'm fairly certain that I read a FAQ on npm that asked if modules could be installed to a different named folder (short answer, no) but I can't seem to be able to find it now for referencing.
cc @nodejs/npm
the module spec is... very very frozen, and it should be. I'm not a Node.js maintainer, but I'd really discourage anyone to change that spec further.
This thing the OP is trying to do is really complicated and no, we probably won't support this usecase for the time being, specially if the only concern here is having duplicate data. Disk storage is pretty cheap these days.
That said, if this sort of thing is a big showstopper for you, I recommend you check out @zkochan's pnpm, which is explicitly designed around global deduplication of modules, without the sort of baroque backflips the OP is trying to do to achieve this. There's a chance npm proper will integrate stuff like this, but there's problems we want to solve first -- one disadvantage of this sort of global deduplication is that any modification to the central database will propagate across all projects. Because the desired feature (reduced disk usage) is achievable through a package manager, I think any sort of modification to the module spec is entirely uncalled for.
As @richardlau pointed out, npm's model is currently one in which npm projects are isolated islands which can be individually reproduces regardless of the global state of your system. This is an important feature, because it greatly simplifies installation of npm projects, and reduces the overall surface area when trying to find issues with a build. The challenge here is to optimize other aspects of npm projects (speed and disk usage), without violating that invariant.
Reacted by Zoltan KochanReacted by Zoltan Kochannpm has a concept of global installations, but these are not the same as the historic global folders above. Instead npm's concept of a global installation is for installing command line tools and not for installing modules that you would require (https://docs.npmjs.com/getting-started/installing-npm-packages-globally). Whether installing locally or globally, the last time I looked npm was hardcoded to install to a node_modules folder. I'm fairly certain that I read a FAQ on npm that asked if modules could be installed to a different named folder (short answer, no) but I can't seem to be able to find it now for referencing.
Sure, for -g option all depends on the system source1, source2:
- on Unix modules go to
{prefix}/lib/node_modulesand bin files go to{prefix}/bin - on Windows modules go to
{prefix}/node_modulesand bin files go to{prefix}
If modules don't have bin then we can still install them with -g, they will goes to
node_modulesfolder and could be share for all (if this folder would be search by node).
To be clear, this one global folder should be add depending on system (like above).This thing the OP is trying to do is really complicated and no, we probably won't support this usecase for the time being, specially if the only concern here is having duplicate data. Disk storage is pretty cheap these days.
It's not a disk space problem, I would say convenient daily usage. I have all needed tools in one place, can fast update, moved or share. Of course for finally project I keep all stuff in the local
node_modules, but sometimes we operate on files or systems (even node REPL) at the moment, and yet we do not have a finall project, such a global set of basic tools is essential.When I download node I get them with npm (located on
node_modulesfolder) and it's strange that this two toys can't operate on thisnode_modulesfolder, and it's node fault because node not use/search this folder. If adding one more folder for search is so big problem then I will not push, I can always do it differently, although it could happen out-of-the-box.- on Unix modules go to
It's been a while and there seems to be no consensus so I'll go ahead and close this out.
Hi, I try configure Node and NPM to use the same folder for install and load modules for all my project to not unnecessarily duplicate data. I want make this much portable as long as it can be, so don't want set
NODE_PATH, just use internal rule or config file inside node root folder (called$PREFIXhere).After some try I see this problems:
Default Node find module from this location (https://nodejs.org/api/modules.html#modules_loading_from_the_global_folders)
Btw. this doc should be correct and put additional info for Windows, where Node use not
$HOMEbut$USERPROFILEenv (per https://github.com/nodejs/node/blob/master/lib/module.js#L618).So in my case I have only one option, use
$PREFIX/lib/nodeto put all module here, but NPM not works like that when we try install module:firstly NPM always make
node_modulesfolder (if not exist), so even if I setprefixinnpmrcfile to my$PREFIX/lib/nodethen Node don't find that module, I must move it manually frome$PREFIX/lib/node/node_modulesto$PREFIX/lib/node.secondly, when we not change
prefixin NPM (default is Node folder) and runnpm installcommand in my$PREFIX/lib/nodefolder then NPM install module in$PREFIX/node_modulesfolder because it's default folder when we download and unpack Node (it sit only one module there - NPM). So still I must manually move my shared modules from$PREFIX/node_modulesto$PREFIX/lib/node. Removing this$PREFIX/node_modulesfolder and move NPM to$PREFIX/lib/nodeneeds some work because defaultnpmandnpm.cmduse it (and I'm not sure that after such a change NPM will work properly).I can make symlink beetwen
$PREFIX/lib/nodeand$PREFIX/node_modules, but the same story asNODE_PATH, I want make this much portable as long as it can be (try not modyfing anything in system).I can put all my project files inside
$PREFIX, so they will get access to$PREFIX/node_modules, but sometimes I want put some projects outside$PREFIXfolder and this is not fit in my case.I'm wondering why Node don't find module in one additional folder
$PREFIX/node_modulesas last step or before$PREFIX/lib/nodestep, so maybe add this somewere here https://github.com/nodejs/node/blob/master/lib/module.js#L633, like:Worth noticing that
$PREFIX/node_modulesfolder is default for NPM when used-gparam, so this change will cover default behavior Node and NPM in wider range.