Repository navigation
Extend --env-file with support for INI sections #58729
Description
Activity
- changed the title
[-]Support for INI sections within --env-file[/-][+]Extend --env-file with support for INI sections[/+]on Jun 16, 2025 You can achieve the same thing by putting it into 2 different files and have 2 --env-file arguments in your run command.
Is there any other reason rather than cosmetic appearances for using an ini format?
You can achieve the same thing by putting it into 2 different files and have 2 --env-file arguments in your run command.
That's not good. We have an INI file that lists variables for 20 servers, i.e. 20 sections, each repeating variable names (different values). Following your suggestion, I'd need to replace 1 file with 20. At the moment we parse that file, to supply correct section to PM2 module. NodeJS should be able to pick the right section for this to be usable. Splitting it into many files is a terrible option.
Is there any other reason rather than cosmetic appearances for using an INI format?
NodeJS already uses INI, effectively, it just doesn't recognize sections, which looks like a limitation that should be resolved.
I'm not opposed to adding it, even though i don't see any use case for most regular non-power users, unless they have 20 env files like you do.
We have a custom parser, so contributions are welcome.
@vitaly-t I gave this a shot 🙂
If you want please check out #58782 and let me know if this looks good for you 😄
Basically I went with your proposed solution of using
#but without the commas. If you want to specify multiple sections you can just include multiple#, that sounds cleaner to me, what do you think?
PS: I don't really like this sort of implicit syntax to be honest.... I would have much rather preferred something like
--env-sec, however I think that that would be very tricky/awkward to get right since developers can load different env files (with multiple--env-fileflags) and I think that we should allow the flexibility for the user to specify which sections to use for each file, and I am not really sure how that can be cleanly done with a flag such as--env-sec🤔Using the
#syntax makes that very clear/clean, with the only downside that if you're using multiple env files and want to use the same section(s) for all of them you'll need to specify that multiple times, although I think that this is acceptableReacted by Code ScratcherReacted by Code Scratcher@dario-piotrowicz I love it! The addition of
#, even with a single section name, solves at least 90% of the issue, while with the minimum of change, and very importantly it is both incremental and extendable, i.e. if it is later extended with a comma-separated list or an extra--env-sec, there won't be any contradiction (compatibility issue).I hope your PR gets merged! 🤞😃
Reacted by Dario PiotrowiczReacted by Dario Piotrowicz@dario-piotrowicz One question - how smart is the section detection in the PR?
INI is a bit of a loose format, but some are more popular than others. For example, sections with aliases -
[section "alias"]. I wouldn't expect your PR to support the alias (there's no need for that), but it would be nice for it not to fail on such syntax either, i.e. extract the section name correctly, and ignore any alias.In the above example, we should expect that for
[section "alias"], thesectionis always one word (cannot have spaces), while "alias bla-bla" can have spaces in it. So, we should extract the name like this:[section_name "optional section alias here"]=
section_name, and nothing else.It is not a big deal, but a nice-have to recognize and skip any section alias.
b.t.w. I did sections parsing in TypeScript not long ago, in case it's of any help. But that code actually uses aliases. In your code we just want to skip them.
@vitaly-t thanks for confirming that the solution would work for you 🫶 (yeah let's hope it gets merged 😄🤞)
Regarding your question, the section detection I'm adding is not really that smart, it can be incrementally be made smarter when/if the need arises. For now it simply takes lines starting and ending respectively with
[and]and uses what's between the brackets as the section's name.So no, it would not ignore aliases, it would basically include them in the session's name (targeting them in the command line would be pretty awkward...).
I can definitely make it so that it does ignore aliases, but could you maybe share some examples/references that I can have a look at? I can definitely see your code doing that but I would really love to see some examples in the wild to understand this better.
One thing that is confusing me is this sections example on wikipedia: https://en.wikipedia.org/wiki/INI_file#Sections
Where they show two sections such as:
[fruit "Apple"] trademark issues = foreseeable taste = known [fruit "Raspberry"] anticipated problems ="logistics (fragile fruit)" Trademark Issues=\ possibleHow would this work in your case? I am guessing that the two sections would be treated as one? But I feel like that's most likely not what that example is going for, is it? 🤔
As shown in the Wiki, a complete example of a section can look like this:
[one.two.three "some count alias"].The section name is any word, but with dots allowed, because some processors turn those into a nested reference (which we do not need). But we should at least consume the entire section name, which may include dots.
And the alias logic can also change from one processor to another. One can treat the alias as a replacement/renaming for the section (like we do here). And another can use for a different purpose altogether. But as far as we are concerned, we should just skip any alias (ignore it completely). So, it would be nice if we could recognize a section name even when it is using an alias.
For your example above:
[fruit "Apple"] [fruit "Raspberry"]this should be treated by NodeJS as just one section, and disregard the aliases.
In all, any
[one.two.three "some alias"]should be consumed the same as[one.two.three], to be justone.two.threesection name, and that's it.In addition, I think it is safer to handle section names as case-sensitive.
Here's an example of practical use:
[node] DB_URL = https://... [node.dev] DB_URL = localhost [node.prod] DB_URL = 1.2.3.4
Let's say
nodesection is to be handled externally, and we want just a specific node environment, by specifying:node --env-file=env/config.ini#node.devWorth noting that if a line starts with
;or#, it is a comment line, to be ignored.As shown in the Wiki, a complete example of a section can look like this:
[one.two.three "some count alias"].The section name is any word, but with dots allowed, because some processors turn those into a nested reference (which we do not need). But we should at least consume the entire section name, which may include dots.
And the alias logic can also change from one processor to another. One can treat the alias as a replacement/renaming for the section (like we do here). And another can use for a different purpose altogether. But as far as we are concerned, we should just skip any alias (ignore it completely). So, it would be nice if we could recognize a section name even when it is using an alias.
For your example above:
[fruit "Apple"] [fruit "Raspberry"]this should be treated by NodeJS as just one section, and disregard the aliases.
Yes I understand, what I was trying to figure out is, are there INI processors that would handle
[fruit "Apple]+[fruit "Raspberry"]differently from what you're suggesting here? (I guess there are). If that's the case why would your suggested behavior be better suited for node? and/or should maybe the behavior around aliases be configurable? (although that might be an overkill....)Regarding different alias logic on different processors, I was trying to understand what the most common/standard one is, is that the one you're suggesting? if aliases should just be ignored what's the point in having them? 🤔
In addition, I think it is safer to handle section names as case-sensitive.
Worth noting that if a line starts with;or#, it is a comment line, to be ignored.Yes my logic is case-sensitive and lines not starting and ending with
[and]are always ignored (mh.... probably it should also allow for spaces after the]... 🤔)Here for NodeJS we only care about narrowing down to a section. And there is no standard for how aliases should be handled. And since we do not need them here, it makes sense to just skip them altogether - it is the safest way. To try anything else would be to needlessly complicate it for NodeJS.
Reacted by Dario PiotrowiczOk, I've updated the PR to ignore aliases 🙂
Reacted by Code Scratcher@dario-piotrowicz have a little doubt now about the overall logic... so, here are some questions...
Before the PR, Node would consume all the variables. And if sections were to occur, it would fail to understand those, and produce invalid variable additions. Is this accurate?
Now after the PR, if
#sectionis provided, it would load from that section only? What happens if the specified section doesn't exist?And after the PR, if no
#section specified, what is the behavior? Is it going to consume all variables from all the sections (ignoring all sections), or is it going to consume only the global ones (outside any section)?I believe the proper behavior should be as follows:
- When no section specified in command line, Node should consume all global variables only (outside any section), and ignore variables inside all other sections.
- When a section is specified (and exists), Node should treat it as incremental update/patch, i.e. still load all variables from the global section, plus then from the section that's specified.
- When specified, section does not exist, Node should fall back on scenario (1), and that's it.
Now, on to why it should be this way. Sections are to be treated as either additions or overrides. So Node should always use what's in the global section (the top one) first, and then use the specified section as addition / override for just some (or all) variables.
Example:
# global variables are added first DB_HOST = localhost DB_USER = quest DB_PORT = 1532 # sections below provide addition or override, if specified in command line... [dev] NODE_ENV = development DB_PORT = 1535 [prod] NODE_ENV = production DB_PORT = 1540
If we add
#prodto the command line, we expect the variable set to be:DB_HOST = localhost DB_USER = quest NODE_ENV = production DB_PORT = 1540Is the PR behavior consistent with the 3 points above?
@vitaly-t Yes the one you're suggesting matches exactly the behavior I went with in my PR 🙂
These tests should make that pretty clear:
node/test/parallel/test-dotenv-sections.js
Lines 27 to 55 in d6e8866
By the way, the env file that they use is:
node/test/fixtures/dotenv/sections.env
Lines 1 to 10 in d6e8866
Reacted by Code ScratcherBefore the PR, Node would consume all the variables. And if sections were to occur, it would fail to understand those, and produce invalid variable additions. Is this accurate?
Yes, currently the behavior is pretty buggy, if a section occurs the first variable present there gets read but its key includes also the section name with a newline. The remaining variables in the section are however still parsed correctly.
Reacted by Code ScratcherThank you for clarifications! 👏
What about this kind of test - to support variables with dots in them?
one.two.three = some valueI only see sections with dots tested, but not the variables themselves.
I'm not sure about why
should allow comments to be present after the section declarationtest was added. The general rule for a good INI syntax - no inline comments of any kind :) I mean, it looks a little redundant, no?Anyway, just for later on,.... the reason why a proper INI processor should never allow inline comments for variables is scenario like this:
DB_PASSWORD = #hello;thereThe above would fail to work as expected twice, first of
#, and then on;😄Thank you for clarifications! 👏
My pleasure 😄
What about this kind of test - to support variables with dots in them?
one.two.three = some value
I only see sections with dots tested, but not the variables themselves.Yeah I don't see tests for those, actually the tests we have for the env parsing include very plain keys all with
the same casing (I think it'd be great to include more edge case keys, such as keys with spaces, dots, etc...)I will look into adding tests for those (or if you'd like to do that just let me know and I can leave it to you 🙂)
That being said I don't think that this matter is part of this issue, so I would prefer not include it in my PR (I am a big fan of smaller focused PRs 😄) (PS: if you'd like feel free to open a dedicated issue for the missing tests 🙂)
I'm not sure about why
should allow comments to be present after the section declarationtest was added. The general rule for a good INI syntax - no inline comments of any kind :) I mean, it looks a little redundant, no?I simply noticed that we do have dedicated tests for inline comments for variables:
node/test/parallel/test-dotenv.js
Lines 45 to 46 in 910a8af
So I figured we should support the same for sections (especially from a consistency point of view). Do you disagree?
Anyway, just for later on,.... the reason why a proper INI processor should never allow inline comments for variables is scenario like this:
DB_PASSWORD = #hello;thereThe above would fail to work as expected twice, first of
#, and then on;😄I think that that would be interpreted as
DB_PASSWORD =here, which seems... ok-ish to me? 🤷Regardless, like for the keys comment before, I feel like this is out of scope here, if strongly believe that inline comments should not be supported I think the best course of action would be to open a dedicated issue requesting its removal (or much better, straight opening a PR removing its support and see how people react to that) 🙂
PS: it looks to me like support for inline comments has purposely been added here, it also matches dotenv which I think was the base for node's implementation,
so I do think that removing it would likely be very contentious, but up to you if you want to give that a try 🙂
(I for one don't particularly mind the comments, I am no INI/env syntax expert though 😅)(I think it'd be great to include more edge case keys, such as keys with spaces, dots, etc...
Just so, this implementation should only recognize what can be used as a valid environment variable name. Spaces, for example, are not allowed for that, dots and hyphens - generally not recommended, but they currently work.
it looks to me like support for inline comments has purposely been added here, it also matches dotenv which I think was the base for node's implementation
It was also the basis for all the problems in dotenv, related to values with comment-like content. Inline comments are a bad idea for INI format, they should be never used for INI ;) Only full-line comments should be supported ;)
(I think it'd be great to include more edge case keys, such as keys with spaces, dots, etc...
Just so, this implementation should only recognize what can be used as a valid environment variable name. Spaces, for example, are not allowed for that, dots and hyphens - generally not recommended, but they currently work.
mh... yes you're mostly right... I thought the logic would be clearer/permissive but dotenv seems to only support valid environment variables (with some few exceptions, like keys with
-s and.s), while node is indeed much more permissive here 🤔I've created a dedicated issue for such discussion: #58807
Two years ago, I suggested that the Node.js project should have a proper specification of the syntax of
.envfiles (see the comment thread in #48890 (comment)). That is still an open issue (and, in fact, the only empty checkbox in #49148). As you can see from these previous discussions, the declared goal of the implementation was compatibility with thedotenvpackage. While the (unspecified) format that Node.js anddotenvimplement bears resemblance of INI files, it is not the same, and does not support sections. This was known back then, see #48890 (comment).Does
dotenv, which @anonrig based his implementation on as far as I know, support sections in a manner that is compatible with what is being proposed here? If the goal remains compatibility, then we should not deviate fromdotenv. Without a proper specification, checking whatdotenvdoes is often the only way to tell whether the behavior of Node.js is correct w.r.t. parsing.envfiles (see, for example, #53461).Reacted by Dario Piotrowicz@tniessen NodeJS needs INI for environment only, while INI elsewhere can be used for a lot more, just like
dotenvdoes a lot more processing than NodeJS needs. NodeJS needs only a small subset of it, to pick valid-ish environment variables, and actuate them (persist into the process environment).This small addition is just to make it usable with INI files that contain a separate section per environment/server, so we don't have to chop one large INI file into many small ones just so it can be used with NodeJS.
@vitaly-t I understand the motivation, but it does not answer the question as to whether the Node.js project wants to deviate from the previously declared goal of supporting the same syntax as the original
dotenvpackage.However, in most cases the file that we pass in has INI format, which is quite expected
Why are these files in INI format? Is that a convention for
.envfiles in some particular framework?Oh, pity, so this isn't happening then 😢
What is the problem this feature will solve?
In our team, we embraced the use of
--env-fileright from NodeJS v20, it is an awesome addition!However, in most cases the file that we pass in has INI format, which is quite expected. And INI files support sections (
[section_name]), and we do not want NodeJS to push all values from all sections automatically intoNODE_ENV, because some may end up overriding useful configuration insideNODE_ENVthat's there by default.What is the feature you are proposing to solve the problem?
For NodeJS to support optional section name(s), so only from those the values would go into
NODE_ENV.For example, supporting
--env-sec, to take one or more section names, so only from those to load the variables:Or alternatively extend
--env-filesyntax, say by appending#+ section name(s):So above it would load
env/dev.ini, and push everything only from sectionssec1andsec2intoNODE_ENV. I personally prefer the latter syntax, to avoid too many parameters to be passed in just for loading configuration.What alternatives have you considered?
Without any such support, we have to split an INI file into several, one for NodeJS, and one for the rest of the application, which is a significant inconvenience.
Afterthought
As an extra benefit, such a feature would make it possible to consolidate environment configurations inside a single INI file when needed:
Presently, NodeJS will just end up with the last value as an override, which is against the INI logic. Sections need to be respected when they are used, since NodeJS already has partial support for INI files, in effect, so this addition would make it a complete support.
It would be sweet, if for the above example we could just specify: