Skip to content

http: support environment-defined proxy #8381

Description

@silverwind

To enable HTTP connectivity behind corporate firewalls, a number of tools and programming languages support HTTP/HTTPS proxies defined through environment variables like

HTTP_PROXY=http://proxy.com
HTTPS_PROXY=https://proxy.com
NO_PROXY="*.home.com,another.com"

Note that there seems to be no consensus on the case of these variables and all-lowercase variable names are also very common. My limited research suggest that at least the following languages automatically obtain and use a proxy from the environment:

  • Python
  • Go
  • Ruby
  • R

The request module also supports these variables, but I feel they show be respected by core http and https for best compatibilty.

Activity

  1. added
    httpIssues and PRs related to the http subsystem.
    httpsIssues and PRs related to the https subsystem.
    feature requestIssues requesting new Node.js features.
    on Sep 2, 2016
  2. martinheidegger commented on Sep 2, 2016

    @martinheidegger

    While I do agree that this is a very common thing to want, I think it is important to point out that so are also many other things that request implements. I think it is pretty cool of Node to allow to bypass the environment variable, so If it is implemented I think it should be optional and for backwards compatibility reasons "off-by-default".

  3. silverwind commented on Sep 2, 2016

    @silverwind
    ContributorAuthor

    off-by-default

    I don't think this is going to work here. Imagine a CLI tool spawning a child process of node. In that case, the user cannot reasonably provide a --flag to enable proxy environment support. I think support should be unconditionally enabled. If one wants to skip the proxy, they can always do export NO_PROXY="*" (or unset the variables).

  4. martinheidegger commented on Sep 2, 2016

    @martinheidegger

    ...cannot reasonably provide a --flag to enable proxy environment support...

    I think this is a fallacy (don't ask me which) because the statement can be reversed to: The user cannot reasonably provide a --not-flag to disable the proxy environment support.

    The person that writes the request:

    http.request(url, {
      respectEnv: true
    })

    decides if this request respects the environment variable (new mode) or not (legacy).

    I am pushing this because HTTP_PROXY environment variables and errors related to them are horrible to debug once you have an application running and just upgraded a node version.

    In any case I would say that a default: respectEnv=false should be treated as semver:minor.
    A default of respectEnv=true as semver:major.

  5. jasnell commented on Sep 2, 2016

    @jasnell
    Member

    Not saying this shouldn't be done, but there is a bit of a security risk inherent in this. Using an environment variable, it would be possible for a rogue module to set the environment variable discreetly causing traffic to be redirected through the proxy without the developers/users awareness.

  6. silverwind commented on Sep 2, 2016

    @silverwind
    ContributorAuthor

    @jasnell Pretty sure something similar could be achieved by monkey-patching http right now. In the end, you just have to trust your modules.

  7. jasnell commented on Sep 2, 2016

    @jasnell
    Member

    Yep, as I said, I didn't say it shouldn't be done ;-) If we do it tho, we need to make sure the risks are well documented.

  8. mscdex commented on Sep 2, 2016

    @mscdex
    Contributor

    -1 as I think this is something that is better handled in userland

  9. silverwind commented on Sep 2, 2016

    @silverwind
    ContributorAuthor

    This would depend on proxy support in the agent, something which #1490 looks to have been for, but was closed for some reason.

    Also to quote @sindresorhus from sindresorhus/got#79:

    I strongly believe this is something that should be a part of Node.js and not every module doing HTTP

  10. tjwebb commented on Dec 5, 2016

    @tjwebb

    it would be possible for a rogue module to set the environment variable discreetly causing traffic to be redirected through the proxy

    A Rogue module can monkey-patch all the methods in the http module if it wants to. Rogue modules are their own separate problem that I don't think we can solve here.

    -1 as I think this is something that is better handled in userland

    I don't think the structure of a network which is beyond my control qualifies as a userland problem. My OS deals with any proxy I may or may not need to connect through so that each application doesn't have to deal with this. Does this example analogy not hold for nodejs core? If not, do we still want to continue burdening every module using http with implementing this option?

  11. bnoordhuis commented on Dec 5, 2016

    @bnoordhuis
    Member

    Related: #1490

    To summarize: "Proxy support?" "Not saying no but..."

  12. matthewwiesen commented on Dec 5, 2016

    @matthewwiesen

    I would agree that ultimately this is something that should be handled directly within node.

    Node should be read my environment variables to see that I am starting the node executable with my http_proxy environment variable and thus I want all http requests to go through my proxy.

    curl, npm, git, etc, all respect these environment variables by default. This is the purpose of the environment. The user has already specified these within their environment, so the fact that they aren't being respected is very confusing.

    It shouldn't be left up to a non-node-core library developer to decide to support my proxy environment by properly configuring node supported HTTP module with my environment variables or via a configuration to be passed into their library because ultimately I may not be directly using those modules, such as in the case of using a framework that includes a library that includes a library that interfaces with the HTTP module. Thus this "userland" solution essentially creates a recursive issue through all dependencies which is much harder to solve in all cases.

    Since the node HTTP library is at the core of this, it should solve this issue by respecting the environment variables used at run time and override other attempts where libraries / modules may try to set these settings itself unless some other environment variable is passed to allow for this.

  13. bnoordhuis commented on Dec 6, 2016

    @bnoordhuis
    Member

    @matthewwiesen The counterargument to your argument is that curl, npm and git are all end user programs, whereas node is a platform.

    A better comparison is python: the builtin httplib doesn't respect http_proxy, that is left to user libraries. Python, unlike node, has a strong "batteries included" philosophy, so that is saying something.

    Also: big slippery slope. Yes, curl respects http_proxy... but it also honors all_proxy, no_proxy with patterns and wildcards, and happily parses your .netrc with every connection. I don't think that has a place in core. Core is for mechanism, not policy.

  14. 81 remaining items

  15. joyeecheung commented on Jul 8, 2025

    @joyeecheung
    Member

    For those who are still following: I opened a pull request to implement support for http/https built-ins #58980, following the support for fetch in #57165, both would be first opt-in under NODE_USE_ENV_PROXY=1 before we figure out whether they can be enabled by default without breakages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    feature requestIssues requesting new Node.js features.httpIssues and PRs related to the http subsystem.httpsIssues and PRs related to the https subsystem.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions