sf project deploy start command throws "connect ENOMEM" error

[m-rodriguez7]

Summary

About two weeks ago, several of our developers started experiencing an issue with the sf project deploy start command: for some reason, the deploy command exits before the deployment completes. This issue happens when deploying to a scratchorg. As a result, the scripts we use to automatically push components to scratchorg are failing. What has made troubleshooting difficult is that the issue happens intermittently. Sometimes the command displays deployment results and other times it doesn't. We even get the error when using the older push/deploy sfdx commands.

Can you help us understand what causes a connect ENOMEM error?

Attached are the logs generated by sf doctor.

Logs

• 1694055708804-command-debug.log
• 1694055708804-command-stdout.log

Steps To Reproduce

Difficult to reproduce. However, this is what we are doing...

1. Create a scratchorg
   sf org create scratch --definition-file config/project-scratch-def.json --alias <org_alias>
2. Deploy components to a scratchorg
   sf project deploy start --target-org <org_alias>
3. After deployment starts, sometimes the deploy command displays deployment results and other times the deploy command exits without showing the deployment results

Expected result

Once metadata components deploy to a scratchorg, SF cli should display deployment results

Actual result

In the command line terminal, SF cli displays the error below. In the scratchorg "Deployment Status" page, however, the deployment progress can still be monitored until completion.

Error (1): Metadata API request failed: request to https://littledipper-strawberry-7315.scratch.my.salesforce.com/services/Soap/m/58.0 failed, reason: connect ENOMEM 13.109.185.171:443 - Local (0.0.0.0:0)

This is breaking our developer scripts.

System Information

{
  "cliVersion": "@salesforce/cli/2.7.11",
  "architecture": "darwin-arm64",
  "nodeVersion": "node-v18.17.1",
  "osVersion": "Darwin 22.6.0",
  "shell": "zsh",
  "rootPath": "/Users/rodrigmi/.npm-global/lib/node_modules/@salesforce/cli",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 2.3.6 (core)",
    "@oclif/plugin-commands 2.2.24 (core)",
    "@oclif/plugin-help 5.2.18 (core)",
    "@oclif/plugin-not-found 2.4.0 (core)",
    "@oclif/plugin-plugins 3.3.2 (core)",
    "@oclif/plugin-search 0.0.22 (core)",
    "@oclif/plugin-update 3.2.0 (core)",
    "@oclif/plugin-version 1.3.9 (core)",
    "@oclif/plugin-warn-if-update-available 2.0.49 (core)",
    "@oclif/plugin-which 2.2.32 (core)",
    "@salesforce/cli 2.7.11 (core)",
    "apex 2.3.12 (core)",
    "auth 2.8.14 (core)",
    "data 2.5.7 (core)",
    "deploy-retrieve 1.17.6 (core)",
    "info 2.6.40 (core)",
    "limits 2.3.32 (core)",
    "login 1.2.28 (core)",
    "marketplace 0.1.1 (core)",
    "org 2.10.5 (core)",
    "schema 2.3.24 (core)",
    "settings 1.4.27 (core)",
    "sobject 0.2.6 (core)",
    "source 2.10.32 (core)",
    "telemetry 2.3.1 (core)",
    "templates 55.5.10 (core)",
    "trust 2.6.7 (core)",
    "user 2.3.31 (core)"
  ]
}

Additional information

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[mshanemc]

That sounds like Salesforce org (on the server) is throwing that error intermittently. That's definitely a salesforce IP address.

We have some code to retry all the intermittent errors that we could add this to.

Thanks for the doctor logs, those help avoid other basic questions! ("how big is your deployment" etc.)

[mshanemc]

when I retire from coding to focus on my rap career, ENOMEM is definitely going to be my stage name

[git2gus]

This issue has been linked to a new work item: W-14084967

[AllanOricil]

My guess is that it exits without showing anything because the main node process kills itself when there isnt enought memory. It would be good if the logs could show how much memory is being consumed.

[mshanemc]

I don't think the error is coming from node

[AllanOricil]

BingGPT also thinks it is due to Node running out of memory. But maybe he is biased like I'm haha

Prompt

what could be the reason for this error message in Node "Error (1): Metadata API request failed: request to https://littledipper-strawberry-7315.scratch.my.salesforce.com/services/Soap/m/58.0 failed, reason: connect ENOMEM 13.109.185.171:443 - Local (0.0.0.0:0)"

The error message `connect ENOMEM` suggests that the Node.js process is running out of memory. This can happen when the system is low on memory or when the Node.js process has reached its maximum memory allocation. You can try increasing the maximum memory allocation for the Node.js process by setting the `--max-old-space-size` flag. For example, you can run `export NODE_OPTIONS=--max-old-space-size=8192` to increase the maximum memory allocation to 8GB¹. If you continue to experience this issue, you may need to investigate further to determine what is causing the high memory usage in your application. It could be due to a memory leak or inefficient memory usage in your code. You may also want to check if your system has enough swap space enabled². If not, you can follow instructions to add a swap file². Let me know if you need more help! 😊

Source: Conversation with Bing, 9/9/2023
(1) node.js - npm ERR! ENOMEM: not enough memory, read - Stack Overflow. https://stackoverflow.com/questions/65899543/npm-err-enomem-not-enough-memory-read.
(2) Node.js catch ENOMEM error thrown after spawn - Stack Overflow. https://stackoverflow.com/questions/26193654/node-js-catch-enomem-error-thrown-after-spawn.
(3) Build failing with ENOMEM error · Issue #7703 · vercel/next.js. https://github.com/vercel/next.js/issues/7703.
(4) FetchError: request to http://localhost:3000/api/projects failed .... https://stackoverflow.com/questions/55233508/fetcherror-request-to-http-localhost3000-api-projects-failed-reason-connec.
(5) undefined. https://support.circleci.com/hc/en-us/articles/360009208393-How-can-I-increase-the-max-memory-for-Node-.

[m-rodriguez7]

I noticed the potential fix has been merged. How do I validate? Do I have to wait for the next cli release?

[AllanOricil]

I noticed the potential fix has been merged. How do I validate? Do I have to wait for the next cli release?

@m-rodriguez7
If you are still seeing this error, could you increase the amount of memory available to your node runtime and see if it fixes the issue?

Open a terminal and run this command depending on your OS:

macos/linux

export NODE_OPTIONS=--max-old-space-size=8192

windows

set NODE_OPTIONS="--max-old-space-size=8192"

[mshanemc]

you can try it in the nightly release. https://developer.salesforce.com/docs/atlas.en-us.sfdx_setup.meta/sfdx_setup/sfdx_setup_install_cli.htm#sfdx_setup_install_cli_rc

We aren't promoting from RC/latest until Friday because of Dreamforce.

[m-rodriguez7]

@AllanOricil I did try export NODE_OPTIONS=--max-old-space-size=8192. Sadly it didn't help.

@mshanemc The fix looks promising. I did installed sf cli version salesforce/cli@2.9.8. I haven't seen the issue with new version.

[RupertBarrow]

when I retire from coding to focus on my rap career, ENOMEM is definitely going to be my stage name

Right, if you're over 80 years old, it might apply 😉