Issue with Javascript heap out of memory

[Mewks]

Hi all

I'm running this on UnRaid and I've tried playing with NODE_OPTIONS=--max_old_space_size=18048 but that didn't make any difference. Anyone have any idea whats happening here?

Using the latest version of everything.

2023-08-07T16:35:08: PM2 log: Launching in no daemon mode
2023-08-07T16:35:08: PM2 log: [Watch] Start watching YoutubeDL-Material
2023-08-07T16:35:08: PM2 log: App [YoutubeDL-Material:0] starting in -fork mode-
2023-08-07T16:35:08: PM2 log: App [YoutubeDL-Material:0] online
2023-08-07T06:35:14.096Z INFO: Config items set using ENV variables.
2023-08-07T06:35:14.408Z INFO: Restarting server...
2023-08-07T16:35:14: PM2 log: App [YoutubeDL-Material:0] exited with code [1] via signal [SIGINT]
2023-08-07T16:35:14: PM2 log: App [YoutubeDL-Material:0] starting in -fork mode-
2023-08-07T16:35:14: PM2 log: App [YoutubeDL-Material:0] online
2023-08-07T06:35:20.298Z INFO: Config items set using ENV variables.
2023-08-07T06:35:23.034Z INFO: YoutubeDL-Material v4.3.2 started on PORT 17442
2023-08-07T06:35:23.043Z VERBOSE: Watching FractalFlow with delay interval of 43200000
2023-08-07T06:35:23.313Z VERBOSE: Subscription: getting list of videos to download for FractalFlow with args: --dump-json,-o,"subscriptions/channels/FractalFlow/%(title)s.%(ext)s",-ciw,--write-info-json,--print-json,-f,bestvideo[ext=mp4]+bestaudio[ext=m4a]/mp4,--cookies,appdata/cookies.txt,--write-thumbnail
<--- Last few GCs --->
[51:0x49d97f0] 266244 ms: Scavenge 4019.9 (4120.4) -> 4018.7 (4121.1) MB, 29.6 / 0.0 ms (average mu = 0.535, current mu = 0.551) allocation failure
[51:0x49d97f0] 266315 ms: Scavenge 4020.8 (4121.1) -> 4019.8 (4141.6) MB, 42.6 / 0.0 ms (average mu = 0.535, current mu = 0.551) allocation failure
[51:0x49d97f0] 272389 ms: Mark-sweep 4033.8 (4141.6) -> 4029.7 (4155.9) MB, 5728.9 / 0.1 ms (average mu = 0.294, current mu = 0.115) allocation failure scavenge might not succeed
<--- JS stacktrace --->
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
1: 0xb09980 node::Abort() [node /app/app.js]
2: 0xa1c235 node::FatalError(char const*, char const*) [node /app/app.js]
3: 0xcf784e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node /app/app.js]
4: 0xcf7bc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node /app/app.js]
5: 0xeaf465 [node /app/app.js]
6: 0xebf12d v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node /app/app.js]
7: 0xec1e2e v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node /app/app.js]
8: 0xe830a2 v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [node /app/app.js]
9: 0xe7b6b4 v8::internal::FactoryBasev8::internal::Factory::AllocateRawWithImmortalMap(int, v8::internal::AllocationType, v8::internal::Map, v8::internal::AllocationAlignment) [node /app/app.js]
10: 0xe7d3c0 v8::internal::FactoryBasev8::internal::Factory::NewRawOneByteString(int, v8::internal::AllocationType) [node /app/app.js]
11: 0xe851fd v8::internal::Factory::NewStringFromUtf8(v8::base::Vector const&, v8::internal::AllocationType) [node /app/app.js]
12: 0xd06415 v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::NewStringType, int) [node /app/app.js]
13: 0xbf5139 [node /app/app.js]
14: 0xae3eb5 [node /app/app.js]
15: 0x15856cc [node /app/app.js]
2023-08-07T16:39:48: PM2 log: App [YoutubeDL-Material:0] exited with code [0] via signal [SIGABRT]
2023-08-07T16:39:48: PM2 log: App [YoutubeDL-Material:0] starting in -fork mode-
2023-08-07T16:39:48: PM2 log: App [YoutubeDL-Material:0] online
2023-08-07T06:39:53.709Z INFO: Config items set using ENV variables.
(node:77) DeprecationWarning: uuidv4() is deprecated. Use v4() from the uuid module instead.
(Use node --trace-deprecation ... to show where the warning was created)
2023-08-07T06:39:54.489Z INFO: YoutubeDL-Material v4.3.2 started on PORT 17442

======================================

Happened again after a completely fresh install.

2023-08-07T07:22:07.465Z VERBOSE: Subscription: finished check for FractalFlow
2023-08-07T07:22:14.537Z VERBOSE: Collecting info for download 76cba51a-0f07-4d09-a503-e2ad11cace0f
2023-08-07T07:22:14.547Z VERBOSE: yt-dlp args being used: -o,subscriptions/channels/FractalFlow/%(title)s.%(ext)s,--write-info-json,-f,bestvideo+bestaudio,--merge-output-format,mp4,--cookies,appdata/cookies.txt,--write-thumbnail,--no-clean-info-json,-j,--no-simulate
2023-08-07T07:22:14.582Z VERBOSE: Collecting info for download 21f023c3-5504-4c38-9ad6-ab2ebbc07566
2023-08-07T07:22:14.595Z VERBOSE: Collecting info for download b391bab2-3878-474a-88a8-8699d28028ab
2023-08-07T07:22:14.606Z VERBOSE: Collecting info for download 0f1bfc25-005b-4e31-aae1-104d1fcd772b
2023-08-07T07:22:14.619Z VERBOSE: Collecting info for download 8933cd9d-d1a4-4a89-826c-4ba06ad7fdaa
2023-08-07T07:22:14.626Z VERBOSE: yt-dlp args being used: -o,subscriptions/channels/FractalFlow/%(title)s.%(ext)s,--write-info-json,-f,bestvideo+bestaudio,--merge-output-format,mp4,--cookies,appdata/cookies.txt,--write-thumbnail,--no-clean-info-json,-j,--no-simulate
2023-08-07T07:22:14.634Z VERBOSE: yt-dlp args being used: -o,subscriptions/channels/FractalFlow/%(title)s.%(ext)s,--write-info-json,-f,bestvideo+bestaudio,--merge-output-format,mp4,--cookies,appdata/cookies.txt,--write-thumbnail,--no-clean-info-json,-j,--no-simulate
2023-08-07T07:22:14.636Z VERBOSE: yt-dlp args being used: -o,subscriptions/channels/FractalFlow/%(title)s.%(ext)s,--write-info-json,-f,bestvideo+bestaudio,--merge-output-format,mp4,--cookies,appdata/cookies.txt,--write-thumbnail,--no-clean-info-json,-j,--no-simulate
2023-08-07T07:22:14.638Z VERBOSE: yt-dlp args being used: -o,subscriptions/channels/FractalFlow/%(title)s.%(ext)s,--write-info-json,-f,bestvideo+bestaudio,--merge-output-format,mp4,--cookies,appdata/cookies.txt,--write-thumbnail,--no-clean-info-json,-j,--no-simulate
2023-08-07T07:22:16.218Z VERBOSE: Downloading 76cba51a-0f07-4d09-a503-e2ad11cace0f
2023-08-07T07:22:16.227Z VERBOSE: Downloading 21f023c3-5504-4c38-9ad6-ab2ebbc07566
2023-08-07T07:22:16.387Z VERBOSE: Downloading b391bab2-3878-474a-88a8-8699d28028ab
2023-08-07T07:22:16.489Z VERBOSE: Downloading 8933cd9d-d1a4-4a89-826c-4ba06ad7fdaa
2023-08-07T07:22:16.711Z VERBOSE: Downloading 0f1bfc25-005b-4e31-aae1-104d1fcd772b
2023-08-07T07:22:20.519Z VERBOSE: Downloading 76cba51a-0f07-4d09-a503-e2ad11cace0f
2023-08-07T07:22:20.637Z VERBOSE: Downloading 21f023c3-5504-4c38-9ad6-ab2ebbc07566
2023-08-07T07:22:20.773Z VERBOSE: Downloading b391bab2-3878-474a-88a8-8699d28028ab
2023-08-07T07:22:20.782Z VERBOSE: Downloading 8933cd9d-d1a4-4a89-826c-4ba06ad7fdaa
2023-08-07T07:22:20.881Z VERBOSE: Downloading 0f1bfc25-005b-4e31-aae1-104d1fcd772b
2023-08-07T07:22:38.921Z ERROR: ERROR: Unable to rename file: [Errno 2] No such file or directory: 'subscriptions/channels/FractalFlow/How to Use Advanced Technical Analysis for Maximum Precision!.f315.webm.part' -> 'subscriptions/channels/FractalFlow/How to Use Advanced Technical Analysis for Maximum Precision!.f315.webm'
WARNING: Cannot update utime of file
ERROR: [Errno 2] No such file or directory: 'subscriptions/channels/FractalFlow/How to Use Advanced Technical Analysis for Maximum Precision!.temp.mp4' -> 'subscriptions/channels/FractalFlow/How to Use Advanced Technical Analysis for Maximum Precision!.mp4'
2023-08-07T07:23:17.970Z ERROR: WARNING: Cannot update utime of file
ERROR: [Errno 2] No such file or directory: 'subscriptions/channels/FractalFlow/The SIMPLE Algorithm to DETECT and CONFIRM a High-Quality Trade from SCRATCH.temp.mp4' -> 'subscriptions/channels/FractalFlow/The SIMPLE Algorithm to DETECT and CONFIRM a High-Quality Trade from SCRATCH.mp4'
2023-08-07T07:23:18.891Z ERROR: ERROR: Unable to rename file: [Errno 2] No such file or directory: 'subscriptions/channels/FractalFlow/The SIMPLE Algorithm to DETECT and CONFIRM a High-Quality Trade from SCRATCH.f315.webm.part' -> 'subscriptions/channels/FractalFlow/The SIMPLE Algorithm to DETECT and CONFIRM a High-Quality Trade from SCRATCH.f315.webm'
ERROR: Unable to rename file: [Errno 2] No such file or directory: 'subscriptions/channels/FractalFlow/The SIMPLE Algorithm to DETECT and CONFIRM a High-Quality Trade from SCRATCH.f251.webm.part' -> 'subscriptions/channels/FractalFlow/The SIMPLE Algorithm to DETECT and CONFIRM a High-Quality Trade from SCRATCH.f251.webm'
2023-08-07T07:23:21.923Z ERROR: ERROR: Unable to rename file: [Errno 2] No such file or directory: 'subscriptions/channels/FractalFlow/The TOP 3 Candlestick Patterns According to SCIENCE.f315.webm.part' -> 'subscriptions/channels/FractalFlow/The TOP 3 Candlestick Patterns According to SCIENCE.f315.webm'
2023-08-07T07:23:22.354Z ERROR: WARNING: Cannot update utime of file
ERROR: [Errno 2] No such file or directory: 'subscriptions/channels/FractalFlow/The TOP 3 Candlestick Patterns According to SCIENCE.temp.mp4' -> 'subscriptions/channels/FractalFlow/The TOP 3 Candlestick Patterns According to SCIENCE.mp4'
2023-08-07T07:23:28.683Z ERROR: ERROR: Unable to rename file: [Errno 2] No such file or directory: 'subscriptions/channels/FractalFlow/Learn VOLUME SPREAD ANALYSIS in less than 20 Minutes!.f315.webm.part' -> 'subscriptions/channels/FractalFlow/Learn VOLUME SPREAD ANALYSIS in less than 20 Minutes!.f315.webm'
WARNING: Cannot update utime of file
ERROR: [Errno 2] No such file or directory: 'subscriptions/channels/FractalFlow/Learn VOLUME SPREAD ANALYSIS in less than 20 Minutes!.temp.mp4' -> 'subscriptions/channels/FractalFlow/Learn VOLUME SPREAD ANALYSIS in less than 20 Minutes!.mp4'
2023-08-07T07:23:28.687Z ERROR: ERROR: Unable to rename file: [Errno 2] No such file or directory: 'subscriptions/channels/FractalFlow/Learn VOLUME SPREAD ANALYSIS in less than 20 Minutes!.f251.webm.part' -> 'subscriptions/channels/FractalFlow/Learn VOLUME SPREAD ANALYSIS in less than 20 Minutes!.f251.webm'
2023-08-07T07:24:01.865Z ERROR: ERROR: Unable to rename file: [Errno 2] No such file or directory: 'subscriptions/channels/FractalFlow/16 Ways of Anticipating a TREND REVERSAL.f315.webm.part' -> 'subscriptions/channels/FractalFlow/16 Ways of Anticipating a TREND REVERSAL.f315.webm'
ERROR: Unable to rename file: [Errno 2] No such file or directory: 'subscriptions/channels/FractalFlow/16 Ways of Anticipating a TREND REVERSAL.f251.webm.part' -> 'subscriptions/channels/FractalFlow/16 Ways of Anticipating a TREND REVERSAL.f251.webm'
WARNING: Cannot update utime of file
ERROR: [Errno 2] No such file or directory: 'subscriptions/channels/FractalFlow/16 Ways of Anticipating a TREND REVERSAL.temp.mp4' -> 'subscriptions/channels/FractalFlow/16 Ways of Anticipating a TREND REVERSAL.mp4'
<--- Last few GCs --->
[73:0x49d97f0] 2170200 ms: Scavenge 4024.1 (4113.1) -> 4023.7 (4123.4) MB, 22.6 / 0.0 ms (average mu = 0.521, current mu = 0.463) allocation failure
[73:0x49d97f0] 2170323 ms: Scavenge 4030.2 (4123.4) -> 4029.0 (4124.1) MB, 30.1 / 0.0 ms (average mu = 0.521, current mu = 0.463) task
[73:0x49d97f0] 2173543 ms: Scavenge 4031.3 (4124.1) -> 4029.7 (4144.6) MB, 3149.3 / 0.0 ms (average mu = 0.521, current mu = 0.463) allocation failure
<--- JS stacktrace --->
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
1: 0xb09980 node::Abort() [node /app/app.js]
2: 0xa1c235 node::FatalError(char const*, char const*) [node /app/app.js]
3: 0xcf784e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node /app/app.js]
4: 0xcf7bc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node /app/app.js]
5: 0xeaf465 [node /app/app.js]
6: 0xebf12d v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node /app/app.js]
7: 0xec1e2e v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node /app/app.js]
8: 0xe830a2 v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [node /app/app.js]
9: 0xe7b6b4 v8::internal::FactoryBasev8::internal::Factory::AllocateRawWithImmortalMap(int, v8::internal::AllocationType, v8::internal::Map, v8::internal::AllocationAlignment) [node /app/app.js]
10: 0xe7d3c0 v8::internal::FactoryBasev8::internal::Factory::NewRawOneByteString(int, v8::internal::AllocationType) [node /app/app.js]
11: 0xe851fd v8::internal::Factory::NewStringFromUtf8(v8::base::Vector const&, v8::internal::AllocationType) [node /app/app.js]
12: 0xd06415 v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::NewStringType, int) [node /app/app.js]
13: 0xbf5139 [node /app/app.js]
14: 0xae3eb5 [node /app/app.js]
15: 0x15856cc [node /app/app.js]
2023-08-07T17:26:46: PM2 log: App [YoutubeDL-Material:0] exited with code [0] via signal [SIGABRT]
2023-08-07T17:26:46: PM2 log: App [YoutubeDL-Material:0] starting in -fork mode-
2023-08-07T17:26:46: PM2 log: App [YoutubeDL-Material:0] online
2023-08-07T07:26:51.420Z INFO: Config items set using ENV variables.
(node:179) DeprecationWarning: uuidv4() is deprecated. Use v4() from the uuid module instead.
(Use node --trace-deprecation ... to show where the warning was created)
2023-08-07T07:26:51.913Z INFO: YoutubeDL-Material v4.3.2 started on PORT 17442
2023-08-07T07:26:51.922Z VERBOSE: Watching FractalFlow with delay interval of 86400000
2023-08-07T07:26:51.964Z VERBOSE: Subscription: getting list of videos to download for FractalFlow with args: --dump-json,-o,"subscriptions/channels/FractalFlow/%(title)s.%(ext)s",-ciw,--write-info-json,--print-json,-f,bestvideo[ext=mp4]+bestaudio[ext=m4a]/mp4,--cookies,appdata/cookies.txt,--write-thumbnail,--no-clean-info-json
2023-08-07T07:27:04.018Z VERBOSE: Disabling checking downloads as none are available.
2023-08-07T07:27:07.679Z VERBOSE: Disabling checking downloads as none are available.
2023-08-07T07:27:07.970Z VERBOSE: Disabling checking downloads as none are available.
2023-08-07T07:27:10.162Z VERBOSE: Disabling checking downloads as none are available.
2023-08-07T07:27:10.477Z VERBOSE: Disabling checking downloads as none are available.

[therealjoejennings]

I'm having the same issue every time I try to download anything using the URL link. A subscription download did sneak through after the errors began but I haven't seen one since that one made it earlier today and at least one other should have been downloaded since.

The Docker starts without error then when I try to download I get the FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory

I'm running the Docker using Docker Compose in an Ubuntu VM inside Proxmox. It worked perfectly for days then this issue sprang up. I've tried multiple versions of the Docker and the different downloader variables, same result.

Docker Compose environment variables

    environment: 
        ytdl_mongodb_connection_string: 'mongodb://ytdl-mongo-db:27017'
        ytdl_use_local_db: 'false'
        write_ytdl_config: 'true'
        max_old_space_size: 18048

[entrypoint] setup permission, this may take a while

backend@1.0.0 start
pm2-runtime --raw pm2.config.js

2023-09-07T02:35:47: PM2 log: Launching in no daemon mode
2023-09-07T02:35:47: PM2 log: [Watch] Start watching YoutubeDL-Material
2023-09-07T02:35:47: PM2 log: App [YoutubeDL-Material:0] starting in -fork mode-
2023-09-07T02:35:47: PM2 log: App [YoutubeDL-Material:0] online
2023-09-07T02:35:48.353Z INFO: Config items set using ENV variables.
(node:36) DeprecationWarning: uuidv4() is deprecated. Use v4() from the uuid module instead.
(Use node --trace-deprecation ... to show where the warning was created)
2023-09-07T02:35:48.483Z INFO: Restarting server...
2023-09-07T02:35:48: PM2 log: App [YoutubeDL-Material:0] exited with code [1] via signal [SIGINT]
2023-09-07T02:35:48: PM2 log: App [YoutubeDL-Material:0] starting in -fork mode-
2023-09-07T02:35:48: PM2 log: App [YoutubeDL-Material:0] online
2023-09-07T02:35:49.589Z INFO: Config items set using ENV variables.
(node:47) DeprecationWarning: uuidv4() is deprecated. Use v4() from the uuid module instead.
(Use node --trace-deprecation ... to show where the warning was created)
2023-09-07T02:35:51.319Z INFO: YoutubeDL-Material v4.3.2 started on PORT 17442
<--- Last few GCs --->
[47:0x49f67f0] 50472 ms: Scavenge 2034.8 (2080.4) -> 2034.5 (2085.6) MB, 5.4 / 0.0 ms (average mu = 0.291, current mu = 0.188) allocation failure
[47:0x49f67f0] 50490 ms: Scavenge 2037.6 (2085.6) -> 2037.1 (2086.1) MB, 7.7 / 0.0 ms (average mu = 0.291, current mu = 0.188) allocation failure
[47:0x49f67f0] 50800 ms: Scavenge 2038.4 (2086.1) -> 2037.8 (2096.1) MB, 304.0 / 0.0 ms (average mu = 0.291, current mu = 0.188) allocation failure
<--- JS stacktrace --->
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
1: 0xb09980 node::Abort() [node /app/app.js]
2: 0xa1c235 node::FatalError(char const*, char const*) [node /app/app.js]
3: 0xcf784e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node /app/app.js]
4: 0xcf7bc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node /app/app.js]
5: 0xeaf465 [node /app/app.js]
6: 0xebf12d v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node /app/app.js]
7: 0xec1e2e v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node /app/app.js]
8: 0xe8336a v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node /app/app.js]
9: 0x11fc0b6 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node /app/app.js]
10: 0x15f0b19 [node /app/app.js]
2023-09-07T02:36:39: PM2 log: App [YoutubeDL-Material:0] exited with code [0] via signal [SIGABRT]
2023-09-07T02:36:39: PM2 log: App [YoutubeDL-Material:0] starting in -fork mode-
2023-09-07T02:36:39: PM2 log: App [YoutubeDL-Material:0] online
2023-09-07T02:36:40.611Z INFO: Config items set using ENV variables.
(node:68) DeprecationWarning: uuidv4() is deprecated. Use v4() from the uuid module instead.
(Use node --trace-deprecation ... to show where the warning was created)
2023-09-07T02:36:40.945Z INFO: YoutubeDL-Material v4.3.2 started on PORT 17442