on linux x86-64 the binary is very large / can be 4.66x smaller with strip/upx --lzma

[alexmyczko]

although strip and compress cut the size, the binary does not work anymore. any idea why? any chance to make it smaller?

[jhheider]

the binary is, as i understand it, the deno binary with our code as a .DATA block. we know that patchelf breaks that loading, so i'm not surprised strip does as well.

if you can reduce the size of deno without breaking it, then compile pkgx, that's probably the way forward.

[mxcl]

basically it will be large until deno can make smaller binaries or we switch to rust. Sorry.

[alexmyczko]

So I've gotten the deno binary for x86-64
original size 139310360 # 139 MB
after strip and upx
new size 39192472 # 39 MB
Now I'm having a hard time to build deno from source, and even a harder time to build pkgx from source.
Would it be possible to use this binary to build a pkgx binary for x86-64 linux to see/test if it would actually work?
https://sid.ethz.ch/debian/deno/bin

[mxcl]

It’s unlikely that stripping deno would result in a smaller pkgx binary.

[alexmyczko]

-rwxr-xr-x 1 root root 139310360 Jan 25 17:06 deno
-rwxr-xr-x 1 root root 101139872 Jan 26 13:18 deno.strip
-rwxr-xr-x 1 root root  39192472 Jan 26 08:32 deno.strip.upx
-rwxr-xr-x 1 root root  29051348 Jan 26 13:18 deno.strip.upx--lzma

and

deno:                              ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[xxHash]=4fec45b0112d0342, with debug_info, not stripped
deno.strip:                        ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[xxHash]=4fec45b0112d0342, stripped
deno.strip.upx:                    ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), statically linked, no section header

using upx with --lzma it gets even below 30 mb!!! (factor 4.66 smaller)

[mxcl]

like, I'm just saying that the size of deno is unlikely to effect the size of the binaries it produces. But who knows, maybe.

Anyway, building pkgx is trivial. We have instructions in the readme, but basically it's deno task compile in a checkout.

[alexmyczko]

@mxcl my hero! it works, well... spits some warnings but it works:

https://sid.ethz.ch/debian/pkgx/build/pkgx/ the binary is 42 MB

and running it with

pkgx bc

gives

warning: Use of deprecated "Deno.stderr.rid" API. This API will be removed in Deno 2.

Stack trace:
  at file:///var/www/debian/pkgx/build/pkgx/entrypoint.ts:7:38

hint: Use `Deno.stderr` instance methods instead.

warning: Use of deprecated "Deno.isatty()" API. This API will be removed in Deno 2.

Stack trace:
  at default (file:///var/www/debian/pkgx/build/pkgx/src/utils/clicolor.ts:10:36)
  at file:///var/www/debian/pkgx/build/pkgx/entrypoint.ts:7:17

hint: Use `Deno.stdin.isTerminal()`, `Deno.stdout.isTerminal()` or `Deno.stderr.isTerminal()` instead.

warning: Use of deprecated "Deno.stdin.rid" API. This API will be removed in Deno 2.

Stack trace:
  at logger_prefix (file:///var/www/debian/pkgx/build/pkgx/entrypoint.ts:66:53)
  at file:///var/www/debian/pkgx/build/pkgx/entrypoint.ts:59:19

hint: Use `Deno.stdin` instance methods instead.

warning: Use of deprecated "Deno.isatty()" API. This API will be removed in Deno 2.

Stack trace:
  at logger_prefix (file:///var/www/debian/pkgx/build/pkgx/entrypoint.ts:66:35)
  at file:///var/www/debian/pkgx/build/pkgx/entrypoint.ts:59:19

hint: Use `Deno.stdin.isTerminal()`, `Deno.stdout.isTerminal()` or `Deno.stderr.isTerminal()` instead.

warning: Use of deprecated "Deno.stderr.rid" API. This API will be removed in Deno 2.

Stack trace:
  at make_logger (file:///var/www/debian/pkgx/build/pkgx/src/app.ts:148:51)
  at default (file:///var/www/debian/pkgx/build/pkgx/src/app.ts:33:18)
  at file:///var/www/debian/pkgx/build/pkgx/entrypoint.ts:59:9

hint: Use `Deno.stderr` instance methods instead.

warning: Use of deprecated "Deno.isatty()" API. This API will be removed in Deno 2.

Stack trace:
  at make_logger (file:///var/www/debian/pkgx/build/pkgx/src/app.ts:148:32)
  at default (file:///var/www/debian/pkgx/build/pkgx/src/app.ts:33:18)
  at file:///var/www/debian/pkgx/build/pkgx/entrypoint.ts:59:9

hint: Use `Deno.stdin.isTerminal()`, `Deno.stdout.isTerminal()` or `Deno.stderr.isTerminal()` instead.

bc 1.07.1
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006, 2008, 2012-2017 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'. 

[mxcl]

k well, this is interesting

[alexmyczko]

the more so since UPX is portable and you could also strip/upx --lzma your other binaries!

but instead opening a new issue. are you interested in distributing pkgx in debian, like in the archive? i can package software for it, however deno is not there yet...

[mxcl]

Interested in how it effects startup time.

Also we clearly need to pkg upx

And would love to get into Debian, but yeah, since they don't have deno yet it seems tricky

[alexmyczko]

memory/cpu > disk (no matter rotational, ssd, nvme). i don't notice anything on my 2011 machine.

for deno in debian there's https://bugs.debian.org/961337

[mxcl]

found some further info here: denoland/deno#9198

So indeed, stripping deno will lead to smaller compiled binaries. I worry that it may result in non functional binaries. Defaulting it is a risk for just-works but maybe that's the wrong trade off and we should aim for performance

[jhheider]

aside: stack traces for deprecated calls seems extreme.

[alexmyczko]

i tried the same strip/upx/upx --lzma on macOS. no luck

[alexmyczko]

is there a place to see all pkgx available binaries? and a way to search for it???

[jhheider]

did you mean pkgx? if so, you can review the packages here: https://pkgx.dev/pkgs/

most of the binaries are lists in the provides: keys in the various package.yml files. as always, the pkgxdev/pantry repository is canon.

[mxcl]

the search box will search for programs (binaries) that pkgs provide

[felipecrs]

BTW #978 will make pkgx binaries a little smaller.