Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cpp-tutorial can't find external linux toolchain #383

Open
inmcm opened this issue Dec 12, 2023 · 4 comments
Open

cpp-tutorial can't find external linux toolchain #383

inmcm opened this issue Dec 12, 2023 · 4 comments

Comments

@inmcm
Copy link

inmcm commented Dec 12, 2023

Trying to run the cpp-tutorial as written on amd64 Ubuntu 22.04, the compilation process cannot find the external toolchain.

$ cd stage1
$ bazelisk build //main:hello-world
Starting local Bazel server and connecting to it...
WARNING: --enable_bzlmod is set, but no MODULE.bazel file was found at the workspace root. Bazel will create an empty MODULE.bazel file. Please consider migrating your external dependencies from WORKSPACE to MODULE.bazel. For more details, please refer to https://github.com/bazelbuild/bazel/issues/18958.
INFO: Deleting stale sandbox base /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox
INFO: Analyzed target //main:hello-world (69 packages loaded, 6451 targets configured).
ERROR: /home/calvin/dev/bazel_tutorial/examples/cpp-tutorial/stage1/main/BUILD:3:10: Compiling main/hello-world.cc failed: (Exit 127): gcc failed: error executing CppCompile command (from target //main:hello-world) external/gcc_toolchain_x86_64/bin/gcc -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 32 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
external/gcc_toolchain_x86_64/bin/gcc: line 45: /tmp/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-gcc: No such file or directory
Target //main:hello-world failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 21.315s, Critical Path: 0.10s
INFO: 5 processes: 5 internal.
ERROR: Build did NOT complete successfully

Re-running with verbose flags enabled:

$ bazelisk build --verbose_failures --sandbox_debug //main:hello-world
INFO: Analyzed target //main:hello-world (0 packages loaded, 0 targets configured).
DEBUG: Sandbox debug output for CppCompile //main:hello-world: 1702409605.535640638: src/main/tools/linux-sandbox.cc:156: calling pipe(2)...
1702409605.535654032: src/main/tools/linux-sandbox.cc:165: Netns is 0
1702409605.535655716: src/main/tools/linux-sandbox.cc:176: calling clone(2)...
1702409605.535783997: src/main/tools/linux-sandbox.cc:185: linux-sandbox-pid1 has PID 12205
1702409605.535823373: src/main/tools/linux-sandbox-pid1.cc:700: Pid1Main started
1702409605.535891975: src/main/tools/linux-sandbox.cc:202: done manipulating pipes
1702409605.535925423: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/execroot -> /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-execroot
1702409605.535936586: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/execroot -> /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-working-directory
1702409605.535941766: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/external/gcc_toolchain_x86_64 -> /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-source-roots/0
1702409605.535946741: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/external/gcc_toolchain_x86_64_files -> /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-source-roots/1
1702409605.535951498: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/external/sysroot_x86_64 -> /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-source-roots/2
1702409605.535956091: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /home/calvin/dev/bazel_tutorial/examples/cpp-tutorial/stage1 -> /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-source-roots/3
1702409605.535961077: src/main/tools/linux-sandbox-pid1.cc:302: bind mount: /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp -> /tmp
1702409605.535969328: src/main/tools/linux-sandbox-pid1.cc:311: writable: /tmp/bazel-execroot/_main
1702409605.535973317: src/main/tools/linux-sandbox-pid1.cc:311: writable: /tmp
1702409605.535980622: src/main/tools/linux-sandbox-pid1.cc:311: writable: /dev/shm
1702409605.535984796: src/main/tools/linux-sandbox-pid1.cc:327: working dir: /tmp/bazel-working-directory/_main
1702409605.536025133: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /
1702409605.536030453: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev
1702409605.536033641: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev/pts
1702409605.536036625: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /dev/shm
1702409605.536039540: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev/mqueue
1702409605.536042036: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /dev/hugepages
1702409605.536044877: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run
1702409605.536047508: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/lock
1702409605.536050598: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/credentials/systemd-sysusers.service
1702409605.536053381: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/rpc_pipefs
1702409605.536056477: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/snapd/ns
1702409605.536059526: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /run/user/1000
1702409605.536062440: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys
1702409605.536079248: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/kernel/security
1702409605.536087480: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/cgroup
1702409605.536090884: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/pstore
1702409605.536093995: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/firmware/efi/efivars
1702409605.536098030: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/bpf
1702409605.536100992: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/kernel/debug
1702409605.536104016: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/kernel/tracing
1702409605.536106895: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/fs/fuse/connections
1702409605.536110174: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /sys/kernel/config
1702409605.536113310: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /proc
1702409605.536116281: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /proc/sys/fs/binfmt_misc
1702409605.536120153: src/main/tools/linux-sandbox-pid1.cc:427: remount(nullptr, /proc/sys/fs/binfmt_misc, nullptr, 2101281, nullptr) failure (Operation not permitted) ignored
1702409605.536123375: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /proc/sys/fs/binfmt_misc
1702409605.536126086: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /proc/fs/nfsd
1702409605.536129194: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /mnt/keys
1702409605.536141526: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/core18/2812
1702409605.536144536: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/core18/2796
1702409605.536146964: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/core20/2015
1702409605.536149392: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/core20/2105
1702409605.536151996: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/core22/1033
1702409605.536154327: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/core22/864
1702409605.536156689: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/go/10426
1702409605.536159395: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/google-cloud-cli/193
1702409605.536161946: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/google-cloud-cli/195
1702409605.536164253: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/lxd/23541
1702409605.536166705: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/lxd/24322
1702409605.536169011: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/snapd/20092
1702409605.536171525: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/snapd/20290
1702409605.536184167: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/yq/2307
1702409605.536186855: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/yq/2407
1702409605.536189100: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /boot
1702409605.536191914: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /boot/efi
1702409605.536194507: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /snap/go/10455
1702409605.536198573: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-execroot
1702409605.536201971: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-working-directory
1702409605.536204727: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-source-roots/0
1702409605.536214910: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-source-roots/1
1702409605.536217741: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-source-roots/2
1702409605.536222633: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-source-roots/3
1702409605.536225462: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp
1702409605.536227781: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-execroot
1702409605.536230198: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-working-directory
1702409605.536232415: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/0
1702409605.536234653: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/1
1702409605.536236933: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/2
1702409605.536239224: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/3
1702409605.536247648: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp/bazel-execroot/_main
1702409605.536250131: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp
1702409605.536252251: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-execroot
1702409605.536254349: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp/bazel-execroot/_main
1702409605.536256418: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-working-directory
1702409605.536258567: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/0
1702409605.536260752: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/1
1702409605.536262953: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/2
1702409605.536265102: src/main/tools/linux-sandbox-pid1.cc:405: remount ro: /tmp/bazel-source-roots/3
1702409605.536267595: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /dev/shm
1702409605.536269986: src/main/tools/linux-sandbox-pid1.cc:405: remount rw: /tmp/bazel-working-directory/_main
1702409605.536293649: src/main/tools/linux-sandbox-pid1.cc:496: calling fork...
1702409605.536351893: src/main/tools/linux-sandbox-pid1.cc:533: child started with PID 2
1702409605.538801294: src/main/tools/linux-sandbox-pid1.cc:550: wait returned pid=2, status=0x7f00
1702409605.538806213: src/main/tools/linux-sandbox-pid1.cc:568: child exited normally with code 127
1702409605.538989528: src/main/tools/linux-sandbox.cc:243: child exited normally with code 127
ERROR: /home/calvin/dev/bazel_tutorial/examples/cpp-tutorial/stage1/main/BUILD:3:10: Compiling main/hello-world.cc failed: (Exit 127): linux-sandbox failed: error executing CppCompile command 
  (cd /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/execroot/_main && \
  exec env - \
    BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 \
    PATH=/bin:/usr/bin:/usr/local/bin \
    PWD=/proc/self/cwd \
    TMPDIR=/tmp \
  /home/calvin/.cache/bazel/_bazel_calvin/install/89a68939cbf63eb54205fdf943a58b15/linux-sandbox -W /tmp/bazel-working-directory/_main -t 15 -w /tmp/bazel-execroot/_main -w /tmp -w /dev/shm -M /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/execroot -m /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-execroot -M /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/execroot -m /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-working-directory -M /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/external/gcc_toolchain_x86_64 -m /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-source-roots/0 -M /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/external/gcc_toolchain_x86_64_files -m /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-source-roots/1 -M /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/external/sysroot_x86_64 -m /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-source-roots/2 -M /home/calvin/dev/bazel_tutorial/examples/cpp-tutorial/stage1 -m /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp/bazel-source-roots/3 -M /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/_hermetic_tmp -m /tmp -S /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/stats.out -D /home/calvin/.cache/bazel/_bazel_calvin/5a1798d49a48b8187b8977a10ae22990/sandbox/linux-sandbox/3/debug.out -- external/gcc_toolchain_x86_64/bin/gcc -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.d '-frandom-seed=bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o' -fPIC -iquote . -iquote bazel-out/k8-fastbuild/bin -iquote external/bazel_tools -iquote bazel-out/k8-fastbuild/bin/external/bazel_tools --sysroot external/sysroot_x86_64/ -no-canonical-prefixes -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' '-fdiagnostics-color=always' -nostdinc -nostdinc++ -Bexternal/gcc_toolchain_x86_64/bin -isystemexternal/sysroot_x86_64//include/c++/10.3.0 -isystemexternal/sysroot_x86_64//include/c++/10.3.0/x86_64-linux -isystemexternal/sysroot_x86_64//lib/gcc/x86_64-linux/10.3.0/include-fixed -isystemexternal/sysroot_x86_64//lib/gcc/x86_64-linux/10.3.0/include -isystemexternal/sysroot_x86_64//usr/include -c main/hello-world.cc -o bazel-out/k8-fastbuild/bin/main/_objs/hello-world/hello-world.pic.o)
external/gcc_toolchain_x86_64/bin/gcc: line 45: /tmp/external/gcc_toolchain_x86_64_files/bin/x86_64-linux-gcc: No such file or directory
Target //main:hello-world failed to build
INFO: Elapsed time: 0.152s, Critical Path: 0.07s
INFO: 2 processes: 2 internal.
ERROR: Build did NOT complete successfully

Checking out commit 996a864 (the commit prior to #355 ) and the build works fine with my internal toolchain

$ bazelisk build //main:hello-world
WARNING: --enable_bzlmod is set, but no MODULE.bazel file was found at the workspace root. Bazel will create an empty MODULE.bazel file. Please consider migrating your external dependencies from WORKSPACE to MODULE.bazel. For more details, please refer to https://github.com/bazelbuild/bazel/issues/18958.
INFO: Analyzed target //main:hello-world (68 packages loaded, 305 targets configured).
INFO: Found 1 target...
Target //main:hello-world up-to-date:
  bazel-bin/main/hello-world
INFO: Elapsed time: 1.770s, Critical Path: 0.22s
INFO: 7 processes: 5 internal, 2 linux-sandbox.
INFO: Build completed successfully, 7 total actions

Running bazel version 7.0.0 (via bazelisk)

$ bazelisk version
Bazelisk version: v1.19.0
Build label: 7.0.0
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Mon Dec 11 16:51:49 2023 (1702313509)
Build timestamp: 1702313509
Build timestamp as int: 1702313509

Running on Ubuntu 22.04

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.3 LTS
Release:	22.04
Codename:	jammy
@sjwo
Copy link

sjwo commented Dec 13, 2023

Seems potentially related to recent thread in the Bazel getstarted slack: https://bazelbuild.slack.com/archives/CD42R4HJL/p1702485704389129. If it's the same issue, then you could try using Bazel 6.4, or specifying the --noincompatible_sandbox_hermetic_tmp build option with 7.0.0; both of these were successful workarounds for me.

@sjwo sjwo mentioned this issue Dec 13, 2023
Closed
@tarjintor
Copy link

I got the same problem,I dig somehow,find it seems the external/gcc_toolchain_x86_64/bin/gcc script parse the EXECROOT not correct,it goes to /tmp,but it seems you need to take bazel-working-directory/_main as your EXECROOT
the dir/libs all you need is all here
I am new to use bazel, not sure

@irshadcc
Copy link

irshadcc commented Feb 9, 2024

Can this option be made default or can this be mentioned in the cpp tutorial https://bazel.build/start/cpp

bazelbuild/bazel#19915

@vangyzen
Copy link

This issue is a significant impediment to new users trying to use the cpp-tutorial to learn bazel. That workflow is completely broken on Ubuntu 20.04 and openSUSE Leap 15.5. I spent two full days finding this issue, mostly because I assumed I was doing something wrong.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@inmcm @vangyzen @tarjintor @irshadcc @sjwo