Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] JDP is broken on ARM64 images #1553

Open
andrewazores opened this issue Jun 15, 2023 · 7 comments
Open

[Bug] JDP is broken on ARM64 images #1553

andrewazores opened this issue Jun 15, 2023 · 7 comments
Labels
bug Something isn't working question Further information is requested

Comments

@andrewazores
Copy link
Member

https://github.com/cryostatio/cryostat/pull/1544#issuecomment-1593114707
@andrewazores andrewazores added the bug Something isn't working label Jun 15, 2023
@andrewazores andrewazores self-assigned this Jun 15, 2023
@andrewazores andrewazores moved this to Backlog in 2.4.0 release Jun 15, 2023
@andrewazores
Copy link
Member Author

Similar investigations here: hazelcast/hazelcast#9081

@andrewazores
Copy link
Member Author

ARM64 image:

$ podman exec -it cryostat /bin/bash
[root@cryostat ~]# microdnf install iproute

...

[root@cryostat ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: tap0: <BROADCAST,UP,LOWER_UP> mtu 65520 qdisc fq_codel state UNKNOWN group default qlen 1000
    link/ether 52:02:80:da:5b:02 brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.100/24 brd 10.0.2.255 scope global tap0
       valid_lft forever preferred_lft forever
    inet6 fd00::5002:80ff:feda:5b02/64 scope global dynamic mngtmpaddr
       valid_lft 86357sec preferred_lft 14357sec
    inet6 fe80::5002:80ff:feda:5b02/64 scope link
       valid_lft forever preferred_lft forever

AMD64 image:

$ podman exec -it cryostat /bin/bash
[root@cryostat ~]# microdnf install iproute

...

[root@cryostat ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: tap0: <BROADCAST,UP,LOWER_UP> mtu 65520 qdisc fq_codel state UNKNOWN group default qlen 1000
    link/ether ba:c8:99:1a:d9:96 brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.100/24 brd 10.0.2.255 scope global tap0
       valid_lft forever preferred_lft forever
    inet6 fd00::b8c8:99ff:fe1a:d996/64 scope global dynamic mngtmpaddr
       valid_lft 86381sec preferred_lft 14381sec
    inet6 fe80::b8c8:99ff:fe1a:d996/64 scope link
       valid_lft forever preferred_lft forever

Looks fine that way.

@andrewazores
Copy link
Member Author 

diff --git a/src/main/extras/app/entrypoint.bash b/src/main/extras/app/entrypoint.bash
index cd83fb51..7748f320 100755
--- a/src/main/extras/app/entrypoint.bash
+++ b/src/main/extras/app/entrypoint.bash
@@ -166,6 +166,8 @@ if [ -n "$CRYOSTAT_JUL_CONFIG" ]; then
     FLAGS+=("-Djava.util.logging.config.file=$CRYOSTAT_JUL_CONFIG")
 fi

+FLAGS+=("-Djava.net.preferIPv4Stack=true")
+
 CLASSPATH="$( cat /app/jib-classpath-file )"
 if [ -n "$CRYOSTAT_CLIENTLIB_PATH" ]; then
     CLASSPATH="$CLASSPATH:$CRYOSTAT_CLIENTLIB_PATH/*"

didn't help. A slightly different stack trace occurs:

INFO: JDP Discovery started
Jun 15, 2023 2:17:40 PM io.cryostat.core.log.Logger warn
WARNING: Exception thrown
java.net.SocketException: Protocol not available
	at java.base/sun.nio.ch.Net.getInterface4(Native Method)
	at java.base/sun.nio.ch.DatagramChannelImpl.getOption(DatagramChannelImpl.java:417)
	at java.base/sun.nio.ch.DatagramSocketAdaptor.getOption(DatagramSocketAdaptor.java:426)
	at java.base/sun.nio.ch.DatagramSocketAdaptor.outgoingNetworkInterface(DatagramSocketAdaptor.java:657)
	at java.base/sun.nio.ch.DatagramSocketAdaptor.defaultNetworkInterface(DatagramSocketAdaptor.java:672)
	at java.base/sun.nio.ch.DatagramSocketAdaptor.joinGroup(DatagramSocketAdaptor.java:522)
	at java.base/sun.nio.ch.DatagramSocketAdaptor.joinGroup(DatagramSocketAdaptor.java:481)
	at java.base/java.net.MulticastSocket.joinGroup(MulticastSocket.java:325)
	at org.openjdk.jmc.jdp.client.JDPClient.start(JDPClient.java:87)
	at io.cryostat.core.net.discovery.JvmDiscoveryClient.start(JvmDiscoveryClient.java:105)
	at io.cryostat.platform.internal.DefaultPlatformClient.start(DefaultPlatformClient.java:83)
	at io.cryostat.discovery.BuiltInDiscovery.lambda$start$5(BuiltInDiscovery.java:134)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.DistinctOps$1$2.accept(DistinctOps.java:174)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1707)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at io.cryostat.discovery.BuiltInDiscovery.start(BuiltInDiscovery.java:97)
	at io.vertx.core.impl.DeploymentManager.lambda$doDeploy$5(DeploymentManager.java:196)
	at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:264)
	at io.vertx.core.impl.WorkerContext.lambda$run$3(WorkerContext.java:106)
	at io.vertx.core.impl.WorkerContext.lambda$null$1(WorkerContext.java:92)
	at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)

This time at the Net.getInterface4 instead of Net.getInterface6, so it looks like this property did have an effect, but the protocol is not available on either an IPv4 or IPv6 interface.

@andrewazores
Copy link
Member Author

Here is the native implementation of getInterface6: https://github.com/openjdk/jdk17u-dev/blob/f9a60759b13f242c45bef8506e25ffea3e972247/src/java.base/unix/native/libnio/ch/Net.c#L816

@andrewazores
Copy link
Member Author

andrewazores commented Jun 15, 2023 •
edited
Loading

https://github.com/openjdk/jdk17u-dev/blob/f9a60759b13f242c45bef8506e25ffea3e972247/src/java.base/unix/native/libnio/ch/Net.c#LL822C13-L822C13

Probably falling through to: https://github.com/openjdk/jdk17u-dev/blob/f9a60759b13f242c45bef8506e25ffea3e972247/src/java.base/unix/native/libnio/ch/Net.c#L132

@andrewazores
Copy link
Member Author

Also somewhat similar problem: fit2anything/keycloak-cluster-setup-and-configuration#8

@andrewazores
Copy link
Member Author

In https://github.com/cryostatio/cryostat/issues/1329 , a user with access to real ARM hardware seemingly had no issue getting Cryostat running. There are no logs showing whether the DefaultPlatformClient was enabled and selected however, but the Deployment did not have configuration to disable it, so I believe it likely was running and working.

Might this JDP issue only be a problem when running the ARM image through qemu?

@andrewazores andrewazores added the question Further information is requested label Jun 15, 2023
@andrewazores andrewazores removed their assignment Jun 15, 2023
@andrewazores andrewazores mentioned this issue Jun 16, 2023
Draft
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working question Further information is requested
Projects
No open projects
2.4.0 release
Status: Backlog
Milestone
No milestone
Development

No branches or pull requests
1 participant
@andrewazores