Can't run integration tests

[solomatov]

Describe the bug
When I run the test command I got errors
[kostik@kostik-laptop j2cl]$ bazel test //transpiler/javatests/com/google/j2cl/transpiler/integration/...
ERROR: /home/kostik/Projects/j2cl/transpiler/javatests/com/google/j2cl/transpiler/integration/box2d/BUILD:8:1: no such package 'third_party/java/jbox2d': BUILD file not found on package path and referenced by '//transpiler/javatests/com/google/j2cl/transpiler/integration/box2d:box2d_j2cl'
ERROR: Analysis of target '//transpiler/javatests/com/google/j2cl/transpiler/integration/box2d_noclassmetadata_checksmin:compiled_test_bin' failed; build aborted: no such package 'third_party/java/jbox2d': BUILD file not found on package path
INFO: Elapsed time: 2.386s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (230 packages loaded, 65 targets configured)
FAILED: Build did NOT complete successfully (230 packages loaded, 65 targets configured)
currently loading: jre/java/javaemul/internal/vmbootstrap/primitives

To Reproduce
See the command above

Expected behavior
Tests pass or at least run successfully to completion.

[rluble]

Could you try on a clean repo? I cannot reproduce on a clean repo.

git clone https://github.com/google/j2cl.git
cd j2cl
bazel test //transpiler/javatests/com/google/j2cl/transpiler/integration/...

gets the expected result in my Linux machine:

Executed 201 out of 201 tests: 196 tests pass and 5 fail locally.

[solomatov]

I got the same result:

[kostik@kostik-laptop Projects]$ git clone https://github.com/google/j2cl.git
Cloning into 'j2cl'...
remote: Enumerating objects: 153, done.
remote: Counting objects: 100% (153/153), done.
remote: Compressing objects: 100% (107/107), done.
remote: Total 120507 (delta 47), reused 116 (delta 18), pack-reused 120354
Receiving objects: 100% (120507/120507), 31.94 MiB | 8.90 MiB/s, done.
Resolving deltas: 100% (96101/96101), done.
[kostik@kostik-laptop Projects]$ cd j2cl
[kostik@kostik-laptop j2cl]$ bazel test //transpiler/javatests/com/google/j2cl/transpiler/integration/...

Starting local Bazel server and connecting to it...
ERROR: /home/kostik/Projects/j2cl/transpiler/javatests/com/google/j2cl/transpiler/integration/box2d/BUILD:8:1: no such package 'third_party/java/jbox2d': BUILD file not found on package path and referenced by '//transpiler/javatests/com/google/j2cl/transpiler/integration/box2d:box2d_j2cl'
ERROR: Analysis of target '//transpiler/javatests/com/google/j2cl/transpiler/integration/box2d_noclassmetadata_checksmin:compiled_test_bin' failed; build aborted: no such package 'third_party/java/jbox2d': BUILD file not found on package path
INFO: Elapsed time: 4.798s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (270 packages loaded, 145 targets configured)
FAILED: Build did NOT complete successfully (270 packages loaded, 145 targets configured)

[rluble]

Could you just remove the deps in transpiler/javatests/com/google/j2cl/transpiler/integration/box2d/BUILD and see if you can build now? I am sending a patch to do that but since it does not reproduce in my setup I would like to be sure the fix I am sending works.

[solomatov]

It went farther, the error is different now:

ERROR: /home/kostik/.cache/bazel/_bazel_kostik/2a81219139f53d2833d3cdf6348d1cfd/external/bazel_tools/tools/jdk/BUILD:197:1: SkylarkAction external/bazel_tools/tools/jdk/platformclasspath.jar failed (Exit 1)
Exception in thread "main" java.io.FileNotFoundException: external/embedded_jdk/jre/lib/ext/jfxrt.jar (No such file or directory)
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.(ZipFile.java:225)
at java.util.zip.ZipFile.(ZipFile.java:155)
at java.util.jar.JarFile.(JarFile.java:166)
at java.util.jar.JarFile.(JarFile.java:130)
at DumpPlatformClassPath.writeClassPathJars(DumpPlatformClassPath.java:216)
at DumpPlatformClassPath.dumpJDK8BootClassPath(DumpPlatformClassPath.java:100)
at DumpPlatformClassPath.main(DumpPlatformClassPath.java:74)

[rluble]

Hmm. I am curious what platform you running under and which version of bazel?

[solomatov]

I am running on Fedora 29. I use open JDK 1.8.0.
Here's the version string of bazel:

Build label: 0.19.1- (@non-git)
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Mon Nov 12 16:31:12 2018 (1542040272)
Build timestamp: 1542040272
Build timestamp as int: 1542040272

[solomatov]

I installed javafx and accessibility parts of JDK and it seems to get better. But the error is different now:

ERROR: /home/kostik/Projects/j2cl/transpiler/javatests/com/google/j2cl/transpiler/integration/box2d/BUILD:8:1: Compiling Java headers transpiler/javatests/com/google/j2cl/transpiler/integration/box2d/libbox2d_j2cl-hjar.jar (2 files) failed (Exit 1)

transpiler/javatests/com/google/j2cl/transpiler/integration/box2d/Box2d.java:25: error: symbol not found org.jbox2d.dynamics.World
import org.jbox2d.dynamics.World;
^

INFO: Elapsed time: 122.499s, Critical Path: 87.10s, Remote (0.00% of the time): [queue: 0.00%, setup: 0.00%, process: 0.00%]
INFO: 756 processes: 441 local, 315 worker.

[gkdn]

There is tags "manual" on that target. That shouldn't build unless you specifically ask it to do.
What exact command are you running? Is there anything special in your bazelrc?

[solomatov]

I run the following command: bazel test //transpiler/javatests/com/google/j2cl/transpiler/integration/...
I don't have any additional bazelrc files.

[rluble]

That is definitely progress. bazel should not even try to build box2d (with the ... invocation) since the targets are marked tag=["manual"].

you can try with bazel test //transpiler/javatests/com/google/j2cl/transpiler/integration/... --keep_going to get unstuck.

[solomatov]

Thanks for you help! It passed forward much further than it used to! Most of the tests passed.

I understand that the difference between Google repo and what you have here is quite large and it creates a lot of problems. You can't just copy everything in one fell swoop. However, I am quite eager at least to try to contribute here, thus my attempts to do something.

[solomatov]

Here's the final message:

Executed 201 out of 201 tests: 196 tests pass and 5 fail locally.

[rluble]

Thank you for your interest. We are looking forward to your contribution! I think we are in most need to make the experience smoother.

W.r.t to failing tests, except for JsInteropRestrictionsCheckerTest that is failing due missing annotations in the public jsinterop-annotation package, the others seem to be failing in phantomjs (logging for example seems to crash phantomjs, the others I have not investigated).

I would be good to look at why they are failing, and see also why all the tests fail if compilation_level="BUNDLE" which would make them easier to debug.

Anyhow if you are looking for more specifics I'd be happy to point you in the right direction.

[solomatov]

And where can I find this option. The only place where I found it is j2cl_application.bzl

[rluble]

Do you mean compilation_level? That is an argument to closure_js_test, used in integration_test.bzl.

The option "BUNDLE" it is undocumented, but it produces the concatenated source with no compilation for faster iteration and easier debugging.

[solomatov]

I made the change, and the test failed. Here's the message:

-> ERROR: G_testRunner not defined. Did you remember to goog.require('goog.testing.jsunit')?

I got a resulting compiled_test_bin.js I create a simple page and loaded, this time, I had more error messages:

goog.require could not find: java.lang.AutoCloseable
goog.logToConsole_ @ compiled_test_bin.js:814
compiled_test_bin.js:2785 Uncaught Error: goog.require could not find: java.lang.AutoCloseable
    at goog.DebugLoader_.load_ (compiled_test_bin.js:2785)
    at Object.goog.require (compiled_test_bin.js:846)
    at compiled_test_bin.js:45806
    at Object.goog.loadModule (compiled_test_bin.js:1082)
    at compiled_test_bin.js:45796
deps.js:1 Failed to load resource: net::ERR_FILE_NOT_FOUND
compiled_test_bin.js:44084 Uncaught Error: No tests found in given test case: Untitled Test Case. By     default, the test runner fails if a test case has no tests. To modify this behavior, see    goog.testing.TestRunner's setStrict() method, or G_testRunner.setStrict()
    at goog.testing.TestRunner.execute (compiled_test_bin.js:44084)
    at compiled_test_bin.js:44491

Is it the right way to debug it?

[solomatov]

If I search the resulting file, I can't find a declaration for AutoCloseable$impl:

goog.module('java.lang.AutoCloseable$impl');

As far as I understand, it's a declaration of the module

const AutoCloseable = goog.require('java.lang.AutoCloseable$impl');

I had 6 occurences 1 of the 1st kind and 5 of the second.

[solomatov]

If I take a look at the resulting file without BUNDLE, there're no mentions of AutoCloseable$impl.

[rluble]

Interseting. Could you set ENABLE_DEBUG_LOADER to false? There is a goog.define in the .js output.

[solomatov]

@rluble If I set this way the page with the message that the test has passed is displayed.

[rluble]

Well that is progress. I am not sure how to disable the debug loader in bundled mode, but I think the test runner should give better messages when there are errors.

[solomatov]

And why is it implemented in this way? I.e. why transpiler doesn't include all deps? For efficiency purposes?

[rluble]

The deps are all there, but in order to avoid cyclical module dependencies, j2cl generates extra requires (which are only needed to make sure the relevant sources are included by jscompiler, this is due to a limitation of jscompiler module system and the fact that Java allows cyclic dependencies as long as the code is compiled together).

But in jscompiler bundled mode, these extra requires might appear before the the code for the actual module and the debug loader complains. I am not too familiar with the runtime details, but it is my understanding that the debug loader can be used to load modules one by one, but that is not what we are doing in bundled mode. That is why turning the ENABLE_DEBUG_LOADER to false works. If dependencies where actually missing that would be an error.

Are you sure AutoCloseable$impl is not there?