isce3 minor build/ test questions - v 0.4 #5

piyushrpt · 2021-07-21T21:12:20Z

Thanks for the new release. It pretty much built out of the box except for some minor issues.

There appears to be some discrepancy in the way isce3 and pyre discover python versions to use. I eventually had to use both these settings:
-DPython_EXECUTABLE=path-to-python3 -DPython3_EXECUTABLE=path-to-python3 to get everything aligned.
Some the tests fail with ctest possibly because they are trying to reach out to a s3 bucket to get data. Is there a flag / option to disable these tests.
yamale is an undocumented dependency. Would it be useful to add this test at cmake stage or have it documented.

The text was updated successfully, but these errors were encountered:

rtburns-jpl · 2021-07-21T21:31:45Z

Regarding your first point, pyre is now using find_package(Python) just like isce3 so once we bump pyre you can get by with just -DPython_EXECUTABLE.

Also, it sounds like you are letting cmake fetch pyre at build time - you can speed up the build a bit using prebuilt pyre 1.9.9 from conda-forge, and won't have to worry about pyre configuration parameters.

piyushrpt · 2021-07-21T21:44:02Z

I was building this on my mac. I will switch to using pyre from conda-forge on linux.

gmgunter · 2021-07-21T22:20:10Z

Hi @piyushrpt, thanks for testing out the new release and keeping us honest!

Some the tests fail with ctest possibly because they are trying to reach out to a s3 bucket to get data.

I think currently the only test that should require s3 bucket access is test.python.pkg.pybind_nisar.workflows.stage_dem. If any other tests are failing unexpectedly, please let us know.

Is there a flag / option to disable these tests

It's on our TODO list for some future release to add unit test markers so that you can easily filter out these tests, but there's nothing like this right now, sorry.

For now, you could use

$ ctest -E "stage_dem"

to exclude the stage_dem test, which is expected to fail. Hopefully, the relevant S3 bucket will be made public sometime soon anyway.

yamale is an undocumented dependency. Would it be useful to add this test at cmake stage or have it documented.

Oops, I forgot to update the public docs pages. While trying to resolve this, I realized that our CI bot, which is supposed to be generating the internal documentation pages for the develop branch, got removed in the most recent round of CI cleanup. Let me see if we can resolve this & update the public documentation shortly.

piyushrpt · 2021-07-21T22:34:58Z

These were the other tests that failed. Digging deeper to see why

The following tests FAILED:
	 44 - test.cxx.isce3.geometry.bbox.geoperimeter_equator (Failed)
	140 - test.python.pkg.pybind_nisar.workflows.geocode_insar (Failed)
	146 - test.python.pkg.pybind_nisar.workflows.rubbersheet (Failed)

piyushrpt · 2021-07-21T22:48:37Z

The test.cxx.isce3.geometry.bbox.geoperimeter_equator is due to ogr->getNumPoints() counting duplicate vertex for closed polygons and reporting 42 points instead of 41, here:

isce3/tests/cxx/isce3/geometry/bbox/geoperimeter_equator.cpp

Line 153 in 59cdd2c

ASSERT_EQ( perimeter.getNumPoints(), 4 * nPtsPerEdge - 4 + 1);

piyushrpt · 2021-07-21T22:59:17Z

The other failures seem to be a transient issue - different order of tests running might resolve it. Maybe dependency between tests?

hfattahi · 2021-07-21T23:33:38Z

The other failures seem to be a transient issue - different order of tests running might resolve it. Maybe dependency between tests?

any more details for the rubbersheet and geocode_insar?

hfattahi · 2021-07-21T23:34:00Z

sorry that was closed by mistake

hfattahi · 2021-07-21T23:39:14Z

Just saw that it was on your mac. rubbersheet test should not be run on CPU since we have only GPU ampcor.

vbrancat · 2021-07-22T00:27:44Z

Give the dependency in the CMakeFile (

isce3/tests/python/packages/CMakeLists.txt

Line 23 in 59cdd2c

if(WITH_CUDA)

), the rubbersheet unit test should not run if the machine has no GPU :s

vbrancat · 2021-07-22T01:25:03Z

Found why the rubbersheet unit test gets executed also when no GPU is available. This line

isce3/tests/python/packages/CMakeLists.txt

Line 18 in 59cdd2c

pybind_nisar/workflows/rubbersheet.py

should not be here. I will submit a PR to solve the issue :s

hfattahi · 2021-07-22T02:28:26Z

Found why the rubbersheet unit test gets executed also when no GPU is available. This line

isce3/tests/python/packages/CMakeLists.txt

Line 18 in 59cdd2c

pybind_nisar/workflows/rubbersheet.py

should not be here. I will submit a PR to solve the issue :s

Great that should take care of one failed test on CPU. I suspect geocode_insar is looking for geocoding the offsets which don't exist on CPU! need to take a closer look.

piyushrpt · 2021-07-22T06:11:28Z

Is there a shortcut to determine if CUDA support is included in a build? Either via isce3/ pyre - something like

isce3.hasCudaSupport or pyre.hasCudaSupport

Or maybe this can be determined by trying to import a cuda-based module and catching an import error.

geocode also is related to offsets from a quick glance at the code.

gmgunter · 2021-07-22T18:25:20Z

Is there a shortcut to determine if CUDA support is included in a build?

Yup, on the python side, you can do

import isce3
hasattr(isce3, "cuda") # True if CUDA support is enabled

On the C++ side, you can use

#include <isce3/config.h>

#if ISCE3_WITH_CUDA
// CUDA support is enabled
#else
// no CUDA support
#endif

update

Narayana-Rao · 2022-05-10T16:46:27Z

Is it possible to build and install ISCE3 by manually disabling CUDACXX ? The GPU I am using doesn't support CUDA>9.0.

CMake Error at /usr/local/share/cmake-3.23/Modules/CMakeDetermineCUDACompiler.cmake:180 (message):
  Failed to find nvcc.

  Compiler requires the CUDA toolkit.  Please set the CUDAToolkit_ROOT
  variable.
Call Stack (most recent call first):
  CMakeLists.txt:47 (enable_language)

rtburns-jpl · 2022-05-10T16:54:58Z

Yes, by default cmake is detecting your existing cuda installation, but you can disable this entirely by configuring with the cmake arg -DWITH_CUDA=OFF.

Narayana-Rao · 2022-05-10T21:48:03Z

Thank you @rtburns-jpl, with -DWITH_CUDA=OFF arg, it is generating the following error (error_log.txt)


CMake Error at /usr/local/share/cmake-3.23/Modules/Internal/CheckFlagCommonConfig.cmake:58 (message):
  check_compiler_flag: CUDA: needs to be enabled before use.
Call Stack (most recent call first):
  /usr/local/share/cmake-3.23/Modules/Internal/CheckCompilerFlag.cmake:13 (cmake_check_flag_common_init)
  /usr/local/share/cmake-3.23/Modules/CheckCompilerFlag.cmake:40 (cmake_check_compiler_flag)
  .cmake/Warnings.cmake:67 (check_compiler_flag)
  .cmake/ConfigISCE.cmake:58 (set_warnings)
  CMakeLists.txt:86 (CheckCXX)

rtburns-jpl · 2022-05-11T01:04:03Z

Can you try deleting your build directory and starting again with a clean cache?

Narayana-Rao · 2022-05-11T20:07:00Z

Thank you @rtburns-jpl. After trying this I got 30 tests failed (Ubuntu 20.04; GDAL=3.0.4; Python=3.8). I have noticed that the majority of the failures are due to corrupted/incorrect paths of *.h5 files (I am not sure).
LastTest.log

0% tests passed, 30 tests failed out of 30

Total Test time (real) =  30.61 sec

The following tests FAILED:
	 87 - test.python.pybind.core.attitude (Failed)
	 98 - test.python.pybind.core.orbit (Failed)
	104 - test.python.pybind.focus.backproject (Failed)
	108 - test.python.pybind.geocode.geocodeSlc (Failed)
	109 - test.python.pybind.geocode.geocodeCov (Failed)
	110 - test.python.pybind.geocode.radar_grid_cube (Failed)
	111 - test.python.pybind.geometry.bbox (Failed)
	113 - test.python.pybind.geometry.geo2rdr (Failed)
	114 - test.python.pybind.geometry.rdr2geo (Failed)
	115 - test.python.pybind.geometry.rtc (Failed)
	116 - test.python.pybind.geometry.metadata_cubes (Failed)
	136 - test.python.pybind.geometry.look_inc_from_sr (Failed)
	139 - test.python.pybind.antenna.el_pattern_est (Failed)
	148 - test.python.pkg.nisar.products.readers.raw (Failed)
	149 - test.python.pkg.nisar.workflows.crossmul (Failed)
	150 - test.python.pkg.nisar.workflows.doppler_lut_from_raw (Failed)
	151 - test.python.pkg.nisar.workflows.focus (Failed)
	152 - test.python.pkg.nisar.workflows.gen_doppler_range_product (Failed)
	153 - test.python.pkg.nisar.workflows.stage_dem (Failed)
	154 - test.python.pkg.nisar.workflows.gcov (Failed)
	155 - test.python.pkg.nisar.workflows.geo2rdr (Failed)
	156 - test.python.pkg.nisar.workflows.geocode_insar (Failed)
	157 - test.python.pkg.nisar.workflows.gslc (Failed)
	158 - test.python.pkg.nisar.workflows.insar (Failed)
	159 - test.python.pkg.nisar.workflows.filter_interferogram (Failed)
	160 - test.python.pkg.nisar.workflows.bandpass_insar (Failed)
	162 - test.python.pkg.nisar.workflows.rdr2geo (Failed)
	163 - test.python.pkg.nisar.workflows.resample_slc (Failed)
	164 - test.python.pkg.nisar.workflows.unwrap (Failed)
	166 - test.python.pkg.nisar.antenna.beamformer (Failed)
Errors while running CTest

gmgunter · 2022-05-11T20:40:53Z

Hi @Narayana-Rao, isce3 uses two different software packages to interact with HDF5 files: the h5py package is used by isce3's Python code, and the underlying libhdf5 library is used directly by isce3's C++ code. Due to the (somewhat awkward) way that isce3 currently passes references to H5Objects between its C++ and Python layers, it's very important for these two packages to be compatible in an ABI sense.

I suspect that the issue in this case is that these two packages (h5py and libhdf5) in your environment are binary incompatible. How did you install these packages? Did you install them using conda or some other package manager?

The packages should be reliably compatible if they were both installed from conda-forge, e.g. via

$ conda install --channel=conda-forge h5py hdf5

However, if, for example, you installed libhdf5 using the APT package manager

$ apt-get install libhdf5-dev

Then you should install h5py from source rather than installing a binary package. Otherwise, you're likely to get a binary that links against a different libhdf5, which will cause issues like what you experienced. In that case, you can try installing h5py from source using pip:

$ pip install --no-binary=h5py h5py

Hope that helps.

Narayana-Rao · 2022-05-11T21:24:53Z

Thank you @gmgunter, it has solved issues with the majority of tests but not all. Here is the list of tests that failed, mostly with E OSError: Unable to open file (file is not already open for SWMR writing) . I am using h5py=3.6.0.

LastTest.log

The following tests FAILED:
	139 - test.python.pybind.antenna.el_pattern_est (Failed)
	152 - test.python.pkg.nisar.workflows.gen_doppler_range_product (Failed)
	153 - test.python.pkg.nisar.workflows.stage_dem (Failed)
	165 - test.python.pkg.nisar.workflows.antenna_parser (Failed)
	166 - test.python.pkg.nisar.antenna.beamformer (Failed)
Errors while running CTest

Narayana-Rao · 2022-05-13T18:25:44Z

After a fresh installation of Ubuntu, now, I am able to successfully install and run tests (except stage_dem ) (Ubuntu 20.04; GDAL 3.0.4; Python 3.8.5). Thanks a lot, @gmgunter @rtburns-jpl for the support.

Narayana-Rao · 2022-08-15T22:02:26Z

I am trying to install and use plant-isce3. Therefore, I have installed latest isce3 ('0.8.0-dev+04b17713'). The following test are failed (Ubuntu 20.04; GDAL 3.0.4; Python 3.8.5)
LastTest.log

140:test.python.pkg.isce3.ionosphere.ionosphere
153:test.python.pkg.nisar.workflows.stage_dem
162:test.python.pkg.nisar.workflows.ionosphere

Still I have installed plant, plant-isce3. However, while running plant_isce3_geocode.py, I have encountered the following error (plant_isce3.log). Could anyone please suggest any fix for this.

AttributeError: 'isce3.ext.isce3.geocode.GeocodeCFloat32' object has no attribute 'dem_block_margin'

piyushrpt closed this as completed Jul 21, 2021

gmgunter reopened this Jul 21, 2021

hfattahi closed this as completed Jul 21, 2021

hfattahi reopened this Jul 21, 2021

seongsujeong mentioned this issue Nov 4, 2021

Building ISCE3 0.4.0 from source code w/ CUDA capability #8

Open

LiangJYu pushed a commit to LiangJYu/isce3 that referenced this issue Apr 16, 2022

Merge pull request isce-framework#5 from isce-3/develop

1b1b85d

update

Narayana-Rao mentioned this issue May 9, 2022

ctest 77% tests passed, 36 tests failed out of 159 (Debian GNU/Linux 10 ) #9

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

isce3 minor build/ test questions - v 0.4 #5

isce3 minor build/ test questions - v 0.4 #5

piyushrpt commented Jul 21, 2021

rtburns-jpl commented Jul 21, 2021 •

edited

Loading

piyushrpt commented Jul 21, 2021

gmgunter commented Jul 21, 2021

piyushrpt commented Jul 21, 2021

piyushrpt commented Jul 21, 2021 •

edited

Loading

piyushrpt commented Jul 21, 2021

hfattahi commented Jul 21, 2021 •

edited

Loading

hfattahi commented Jul 21, 2021

hfattahi commented Jul 21, 2021

vbrancat commented Jul 22, 2021

vbrancat commented Jul 22, 2021

hfattahi commented Jul 22, 2021

piyushrpt commented Jul 22, 2021 •

edited

Loading

gmgunter commented Jul 22, 2021 •

edited

Loading

Narayana-Rao commented May 10, 2022

rtburns-jpl commented May 10, 2022

Narayana-Rao commented May 10, 2022 •

edited

Loading

rtburns-jpl commented May 11, 2022

Narayana-Rao commented May 11, 2022

gmgunter commented May 11, 2022

Narayana-Rao commented May 11, 2022 •

edited

Loading

Narayana-Rao commented May 13, 2022 •

edited

Loading

Narayana-Rao commented Aug 15, 2022 •

edited

Loading

isce3 minor build/ test questions - v 0.4 #5

isce3 minor build/ test questions - v 0.4 #5

Comments

piyushrpt commented Jul 21, 2021

rtburns-jpl commented Jul 21, 2021 • edited Loading

piyushrpt commented Jul 21, 2021

gmgunter commented Jul 21, 2021

piyushrpt commented Jul 21, 2021

piyushrpt commented Jul 21, 2021 • edited Loading

piyushrpt commented Jul 21, 2021

hfattahi commented Jul 21, 2021 • edited Loading

hfattahi commented Jul 21, 2021

hfattahi commented Jul 21, 2021

vbrancat commented Jul 22, 2021

vbrancat commented Jul 22, 2021

hfattahi commented Jul 22, 2021

piyushrpt commented Jul 22, 2021 • edited Loading

gmgunter commented Jul 22, 2021 • edited Loading

Narayana-Rao commented May 10, 2022

rtburns-jpl commented May 10, 2022

Narayana-Rao commented May 10, 2022 • edited Loading

rtburns-jpl commented May 11, 2022

Narayana-Rao commented May 11, 2022

gmgunter commented May 11, 2022

Narayana-Rao commented May 11, 2022 • edited Loading

Narayana-Rao commented May 13, 2022 • edited Loading

Narayana-Rao commented Aug 15, 2022 • edited Loading

rtburns-jpl commented Jul 21, 2021 •

edited

Loading

piyushrpt commented Jul 21, 2021 •

edited

Loading

hfattahi commented Jul 21, 2021 •

edited

Loading

piyushrpt commented Jul 22, 2021 •

edited

Loading

gmgunter commented Jul 22, 2021 •

edited

Loading

Narayana-Rao commented May 10, 2022 •

edited

Loading

Narayana-Rao commented May 11, 2022 •

edited

Loading

Narayana-Rao commented May 13, 2022 •

edited

Loading

Narayana-Rao commented Aug 15, 2022 •

edited

Loading