3D scene reconstruction

[Geada8]

Checklist

• I have searched for similar issues.
• For Python issues, I have tested with the latest development wheel.
• I have checked the release documentation and the latest documentation (for main branch).

My Question

I'm using a intel realsense D455 to acquire depth images, turning them into point clouds and then using ICP to do scene reconstruction. I'm using python 3.11.7, open 3D 0.18.0. When performing ICP, not only the code is not working, it is crashing with no apparent error.

My current code is:

import numpy as np
import open3d as o3d

if __name__ == "__main__":

  print("1. Load two point clouds and show initial pose")
  source = o3d.io.read_point_cloud(".../testclouda.ply")
  target = o3d.io.read_point_cloud(".../testcloudb.ply")

  # Scale down points by dividing by 100 (assuming depth units in cm)
  source.points = o3d.utility.Vector3dVector(np.asarray(source.points) / 100.0)
  target.points = o3d.utility.Vector3dVector(np.asarray(target.points) / 100.0)

  # Estimate normals
  source.estimate_normals(search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=0.05, max_nn=30))
  target.estimate_normals(search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=0.05, max_nn=30))

  # Draw initial alignment
  current_transformation = np.identity(4)

  # Step 1: Point-to-point ICP Registration
  print("2. Performing point-to-point ICP registration with a distance threshold of 0.02")
  result_icp = o3d.pipelines.registration.registration_icp(source, target, 0.02 , current_transformation,o3d.pipelines.registration.TransformationEstimationPointToPoint())
  current_transformation = result_icp.transformation

  # Step 2: Point-to-plane ICP Registration
  print("3. Performing point-to-plane ICP registration with a distance threshold of 0.02")
  result_icp = o3d.pipelines.registration.registration_icp(
      source, target, 0.02, current_transformation,
      o3d.pipelines.registration.TransformationEstimationPointToPlane()
  )
  current_transformation = result_icp.transformation

  # Step 3: Colored Point Cloud Registration
  voxel_radius = [0.04, 0.02, 0.01]
  max_iter = [50, 30, 14]
  print("4. Performing multi-scale colored point cloud registration")

  for scale in range(3):
      iter_count = max_iter[scale]
      radius = voxel_radius[scale]
      print(f"Scale {scale+1} - Voxel size: {radius}, Iterations: {iter_count}")

      # Downsample with voxel size
      source_down = source.voxel_down_sample(radius)
      target_down = target.voxel_down_sample(radius)

      # Estimate normals for downsampled point clouds
      source_down.estimate_normals(search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=radius * 2, max_nn=30))
      target_down.estimate_normals(search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=radius * 2, max_nn=30))

      # Apply colored point cloud registration
      result_icp = o3d.pipelines.registration.registration_colored_icp(
          source_down, target_down, radius, current_transformation,
          o3d.pipelines.registration.TransformationEstimationForColoredICP(),
          o3d.pipelines.registration.ICPConvergenceCriteria(
              relative_fitness=1e-6, relative_rmse=1e-6, max_iteration=iter_count
          )
      )
      current_transformation = result_icp.transformation
      print(result_icp)

  # Display the final registration result
  # Create a deep copy of the source to apply the final transformation for visualization
  source_temp = source.voxel_down_sample(0.02)  # Downsample for better visualization if necessary
  source_temp.transform(current_transformation)
  o3d.visualization.draw_geometries([source_temp, target], window_name="Final Point Cloud Alignment")

However, in step 2, the code stops with the function:

o3d.pipelines.registration.registration_icp

Running the debugger shows:

Server[1] disconnected unexpectedly
Server[pid=2212] disconnected unexpectedly

The same problem also happens with other functions like: draw_registration_result_original_color.

Not sure what is the problem, any help or sugestions to better do scene reconstruction would be appreciated.

[sctrueew]

@Geada8 Hi, I need to create a map like this image. I estimated the depth and converted it to a point cloud. What do you think about this?