Questions regarding camera lidar preprocessing

[khuechuong]

Hi! So I have a few questions about the camera specifically before inputting in. Specifically in program details, example 2:

Does is the image topic need to be rect or image raw is fine?

This one doesn't use camera_info topic?

The reason I'm asking is because I use kalibr for intrinsic calibration. I believe their equivalent of plumb_bob is pinhole camera model (pinhole)1 (intrinsics vector: [fu fv pu pv]) and radial-tangential (radtan)* (distortion_coeffs: [k1 k2 r1 r2]). However it doesn't give me the rect matrix or the extrinsic so I can't compute the re-projection matrix or the rect matrix. Without the re-projection matrix, I am getting something like this https://user-images.githubusercontent.com/2222562/166346133-c1b3e678-e660-490c-addf-2740661ab6ca.png which I don't think is right to input.

What do you recommend? What did u guys use for intrinsic calibration?

[koide3]

The image doesn't need to be rectified, this package does it internally. The camera info data is saved in a json file, and programs after preprocessing use the saved camera info data.

You can use kalibr or any other camera calibration programs to calibrate your camera, then, you can give the calibrated parameters as command line parameters for the preprocessing program.

[khuechuong]

Can I know which file of code that does it internally? I'm stuck with images like this:

when I rectify it. I want to know how you get the image the project right. Thanks!

[koide3]

Here is the camera models that computes projection and lens distortion: https://github.com/koide3/direct_visual_lidar_calibration/tree/main/include/camera

[khuechuong]

which cpp file that did the distortion and rectify image?

[koide3]

It doesn't generate rectified images explicitly but does project and undistort 3D object points on the image space via the following function:

direct_visual_lidar_calibration/include/camera/pinhole.hpp

Line 41 in d29fec3

auto operator()(const T* const intrinsic, const T* const distortion, const Eigen::Matrix<T2, 3, 1>& point_3d) const -> Eigen::Matrix<decltype(T() * T2()), 2, 1> {

[khuechuong]

so that function take into account the black stuff when it is undistort and trim it off?