Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

warp操作中 都把视差取负数了为什么还assert disp.min()>=0 还有可变形卷积到底用没用呢 #11

Open
dididichufale opened this issue Nov 1, 2023 · 3 comments

Comments

@dididichufale
Copy link

low_disp = -low_disp
# low_disp = low_disp.unsqueeze(1) # [B, 1, H, W]
scale_factor = left_img.size(-1) / low_disp.size(-1)
if scale_factor == 1.0:
disp = low_disp

def disp_warp(img, disp, padding_mode='border'):
"""Warping by disparity
Args:
img: [B, 3, H, W]
disp: [B, 1, H, W], positive
padding_mode: 'zeros' or 'border'
Returns:
warped_img: [B, 3, H, W]
valid_mask: [B, 3, H, W]
"""
assert disp.min() >= 0

@dididichufale dididichufale changed the title warp操作中 都把视差去负数了为什么还assert disp.min()>=0 还有可变形卷积到底用没用呢 warp操作中 都把视差取负数了为什么还assert disp.min()>=0 还有可变形卷积到底用没用呢 Nov 1, 2023
@David-Zhao-1997
Copy link
Owner

我们在RAFT-Stereo的基础上构建了我们的模型,因为网络预测出的视差值都是负值,我们在进行warp操作时将其取相反数后得到正值再进行具体的warp操作。此时视差值应当全部为>=0的值,因此assert disp.min() >= 0。

在我们的网络中引入可变形卷积可以提高在单一场景下的视觉效果(但视差值不一定更为准确)和应对弱纹理区域的性能;但引入可变形卷积在直接将模型迁移到其他场景时会表现出较差的泛化性。

@dididichufale
Copy link
Author

兄弟,我把你的warp操作和后面的一个unet结构加到传统的3D卷积形成的沙漏结构后面没有放到raft后面 效果没有增加,反而是副作用,这种你可以解释吗

@dididichufale
Copy link
Author

我想把你的warp模块放到我的网络里面来,拼接一下,感谢指点

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants