Awesome Visual Localization

A curated list of visual (re)localization resources, inspired by awesome-computer-vision. The list focuses on the research of visual localization, i.e. estimates 6 DoF camera pose of a query RGB/RGB-D frame in known scenes.

This document is a work in progress. Please suggest papers/resources through pull requests. If you believe this list is missing something or has factually inaccurate info, you can also new an issue. All contributions are appreciated.

Table of Contents

• Direct (One-Step) Approaches
  • Image Retrieval
  • Pose Regression
• Two-Step Approaches
  • Feature Matching Based (Structure-from-Motion)
  • Scene Coordinate Regression
• Datasets and Benchmarks

Direct Approaches

Approaches of this type estimate the camera pose directly from the query frame.

Image Retrieval

• [2018 CVPR] PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition [paper] [code]
• [2017 CVPR] Are Large-Scale 3D Models Really Necessary for Accurate Visual Localization? [paper]
• [2017 CVPR] Large-Scale Image Retrieval with Attentive Deep Local Features [paper]
• [2016 CVPR] NetVLAD: CNN architecture for weakly supervised place recognition [paper] [code]
• [2016 CVPR] Large-Scale Location Recognition and the Geometric Burstiness Problem [paper] [code]
• [2015 CVPR] 24/7 place recognition by view synthesis [paper] [code]
• [2014 TVCG] Real-Time RGB-D Camera Relocalization via Randomized Ferns for Keyframe Encoding [paper]
• [2013 CVPR] All about VLAD [paper]
• [2012 BMVC] Image Retrieval for Image-Based Localization Revisited [paper]
• [2012 BMVC] 6D Relocalisation for RGBD Cameras Using Synthetic View Regression
• [2011 IROS] Real-Time Loop Detection with Bags of Binary Words [paper]

Pose Regression

• [2020 ECCV] 6D Camera Relocalization in Ambiguous Scenes via Continuous Multimodal Inference [paper] [code]
• [2020 AAAI] AtLoc: Attention Guided Camera Localization [paper] [code]
• [2020 ICRA] To Learn or Not to Learn: Visual Localization from Essential Matrices [paper]
• [2019 ECCV] CamNet: Coarse-to-Fine Retrieval for Camera Re-Localization [paper]
• [2019 CVPR] Understanding the Limitations of CNN-based Absolute Camera Pose Regression [paper]
• [2018 ECCV] RelocNet: Continuous Metric Learning Relocalisation using Neural Nets [paper]
• [2018 BMVC] Improved Visual Relocalization by Discovering Anchor Points [paper] [code]
• [2018 CVPR] Geometry-Aware Learning of Maps for Camera Localization [paper] [code]
• [2018 RA-L] VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry [paper]
• [2017 ICCV Workshop] Camera Relocalization by Computing Pairwise Relative Poses Using Convolutional Neural Network [paper] [code]
• [2017 IROS] Deep regression for monocular camera-based 6-dof global localization in outdoor environments [paper]
• [2017 CVPR] Geometric loss functions for camera pose regression with deep learning [paper]
• [2017 CVPR] Image-based localization using LSTMs for structured feature correlation [paper]
• [2016 ICRA] Modelling Uncertainty in Deep Learning for Camera Relocalization [paper]
• [2015 ICCV] PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization [paper]

Two-Step Approaches

Approaches of this type first find correspondences between the query frame and the scene, then estimate the camera pose through optimization with RANSAC.

Feature Matching Based

• [2021 ICCV Workshop] MegLoc: A Robust and Accurate Visual Localization Pipeline [paper]
• [2021 ICCV Workshop] Pose Refinement with Joint Optimization of Visual Points and Lines [paper]
• [2021 ICRA & ICCV Workshop] Retrieval and Localization with Observation Constraints [paper]
• [2021 CVPR] Back to the Feature: Learning Robust Camera Localization from Pixels to Pose [paper] [code]
• [2020 ECCV Workshop] Hierarchical Localization with hloc and SuperGlue [slides] [code]
• [2020 IROS] KR-Net: A Dependable Visual Kidnap Recovery Network for Indoor Spaces [paper]
• [2020 arXiv] Robust Image Retrieval-based Visual Localization using Kapture [paper] [code]
• [2019 CVPR] From Coarse to Fine: Robust Hierarchical Localization at Large Scale [paper] [code]
• [2019 CVPR] D2-Net: A Trainable CNN for Joint Description and Detection of Local Features [paper] [code]
• [2018 CoRL] Leveraging Deep Visual Descriptors for Hierarchical Efficient Localization [paper] [code]
• [2018 CVPR] InLoc: Indoor Visual Localization with Dense Matching and View Synthesis [paper] [code]
• [2017 ICCV] Efficient Global 2D-3D Matching for Camera Localization in a Large-Scale 3D Map [paper]
• [2017 ICRA] [RA-L] Self-supervised Visual Descriptor Learning for Dense Correspondence [paper]
• [2016 TPAMI] Efficient & Effective Prioritized Matching for Large-Scale Image-Based Localization [paper]
• [2015 ICCV] Camera Pose Voting for Large-Scale Image-Based Localization [paper]
• [2012 ECCV] Improving Image-Based Localization by Active Correspondence Search [paper] [code]
• [2012 CVPR] Real-time Image-based 6-DOF Localization in Large-Scale Environments [paper]
• [2011 ICCV] Fast Image-Based Localization using Direct 2D-to-3D Matching [paper]

Scene Coordinate Regression

• [2022 CVPR] CrossLoc: Scalable Aerial Localization Assisted by Multimodal Synthetic Data [paper] [code]
• [2021 ICCV] Continual Learning for Image-Based Camera Localization [paper] [code]
• [2021 TPAMI] Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC [paper] [code]
• [2021 CVPR] Robust Neural Routing Through Space Partitions for Camera Relocalization in Dynamic Indoor Environments [paper] [code]
• [2021 CVPR] Learning Camera Localization via Dense Scene Matching [paper] [code]
• [2021 CVPR] VS-Net: Voting with Segmentation for Visual Localization [paper] [code]
• [2020 CVPR] KFNet: Learning Temporal Camera Relocalization using Kalman Filtering [paper] [code]
• [2020 CVPR] Hierarchical Scene Coordinate Classification and Regression for Visual Localization [paper] [code]
• [2019 TPAMI] Real-Time RGB-D Camera Pose Estimation in Novel Scenes using a Relocalisation Cascade [paper] [code]
• [2019 ICCV] SANet: Scene Agnostic Network for Camera Localization [paper] [code]
• [2019 ICCV] Expert Sample Consensus Applied to Camera Re-Localization [paper] [code]
• [2018 IROS] Exploiting Points and Lines in Regression Forests for RGB-D Camera Relocalization [paper]
• [2018 RSS] Full-Frame Scene Coordinate Regression for Image-Based Localization [paper]
• [2018 CVPR] Learning Less is More – 6D Camera Localization via 3D Surface Regression [paper] [code]
• [2017 IROS] Backtracking Regression Forests for Accurate Camera Relocalization [paper]
• [2017 CVPR] DSAC - Differentiable RANSAC for Camera Localization [paper] [code]
• [2017 CVPR] On-the-Fly Adaptation of Regression Forests for Online Camera Relocalisation [paper] [code]
• [2017 ICRA] Random Forests versus Neural Networks − What’s Best for Camera Localization? [paper]
• [2016 CVPR] Uncertainty-Driven 6D Pose Estimation of Objects and Scenes from a Single RGB Image [paper]
• [2015 CVPR] Exploiting Uncertainty in Regression Forests for Accurate Camera Relocalization [paper]
• [2014 CVPR] Multi-Output Learning for Camera Relocalization [paper]
• [2013 CVPR] Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images [paper]

Datasets and Benchmarks

Indoor

• [2021 ICCV] SfM pGT for 7-Scenes and 12-Scenes [paper]
• [2021 ICCV Workshop] ETH-Microsoft
• [2021 CVPR] NAVERLABS [paper]
• [2020 ECCV] RIO10 [paper]
• [2018 CVPR] InLoc [paper]
• [2016 3DV] 4-Scenes(12-Scenes) [paper]
• [2013 CVPR] 7-Scenes [paper]

Outdoor

• [2022 CVPR] CrossLoc [paper]
• [2021 ICCV] CrowdDriven [paper]
• [2021 ICCV Workshop] ETH-Microsoft
• [2020 ECCV Workshop] Symphony Seasons
• [2019 CVPR Workshop] SILDa
• [2018 CVPR] Aachen Day-Night, RobotCar Seasons, CMU Seasons [paper]
• [2015 ICCV] Cambridge [paper]

Challenges

• [2021 ICCV] Long-Term Visual Localization under Changing Conditions
• [2021 ICCV] Map-Based Localization for Autonomous Driving
• [2020 ECCV] Long-Term Visual Localization under Changing Conditions
• [2020 ECCV] Map-Based Localization for Autonomous Driving
• [2019 CVPR] Long-Term Visual Localization under Changing Conditions

Tools

• kapture
• TOPO-DataGen