Day-Night Retrieval
A project website for the following publication:
No Fear of the Dark: Image Retrieval under Varying Illumination Conditions
Tomas Jenicek and Ondřej Chum
In International Conference on Computer Vision (ICCV), 2019
Related: paper pdf, GitHub project
Downloads
- day-night training dataset rSfM-N/D 1:3 (ratio of night-day image pairs)
- day-night whitening dataset rSfM-N/D whiten (all night-day image pairs used)
- day-night evaluation dataset 24/7 Tokyo 1k
- fine-tuned VGG (CLAHE) with whitening
- fine-tuned composition U-Net + VGG jointly with whitening
All datasets are in the cirtorch format
External
- images for rSfM-N/D – rSfM 120k images
- images for 24/7 Tokyo – query images v3
Method
Normalization network is pre-trained, pre-pended to an embedding network and the composition is fine-tuned.
Normalization Network Pre-Training
Normalization network is pre-trained on multi-exposure pixel-aligned image pairs.
Composition Network Fine-Tuning
Normalization (U-Net) and embedding (VGG) networks are fine-tuned end-to-end for retrieval with a contrastive loss.
Datasets
Two datasets are used, one for image-to-image translation pre-training (SID) and one for image retrieval fine-tuning (rSfM).
Image-to-Image Translation (SID)
For a pair of pixel-aligned images, additional exposures were synthesized.
Image Retrieval Training (rSfM N/D)
A new dataset with night-day (night is always a query) image pairs was created.
Image Retrieval Evaluation (24/7 Tokyo)
A new evaluation protocol for day-night image retrieval is defined.
Results
mAP measured on the new day-night (Tokyo) and standard day-day (ROxf, RPar) datasets.
| method | Tokyo | ROxf (M) | RPar (M) |
|---|---|---|---|
| VGG rSfM 120k | 79.4 | 60.9 | 69.3 |
| VGG rSfM N/D | 83.5 | 60.0 | 69.8 |
| CLAHE + VGG rSfM N/D | 87.0 | 60.2 | 70.0 |
| U-Net + VGG rSfM N/D | 86.5 | 60.2 | 69.6 |
Normalization Effect
Visualization of the embedding (VGG) network input after normalization.
Resources
- training datasets: SID, rSfM 120k
- evaluation datasets: 24/7 Tokyo, ROxf + RPar
- evaluated methods: VGG GeM rSfM-120k, cv2 CLAHE
Papers
[1]
No Fear of the Dark: Image Retrieval under Varying Illumination Conditions
Jenicek, Tomas and Chum, Ondřej
In ICCV, 2019
Bibtex
@inproceedings{jenicek2019no,
title={No Fear of the Dark: Image Retrieval under Varying Illumination Conditions},
author={Jenicek, Tomas and Chum, Ond{\v{r}}ej},
booktitle = {ICCV},
year={2019}
}