📖 DIRS: Single Image Reflection Separation via Deep Feature Interaction

[Paper] [Project]🤗[Demo]
Qiming Hu, Mingjia Li, Yuntong Li, Xiaojie Guo
College of Intelligence and Computing, Tianjin University
Journal version of our previous SIRS works: [YTMT], [DSRNet], and [DSIT].

1. Network Architecture

2. Interactive Block Designs

<iframe src="https://mingcv.github.io/DIRS-Assets/" width="100%" height="650" frameborder="0"></iframe>

3. Environment Preparation (Python 3.9)

pip install -r requirements.txt

4. Data Preparation

Training dataset

• 7,643 images from the Pascal VOC dataset, center-cropped as 224 x 224 slices to synthesize training pairs;
• 90 real-world training pairs provided by Zhang et al.;
• 200 real-world training pairs provided by IBCLN (In our training setting 2, † labeled in our paper).

Testing dataset

• 45 real-world testing images from CEILNet dataset;
• 20 real testing pairs provided by Zhang et al.;
• 20 real testing pairs provided by IBCLN;
• 454 real testing pairs from SIR^2 dataset, containing three subsets (i.e., Objects (200), Postcard (199), Wild (55)).

Download all in one by Google Drive or 百度云.

4. Usage

Training

Setting I (w/o Nature): python train_sirs_setting1.py --name dsit_large_train_setting1 --arch dsit_large --model dsit_model_sirs_lrm --dataset sirs_dataset --loss losses --lambda_vgg 0.01 --lambda_rec 0.2 --size_rounded --seed 2024 --backbone_weight_path "./weights/swin_large_o365_finetune.pth" --base_dir "YOUR_DATA_DIR]" --batchSize 1

Setting II (w/ Nature): python train_sirs_setting2.py --name dsit_large_train_setting2 --arch dsit_large --model dsit_model_sirs_lrm --dataset sirs_dataset --loss losses --lambda_vgg 0.01 --lambda_rec 0.2 --size_rounded --seed 2024 --backbone_weight_path "./weights/swin_large_o365_finetune.pth" --base_dir "[YOUR_DATA_DIR]" --batchSize 1

Evaluation

Setting I (w/o Nature): python eval_sirs.py --name dsit_large_setting1_eval_epoch20 --arch dsit_large --model dsit_model_sirs_lrm --dataset sirs_dataset --size_rounded --weight_path "./weights/dist-large-setting1-epoch20.pth" --backbone_weight_path "./weights/swin_large_o365_finetune.pth" --base_dir "[YOUR_DATA_DIR]"

Setting II (w/ Nature): eval_sirs.py --name dsit_large_setting2_eval_epoch66 --arch dsit_large --model dsit_model_sirs_lrm --dataset sirs_dataset --size_rounded --test_nature --weight_path "./weights/dist-large-setting2-epoch66.pth" --backbone_weight_path "./weights/swin_large_o365_finetune.pth" --base_dir "[YOUR_DATA_DIR]"

Testing

python test_sirs.py --name dsit_large_setting2_test_epoch66 --arch dsit_large --model dsit_model_sirs_lrm --dataset sirs_dataset --size_rounded --test_nature --weight_path "./weights/dist-large-setting2-epoch66.pth" --backbone_weight_path "./weights/swin_large_o365_finetune.pth" --test_dir "[YOUR_TESTING_DATA_DIR]"

5. Trained weights

Download the trained weights by Google Drive or 百度云 and drop them into the "weights" dir.

6. Visual Comparison on Real20 and SIR^2 Datasets

7. Extra Visual Comparison on Our Additionally Captured Real-World Cases