Official PyTorch implementation of WACV 2025 paper: Event-guided Low-light Video Semantic Segmentation
Recent video semantic segmentation (VSS) methods have demonstrated promising results in well-lit environments. However, their performance significantly drops in low-light scenarios due to limited visibility and reduced contextual details. In addition, unfavorable low-light conditions make it harder to incorporate temporal consistency across video frames and thus, lead to video flickering effects. Compared with conventional cameras, event cameras can capture motion dynamics, filter out temporal-redundant information, and are robust to lighting conditions. To this end, we propose EVSNet, a lightweight framework that leverages event modality to guide the learning of a unified illumination-invariant representation. Specifically, we leverage a Motion Extraction Module to extract short-term and long-term temporal motions from event modality and a Motion Fusion Module to integrate image features and motion features adaptively. Furthermore, we use a Temporal Decoder to exploit video contexts and generate segmentation predictions. Such designs in EVSNet result in a lightweight architecture while achieving SOTA performance. Experimental results on 3 large-scale datasets demonstrate our proposed EVSNet outperforms SOTA methods with up to x11 higher parameter efficiency.
Authors: Zhen Yao, Mooi Choo Chuah.
Please follow the guidelines in MMSegmentation v0.13.0.
Other requirements:
timm==0.3.0, CUDA11.0, pytorch==1.7.1, torchvision==0.8.2, mmcv==1.3.0, opencv-python==4.5.2
Download this repository and install by:
cd EVSNet && pip install -e . --user
Please follow VSPW to download VSPW 480P dataset. After correctly downloading, next generate synthetic lowlight images based on process_darken.py and then generate synthetic event image using v2e. Finally, the file system is as follows:
vspw-480
├── video1
├── origin
├── .jpg
├── lowlight
├── .jpg
├── event
├── .jpg
└── mask
└── .png
The dataset should be put in /repo_path/data/vspw/. Or you can use Symlink:
cd EVSNet
mkdir -p data/vspw/
ln -s /dataset_path/VSPW_480p data/vspw/
Run the following commands:
# Single-gpu testing
python tools/test.py local_configs/evsnet/evsnet.afbase.480x480.llvspw2.160k.py /path/to/checkpoint_file
python tools/test.py local_configs/evsnet/evsnet.mitb0.480x480.llvspw2.160k.py /path/to/checkpoint_file
# Single-gpu training
python tools/train.py local_configs/evsnet/evsnet.afbase.480x480.llvspw2.160k.py
python tools/train.py local_configs/evsnet/evsnet.mitb0.480x480.llvspw2.160k.py
# Multi-gpu training
./tools/dist_train.sh local_configs/evsnet/evsnet.afbase.480x480.llvspw2.160k.py <GPU_NUM>
For more details, please access MMSeg. Some other references include CFFM and BRENet.
This project is only for academic use.
The code is heavily based on the following repositories:
- https://github.com/open-mmlab/mmsegmentation
- https://github.com/NVlabs/SegFormer
- https://github.com/GuoleiSun/VSS-CFFM
Thanks for their amazing works.
@inproceedings{yao2025event,
title={Event-guided low-light video semantic segmentation},
author={Yao, Zhen and Chuah, Mooi Choo},
booktitle={2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
pages={3330--3341},
year={2025},
organization={IEEE}
}- Zhen Yao, zhenyaocv@gmail.com