RepMCBNet

Multi-Vision Services Acceleration Framework for IoT Devices [IEEE ICWS 2024], [Accepted Manuscript]

Environment

This demo of re-parameterized multi-controllable branching networks (RepMCBNet) requires the PyTorch >=1.10, but not strictly required.

Datasets

MTL datasets: the CelebA and the preprocessed Cityscapes.

Usage

Training CelebA Grouped 8-tasks baseline

# hard-sharing ResNet-18 CelebA Grouped
python train_rep_mcb_celeba.py --dataset celeba_grouped --exp rs18_celeb_grp_s0

# ETR-NLP ResNet-18 CelebA Grouped gamma=0.9
python train_tr_etr_nlp_celeba.py --dataset celeba_grouped --sigma 0.9 --etr --exp etr_nlp_rs18_g9_celeb_grp_s0

# Task-Routing ResNet-18 CelebA Grouped sigma=0.6
python train_tr_etr_nlp_celeba.py --dataset celeba_grouped --sigma 0.6 --route --exp taskroute_rs18_sigma6_celeb_grp_s0

# Task-Routing ResNet-18 CelebA Grouped sigma=0.9
python train_tr_etr_nlp_celeba.py --dataset celeba_grouped --sigma 0.9 --route --exp taskroute_rs18_sigma9_celeb_grp_s0

# Max-Roaming ResNet-18 CelebA Grouped p=0.8
python train_max_roaming_celeba.py --dataset celeba_grouped --sigma 0.8 --exp MR_rs18_sigma8_celeb_grp_s0

Training CelebA Grouped 8-tasks RepMCBNet with different gamma (sigma)

# RepMCBNet ResNet-18 CelebA Grouped gamma=0.9
python train_rep_mcb_celeba.py --dataset celeba_grouped --sigma 0.9 --mcb --exp mcb_rs18_celeb_grp_s0

# RepMCBNet ResNet-18 CelebA Grouped gamma=0.7
python train_rep_mcb_celeba.py --dataset celeba_grouped --sigma 0.7 --mcb --exp mcb_rs18_g7_celeb_grp_s0

# RepMCBNet ResNet-18 CelebA Grouped gamma=0.5
python train_rep_mcb_celeba.py --dataset celeba_grouped --sigma 0.5 --mcb --exp mcb_rs18_g5_celeb_grp_s0

# RepMCBNet ResNet-18 CelebA Grouped gamma=0.3
python train_rep_mcb_celeba.py --dataset celeba_grouped --sigma 0.3 --mcb --exp mcb_rs18_g3_celeb_grp_s0

# RepMCBNet ResNet-18 CelebA Grouped gamma=0.1
python train_rep_mcb_celeba.py --dataset celeba_grouped --sigma 0.1 --mcb --exp mcb_rs18_g1_celeb_grp_s0

# RepMCBNet ResNet-18 CelebA Grouped gamma=0.
python train_rep_mcb_celeba.py --dataset celeba_grouped --sigma 0. --mcb --exp mcb_rs18_g0_celeb_grp_s0

Training Cityscapes-tasks 7-class baseline

# hard-sharing SegNet Cityscapes
python train_rep_mcb_cityscapes.py --exp segnet_cityscapes_s0

# Max Roaming SegNet Cityscapes p=0.6
python train_max_roaming_cityscapes.py --sigma 0.6 --exp MR_segnet_sigma6_cityscapes_s0

# ETR-NLP SegNet Cityscapes gamma=0.9
python train_tr_etr_nlp_cityscapes.py --sigma 0.9 --etr --exp etr_nlp_segnet_g9_cityscapes_s0

# Task-Routing SegNet Cityscapes sigma=0.6
python train_tr_etr_nlp_cityscapes.py --sigma 0.6 --route --exp taskroute_segnet_sigma6_cityscapes_s0

Training Cityscapes-tasks 7-class RepMCBNet with different gamma (sigma)

# RepMCBNet SegNet Cityscapes gamma=0.9
python train_rep_mcb_cityscapes.py --sigma 0.9 --mcb --exp mcb_segnet_g9_cityscapes_s0

# RepMCBNet SegNet Cityscapes gamma=0.7
python train_rep_mcb_cityscapes.py --sigma 0.7 --mcb --exp mcb_segnet_g7_cityscapes_s0

# RepMCBNet SegNet Cityscapes gamma=0.5
python train_rep_mcb_cityscapes.py --sigma 0.5 --mcb --exp mcb_segnet_g5_cityscapes_s0

# RepMCBNet SegNet Cityscapes gamma=0.3
python train_rep_mcb_cityscapes.py --sigma 0.3 --mcb --exp mcb_segnet_g3_cityscapes_s0

# RepMCBNet SegNet Cityscapes gamma=0.1
python train_rep_mcb_cityscapes.py --sigma 0.1 --mcb --exp mcb_segnet_g1_cityscapes_s0

# RepMCBNet SegNet Cityscapes gamma=0.
python train_rep_mcb_cityscapes.py --sigma 0. --mcb --exp mcb_segnet_g0_cityscapes_s0

Publication

Jianhang Xie, Chuntao Ding, Shenyuan Ren, Yidong Li, “Multi-Vision Services Acceleration Framework for IoT Devices,” in Proceedings of the IEEE International Conference on Web Services, 2024, pp.888-898.

URL: DOI, Accepted Manuscript

@inproceedings{xie.icws2024multi-vision,
  author = {Xie, Jianhang and Ding, Chuntao and Ren, Shenyuan and Li, Yidong},
  booktitle = {Proceedings of the IEEE International Conference on Web Services},
  title = {Multi-Vision Services Acceleration Framework for IoT Devices},
  year = {2024},
  pages = {888-898},
  doi = {10.1109/ICWS62655.2024.00107}
}

Acknowledgement

MTL training framework: https://github.com/lucaspascal/Maximum-Roaming-Mutli-Task-Learning

Building the re-parameterization modules: https://github.com/DingXiaoH/RepVGG https://github.com/DingXiaoH/DiverseBranchBlock