How to obtain correct line segments using hawp and make the neat-final-parsing.py run normally

[white6zz]

Thanks for your work. I noticed that there were configurations for scannet in the history of vault. Now I try to obtain the 3D wireframe of indoor scenes. The steps are shown as following:

1. recovery the scannet.conf from the git history

2. use hawp to get the 2D line segments

   python -m hawp.ssl.predict --ckpt hawpv3-imagenet-03a84.pth --threshold 0.05 --img_path scan/scene0524_01/images --saveto scan/scene0524_01/hawp --ext json

   I found that the parameters of hawp @ 00322cb is different from hawp @ main , especially the --ext. There is *.json in hawp content from processed data . Howerer, hawp @ 00322cb can't output the *.json , which represent the 2D line segments of each image.

3. train

   python training/exp_runner.py --conf confs/iccv-submission/indoor-trials/scannet.conf --nepoch 2000 --tbvis --scan_id 0524_01

4. parse

   python neat-final-parsing.py --conf ../exps/scannet/xxxxxx/runconf.conf --checkpoint 1000

Then the error occurs.

Traceback (most recent call last):
  File "/data/username/neat/code/neat-final-parsing.py", line 460, in <module>
    wireframe_recon(conf=opt.conf,
  File "/data/username/neat/code/neat-final-parsing.py", line 405, in wireframe_recon
    initial_recon_results = initial_recon(
  File "/data/username/neat/code/neat-final-parsing.py", line 296, in initial_recon
    junctions3d_initial = torch.stack([global_junctions[k] for k,v in gjc_dict.items() if v.shape[0]>1]) #TODO check if the voting threshold should be added as a parameter
RuntimeError: stack expects a non-empty TensorList

I found following issues:

• In code/scripts/hawp-dtu.sh and preprocess/readme.md from , use the following code to obtain line segments:

  python -m sslib.predict \
      --metarch HAWP \
      --modelcfg /home/xn/repo/hawp2vision/exp-ssl/imagenet-train-small/model.yaml \
      --ckpt /home/xn/repo/hawp2vision/exp-ssl/imagenet-train-small/model-00030.pth \
      --img ../data/DTU/$scan/image/*.png  --threshold 0.05 --save-txt ../data/DTU/$scan/hawp

  However, in hawp @ main , it is:

  python -m hawp.ssl.predict --ckpt checkpoints/hawpv3-imagenet-03a84.pth  \
      --threshold 0.05  \
      --img ~/datasets/DTU/scan24/image/*.png \ 
      --saveto docs/figures/dtu-24 --ext png \

  I was confused how to get the line segments using hawp.

• In processed data , the "vertices-score" in *.json files generated by HAWP looks like this:

  "vertices-score": [0.8197214603424072, 0.7438260316848755, 0.732781171798706, ...]

  However, using python -m hawp.ssl.predict --ckpt hawpv3-imagenet-03a84.pth --threshold 0.05 --img_path scan/scene0524_01/images --saveto scan/scene0524_01/hawp --ext json , I found the result is different:

  "vertices-score": [[0, 6], [0, 13], [0, 15], [0, 24],...]

In summary, I want to know how to obtain the correct 2D line segments, making the training and the parsing processes run normally. Thanks.

[white6zz]

I know why the vertices-score of *.json in hawp is different from processed data, in hawp/ssl/models/detector.py:287-290 from hawp @ main :

juncs_pred, _ = self.get_junctions(jloc_pred_nms,joff_pred[0], topk=topK,th=self.junction_threshold_hm)
if juncs_pred.shape[0] == 0:
    return {'lines_pred': None, 'lines_score': None}
lines_adjusted, lines_init, perm, _ = self.wireframe_matcher(juncs_pred, lines_pred)

...

juncs_score = _

Obsiviously the juncs_score get a wrong value.