ReferWhat

Source code and data for paper Referring Expression Generation via Visual Dialogue

Install requirment

• yaml
• NLTK
• h5py
• json
• TensorboardX
• Java1.8.0
• Python3.6
• PyTorch 1.1

Dataset

• Image: Download from COCO_train2014
• ReferWhat?! append only dialog: Download from ReferWhat_dataset, and save into /data folder.
• instances.json: Download from refcoco/refcoco+

Prepare

cd code/prepare

• image: python extract_feats.py --dataset refcoco --cnn resnet152 --image_root path_fo_COCO_image --data_root ../../data/
• dataset: python load_VD_data.py --path ../../data/refcoco/unc_split

Train

• Step1: modify the config file in code/config
• Step2: python main.py

Evaluation

python Gen_eval.py