How to generate “mask_rcnn_r50.val.segm.json” and "mask_rcnn_r50.train.segm.json"?

[yuzeWAN]

Dear author, hello. I have read the instructions on training networks using my own dataset. My dataset includes images and corresponding category annotation JSON files. How can I create my own dataset and how can I create “mask_rcnn_r50.val.segm.json” and "mask_rcnn_r50.train.segm.json" these two files based on my dataset? Looking forward to your reply!

[chenhang98]

Note that our method is for refinement instead of directly performing segmentation, therefore an instance segmentation network is needed to provide coarse results. Assuming you are using mmdetection, you may refer to #33 for generating these files.

[yuzeWAN]

Dear author, thank you very much for your reply!

[LEONHWH]

Note that our method is for refinement instead of directly performing segmentation, therefore an instance segmentation network is needed to provide coarse results. Assuming you are using mmdetection, you may refer to #33 for generating these files.

The following link cant be accessed : https://mmdetection.readthedocs.io/en/stable/1_exist_data_model.html#examples. Can you offer the Detailed structure format of "mask_rcnn_r50.train.segm.json"? I have try to build the dataset using coco format from the scpit https://github.com/wkentaro/labelme/tree/main/examples/instance_segmentation#convert-to-coco-format-dataset, but how to build the coarse segm datasets still makes me very confused. I am waitting for your reply,thanks.如果有帮助，我将十分感谢。

[yuzeWAN]

Note that our method is for refinement instead of directly performing segmentation, therefore an instance segmentation network is needed to provide coarse results. Assuming you are using mmdetection, you may refer to #33 for generating these files.

The following link cant be accessed : https://mmdetection.readthedocs.io/en/stable/1_exist_data_model.html#examples. Can you offer the Detailed structure format of "mask_rcnn_r50.train.segm.json"? I have try to build the dataset using coco format from the scpit https://github.com/wkentaro/labelme/tree/main/examples/instance_segmentation#convert-to-coco-format-dataset, but how to build the coarse segm datasets still makes me very confused. I am waitting for your reply,thanks.如果有帮助，我将十分感谢。

I have the same question!

[chenhang98]

mmdetection has changed this link. Now you can refer to open-mmlab/mmdetection#2693 or its document for details. You can also get an example here https://cloud.tsinghua.edu.cn/f/d86b4a4e67544dc494ae/?dl=1

[LEONHWH]

mmdetection has changed this link. Now you can refer to open-mmlab/mmdetection#2693 or its document for details. You can also get an example here https://cloud.tsinghua.edu.cn/f/d86b4a4e67544dc494ae/?dl=1

OK,I found it's not a standard Object Instance COCO format but a list like:
[
{'image_id': 397133,
'bbox': [385.7433776855469, 68.52386474609375, 117.4853515625, 285.50604248046875],
'score': 0.998802661895752},
'category_id': 1},
'segmentation': {'size': [427, 640], 'counts': 'mmR5d0a<;G6H7\KSOWL_1d3iNgKi1U4\NaKm1\4WNYKR2e4SNoJW2o4lMgJ\2X5gMaJ^2]5eM\Ja2b5bMUJe2i5aMmIf2R6`McIe2]6U2O01O001O0N2N101O1N2L5L3N3K5J6K4M4M3K5L4M2N2K4M3O02WOi0N2O10002N1O2N1N4N3L4L3N3M5J7J4L4L:F=C6J4L5K3M3M3M1O2N1O001O00010OO2O001O1N3N2gHRKP6P5mITKP6o4kIUK_2d0ZO\4QNTKd1S2DU3XNRKo1R3\NR3b1^4N2N101N100O1O3M2N1N3N1O2N2M4L3M3M3L6J7I8Hn^k1'},
......
]

My JSON file follows the the standard COCO format:
{
"info": {
"description": "COCO 2017 Dataset",
"url": "http://cocodataset.org",
"version": "1.0",
...
},
"licenses": {
{
"url": "http://creativecommons.org/licenses/by-nc-sa/2.0/",
"id": 1,
"name": "Attribution-NonCommercial-ShareAlike License"
},
...
},
"images": [
{
"license": 4,
"file_name": "000000397133.jpg",
"coco_url": "http://images.cocodataset.org/val2017/000000397133.jpg",
"height": 427,
"width": 640,
"date_captured": "2013-11-14 17:02:52",
"flickr_url": "http://farm7.staticflickr.com/6116/6255196340_da26cf2c9e_z.jpg",
"id": 397133
},
...
],
"annotations": [
{
"segmentation": RLE or [polygon],
"area": float,
"iscrowd": 0 or 1,
"image_id": int,
"bbox": [x, y, width, height],
"category_id": int,
"id": int
},
...
],
"categories": [
{
"supercategory": str,
"id": int,
"name": str
},
...
]
}

Can you tell which keys in your example(https://cloud.tsinghua.edu.cn/f/d86b4a4e67544dc494ae/?dl=1) are necessary? I will convert my JOSN file to your format. I guess 'bbox' and 'score' are not necessary?

[chenhang98]

I think you are misunderstanding the usage of BPR. BPR is a post-processing method instead of an instance segmentation model, which means it is used to refine the segmentation results that already exist (yielded by Mask R-CNN for example). After applying BPR, the coarse segmentation results will be finer. Thus apart from the ground truth, BPR also requires the coarse segmentation results for training and inference. "mask_rcnn_r50.train.segm.json" and "mask_rcnn_r50.val.segm.json" are both the coarse segmentation results, while the JOSN file you provided is the ground truth.

[LEONHWH]

I think you are misunderstanding the usage of BPR. BPR is a post-processing method instead of an instance segmentation model, which means it is used to refine the segmentation results that already exist (yielded by Mask R-CNN for example). After applying BPR, the coarse segmentation results will be finer. Thus apart from the ground truth, BPR also requires the coarse segmentation results for training and inference. "mask_rcnn_r50.train.segm.json" and "mask_rcnn_r50.val.segm.json" are both the coarse segmentation results, while the JOSN file you provided is the ground truth.

I see that . I generate the coarse segmentation results from deeplabv3+, and I can produce the polygen or mask result into the key of RLE "segmentation" in some way. These results dont contain the 'bbox' and 'score' information ,and I want to konw if the 'bbox' and 'score' are useful to the project.

[chenhang98]

I see, the 'bbox' and 'score' are not necessary.

[yuzeWAN]

mmdetection has changed this link. Now you can refer to open-mmlab/mmdetection#2693 or its document for details. You can also get an example here https://cloud.tsinghua.edu.cn/f/d86b4a4e67544dc494ae/?dl=1

OK,I found it's not a standard Object Instance COCO format but a list like: [ {'image_id': 397133, 'bbox': [385.7433776855469, 68.52386474609375, 117.4853515625, 285.50604248046875], 'score': 0.998802661895752}, 'category_id': 1}, 'segmentation': {'size': [427, 640], 'counts': 'mmR5d0a<;G6H7\KSOWL_1d3iNgKi1U4\NaKm1\4WNYKR2e4SNoJW2o4lMgJ\2X5gMaJ^2]5eM\Ja2b5bMUJe2i5aMmIf2R6`McIe2]6U2O01O001O0N2N101O1N2L5L3N3K5J6K4M4M3K5L4M2N2K4M3O02WOi0N2O10002N1O2N1N4N3L4L3N3M5J7J4L4L:F=C6J4L5K3M3M3M1O2N1O001O00010OO2O001O1N3N2gHRKP6P5mITKP6o4kIUK_2d0ZO\4QNTKd1S2DU3XNRKo1R3\NR3b1^4N2N101N100O1O3M2N1N3N1O2N2M4L3M3M3L6J7I8Hn^k1'}, ...... ]

My JSON file follows the the standard COCO format: { "info": { "description": "COCO 2017 Dataset", "url": "http://cocodataset.org", "version": "1.0", ... }, "licenses": { { "url": "http://creativecommons.org/licenses/by-nc-sa/2.0/", "id": 1, "name": "Attribution-NonCommercial-ShareAlike License" }, ... }, "images": [ { "license": 4, "file_name": "000000397133.jpg", "coco_url": "http://images.cocodataset.org/val2017/000000397133.jpg", "height": 427, "width": 640, "date_captured": "2013-11-14 17:02:52", "flickr_url": "http://farm7.staticflickr.com/6116/6255196340_da26cf2c9e_z.jpg", "id": 397133 }, ... ], "annotations": [ { "segmentation": RLE or [polygon], "area": float, "iscrowd": 0 or 1, "image_id": int, "bbox": [x, y, width, height], "category_id": int, "id": int }, ... ], "categories": [ { "supercategory": str, "id": int, "name": str }, ... ] }

Can you tell which keys in your example(https://cloud.tsinghua.edu.cn/f/d86b4a4e67544dc494ae/?dl=1) are necessary? I will convert my JOSN file to your format. I guess 'bbox' and 'score' are not necessary?

Dear partner: Hello! I noticed that you mentioned that you have already generated a JSON file for the segmentation results. How did you generate it? The JSON file I generated only includes information such as evaluation metrics (miou), segmentation image names, and does not include information such as segmentation categories and point coordinates. I hope you can answer me in your free time! Looking forward to your reply!

[LEONHWH]

mmdetection has changed this link. Now you can refer to open-mmlab/mmdetection#2693 or its document for details. You can also get an example here https://cloud.tsinghua.edu.cn/f/d86b4a4e67544dc494ae/?dl=1

OK,I found it's not a standard Object Instance COCO format but a list like: [ {'image_id': 397133, 'bbox': [385.7433776855469, 68.52386474609375, 117.4853515625, 285.50604248046875], 'score': 0.998802661895752}, 'category_id': 1}, 'segmentation': {'size': [427, 640], 'counts': 'mmR5d0a<;G6H7\KSOWL_1d3iNgKi1U4\NaKm1\4WNYKR2e4SNoJW2o4lMgJ\2X5gMaJ^2]5eM\Ja2b5bMUJe2i5aMmIf2R6`McIe2]6U2O01O001O0N2N101O1N2L5L3N3K5J6K4M4M3K5L4M2N2K4M3O02WOi0N2O10002N1O2N1N4N3L4L3N3M5J7J4L4L:F=C6J4L5K3M3M3M1O2N1O001O00010OO2O001O1N3N2gHRKP6P5mITKP6o4kIUK_2d0ZO\4QNTKd1S2DU3XNRKo1R3\NR3b1^4N2N101N100O1O3M2N1N3N1O2N2M4L3M3M3L6J7I8Hn^k1'}, ...... ]
My JSON file follows the the standard COCO format: { "info": { "description": "COCO 2017 Dataset", "url": "http://cocodataset.org", "version": "1.0", ... }, "licenses": { { "url": "http://creativecommons.org/licenses/by-nc-sa/2.0/", "id": 1, "name": "Attribution-NonCommercial-ShareAlike License" }, ... }, "images": [ { "license": 4, "file_name": "000000397133.jpg", "coco_url": "http://images.cocodataset.org/val2017/000000397133.jpg", "height": 427, "width": 640, "date_captured": "2013-11-14 17:02:52", "flickr_url": "http://farm7.staticflickr.com/6116/6255196340_da26cf2c9e_z.jpg", "id": 397133 }, ... ], "annotations": [ { "segmentation": RLE or [polygon], "area": float, "iscrowd": 0 or 1, "image_id": int, "bbox": [x, y, width, height], "category_id": int, "id": int }, ... ], "categories": [ { "supercategory": str, "id": int, "name": str }, ... ] }
Can you tell which keys in your example(https://cloud.tsinghua.edu.cn/f/d86b4a4e67544dc494ae/?dl=1) are necessary? I will convert my JOSN file to your format. I guess 'bbox' and 'score' are not necessary?

Dear partner: Hello! I noticed that you mentioned that you have already generated a JSON file for the segmentation results. How did you generate it? The JSON file I generated only includes information such as evaluation metrics (miou), segmentation image names, and does not include information such as segmentation categories and point coordinates. I hope you can answer me in your free time! Looking forward to your reply!

As the author say, you can refer to open-mmlab/mmdetection#2693 or its document and produce the coarse segmentation results and their JSON file for input. If you want to produce GT datatsets, you can use pip-tool (pycocotools).

[yuzeWAN]

mmdetection has changed this link. Now you can refer to open-mmlab/mmdetection#2693 or its document for details. You can also get an example here https://cloud.tsinghua.edu.cn/f/d86b4a4e67544dc494ae/?dl=1

OK,I found it's not a standard Object Instance COCO format but a list like: [ {'image_id': 397133, 'bbox': [385.7433776855469, 68.52386474609375, 117.4853515625, 285.50604248046875], 'score': 0.998802661895752}, 'category_id': 1}, 'segmentation': {'size': [427, 640], 'counts': 'mmR5d0a<;G6H7\KSOWL_1d3iNgKi1U4\NaKm1\4WNYKR2e4SNoJW2o4lMgJ\2X5gMaJ^2]5eM\Ja2b5bMUJe2i5aMmIf2R6`McIe2]6U2O01O001O0N2N101O1N2L5L3N3K5J6K4M4M3K5L4M2N2K4M3O02WOi0N2O10002N1O2N1N4N3L4L3N3M5J7J4L4L:F=C6J4L5K3M3M3M1O2N1O001O00010OO2O001O1N3N2gHRKP6P5mITKP6o4kIUK_2d0ZO\4QNTKd1S2DU3XNRKo1R3\NR3b1^4N2N101N100O1O3M2N1N3N1O2N2M4L3M3M3L6J7I8Hn^k1'}, ...... ]
My JSON file follows the the standard COCO format: { "info": { "description": "COCO 2017 Dataset", "url": "http://cocodataset.org", "version": "1.0", ... }, "licenses": { { "url": "http://creativecommons.org/licenses/by-nc-sa/2.0/", "id": 1, "name": "Attribution-NonCommercial-ShareAlike License" }, ... }, "images": [ { "license": 4, "file_name": "000000397133.jpg", "coco_url": "http://images.cocodataset.org/val2017/000000397133.jpg", "height": 427, "width": 640, "date_captured": "2013-11-14 17:02:52", "flickr_url": "http://farm7.staticflickr.com/6116/6255196340_da26cf2c9e_z.jpg", "id": 397133 }, ... ], "annotations": [ { "segmentation": RLE or [polygon], "area": float, "iscrowd": 0 or 1, "image_id": int, "bbox": [x, y, width, height], "category_id": int, "id": int }, ... ], "categories": [ { "supercategory": str, "id": int, "name": str }, ... ] }
Can you tell which keys in your example(https://cloud.tsinghua.edu.cn/f/d86b4a4e67544dc494ae/?dl=1) are necessary? I will convert my JOSN file to your format. I guess 'bbox' and 'score' are not necessary?

Dear partner: Hello! I noticed that you mentioned that you have already generated a JSON file for the segmentation results. How did you generate it? The JSON file I generated only includes information such as evaluation metrics (miou), segmentation image names, and does not include information such as segmentation categories and point coordinates. I hope you can answer me in your free time! Looking forward to your reply!

As the author say, you can refer to open-mmlab/mmdetection#2693 or its document and produce the coarse segmentation results and their JSON file for input. If you want to produce GT datatsets, you can use pip-tool (pycocotools).

Yes, I referred to the reference link provided by the author(See image for details) and ran the following command in Python/ Tools/test.py (config file path) (The path where the trained model is located)-- format only -- options "jsonfile_prefix=./results", but an error was reported as follows: "TypeError:SpineDataset: init() got an unexpected keyword argument 'jsonfile_prefix'"

[yuzeWAN]

mmdetection has changed this link. Now you can refer to open-mmlab/mmdetection#2693 or its document for details. You can also get an example here https://cloud.tsinghua.edu.cn/f/d86b4a4e67544dc494ae/?dl=1

OK,I found it's not a standard Object Instance COCO format but a list like: [ {'image_id': 397133, 'bbox': [385.7433776855469, 68.52386474609375, 117.4853515625, 285.50604248046875], 'score': 0.998802661895752}, 'category_id': 1}, 'segmentation': {'size': [427, 640], 'counts': 'mmR5d0a<;G6H7\KSOWL_1d3iNgKi1U4\NaKm1\4WNYKR2e4SNoJW2o4lMgJ\2X5gMaJ^2]5eM\Ja2b5bMUJe2i5aMmIf2R6`McIe2]6U2O01O001O0N2N101O1N2L5L3N3K5J6K4M4M3K5L4M2N2K4M3O02WOi0N2O10002N1O2N1N4N3L4L3N3M5J7J4L4L:F=C6J4L5K3M3M3M1O2N1O001O00010OO2O001O1N3N2gHRKP6P5mITKP6o4kIUK_2d0ZO\4QNTKd1S2DU3XNRKo1R3\NR3b1^4N2N101N100O1O3M2N1N3N1O2N2M4L3M3M3L6J7I8Hn^k1'}, ...... ]
My JSON file follows the the standard COCO format: { "info": { "description": "COCO 2017 Dataset", "url": "http://cocodataset.org", "version": "1.0", ... }, "licenses": { { "url": "http://creativecommons.org/licenses/by-nc-sa/2.0/", "id": 1, "name": "Attribution-NonCommercial-ShareAlike License" }, ... }, "images": [ { "license": 4, "file_name": "000000397133.jpg", "coco_url": "http://images.cocodataset.org/val2017/000000397133.jpg", "height": 427, "width": 640, "date_captured": "2013-11-14 17:02:52", "flickr_url": "http://farm7.staticflickr.com/6116/6255196340_da26cf2c9e_z.jpg", "id": 397133 }, ... ], "annotations": [ { "segmentation": RLE or [polygon], "area": float, "iscrowd": 0 or 1, "image_id": int, "bbox": [x, y, width, height], "category_id": int, "id": int }, ... ], "categories": [ { "supercategory": str, "id": int, "name": str }, ... ] }
Can you tell which keys in your example(https://cloud.tsinghua.edu.cn/f/d86b4a4e67544dc494ae/?dl=1) are necessary? I will convert my JOSN file to your format. I guess 'bbox' and 'score' are not necessary?

Dear partner: Hello! I noticed that you mentioned that you have already generated a JSON file for the segmentation results. How did you generate it? The JSON file I generated only includes information such as evaluation metrics (miou), segmentation image names, and does not include information such as segmentation categories and point coordinates. I hope you can answer me in your free time! Looking forward to your reply!

As the author say, you can refer to open-mmlab/mmdetection#2693 or its document and produce the coarse segmentation results and their JSON file for input. If you want to produce GT datatsets, you can use pip-tool (pycocotools).

Dear friend: Perhaps you can tell me the instructions for generating JSON files? Looking forward to your reply!

[chenhang98]

mmdetection has changed this link. Now you can refer to open-mmlab/mmdetection#2693 or its document for details. You can also get an example here https://cloud.tsinghua.edu.cn/f/d86b4a4e67544dc494ae/?dl=1

OK,I found it's not a standard Object Instance COCO format but a list like: [ {'image_id': 397133, 'bbox': [385.7433776855469, 68.52386474609375, 117.4853515625, 285.50604248046875], 'score': 0.998802661895752}, 'category_id': 1}, 'segmentation': {'size': [427, 640], 'counts': 'mmR5d0a<;G6H7\KSOWL_1d3iNgKi1U4\NaKm1\4WNYKR2e4SNoJW2o4lMgJ\2X5gMaJ^2]5eM\Ja2b5bMUJe2i5aMmIf2R6`McIe2]6U2O01O001O0N2N101O1N2L5L3N3K5J6K4M4M3K5L4M2N2K4M3O02WOi0N2O10002N1O2N1N4N3L4L3N3M5J7J4L4L:F=C6J4L5K3M3M3M1O2N1O001O00010OO2O001O1N3N2gHRKP6P5mITKP6o4kIUK_2d0ZO\4QNTKd1S2DU3XNRKo1R3\NR3b1^4N2N101N100O1O3M2N1N3N1O2N2M4L3M3M3L6J7I8Hn^k1'}, ...... ]
My JSON file follows the the standard COCO format: { "info": { "description": "COCO 2017 Dataset", "url": "http://cocodataset.org", "version": "1.0", ... }, "licenses": { { "url": "http://creativecommons.org/licenses/by-nc-sa/2.0/", "id": 1, "name": "Attribution-NonCommercial-ShareAlike License" }, ... }, "images": [ { "license": 4, "file_name": "000000397133.jpg", "coco_url": "http://images.cocodataset.org/val2017/000000397133.jpg", "height": 427, "width": 640, "date_captured": "2013-11-14 17:02:52", "flickr_url": "http://farm7.staticflickr.com/6116/6255196340_da26cf2c9e_z.jpg", "id": 397133 }, ... ], "annotations": [ { "segmentation": RLE or [polygon], "area": float, "iscrowd": 0 or 1, "image_id": int, "bbox": [x, y, width, height], "category_id": int, "id": int }, ... ], "categories": [ { "supercategory": str, "id": int, "name": str }, ... ] }
Can you tell which keys in your example(https://cloud.tsinghua.edu.cn/f/d86b4a4e67544dc494ae/?dl=1) are necessary? I will convert my JOSN file to your format. I guess 'bbox' and 'score' are not necessary?

Dear partner: Hello! I noticed that you mentioned that you have already generated a JSON file for the segmentation results. How did you generate it? The JSON file I generated only includes information such as evaluation metrics (miou), segmentation image names, and does not include information such as segmentation categories and point coordinates. I hope you can answer me in your free time! Looking forward to your reply!

As the author say, you can refer to open-mmlab/mmdetection#2693 or its document and produce the coarse segmentation results and their JSON file for input. If you want to produce GT datatsets, you can use pip-tool (pycocotools).

Yes, I referred to the reference link provided by the author(See image for details) and ran the following command in Python/ Tools/test.py (config file path) (The path where the trained model is located)-- format only -- options "jsonfile_prefix=./results", but an error was reported as follows: "TypeError:SpineDataset: init() got an unexpected keyword argument 'jsonfile_prefix'"

It seems that SpineDataset is your customized dataset, so you may need to implement the saving json code yourself. You can refer to COCODataset in mmdetection.