Skip to content

COCO export is very memory inefficient.  #63

@jrdalenberg

Description

@jrdalenberg

When I try to export a dataset to COCO (ds.export(style='coco')) with 75k images (200-800kb), each containing a single segmentation, the export function uses around 128GB RAM and 60GB swap.

I assume that images are loaded but not released from memory when coco annotations are generated.

I am using python 3.8.18
opencv-python==4.5.3.56
imantics 3cd6526

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions