Skip to content

File checking for meta.bin with Imagenet data loader is not required #66

@BenjaminCarpenter480

Description

@BenjaminCarpenter480

The current ImageNet data loader enforces the presence of a meta.bin file (imagenet.py#L67-L72). This requirement prevents users from easily using the Imagenet dataset without a meta.bin file, such as those available through PyTorch's Vision library.

The meta.bin file is not widely distributed online and typically must be generated by the user.
PyTorch-Vision can generate this file automatically when needed, as discussed in pytorch/vision#1647 (comment).
Enforcing this file in the loader restricts flexibility and complicates the use of standard PyTorch workflows.

Expected Behaviour

The data loader should not require an existing meta.bin file as the underlying PyTorch-Vision library can generate it automatically. This would enable greater compatibility with standard datasets and tools.

Reproduction

Can be reproduced by attempting to use the ImageNet data loader with a dataset structure compatible with PyTorch-Vision, but without a pre-existing meta.bin.

Observe that loading fails due to the missing file, even though PyTorch-Vision could generate it if allowed.

Fix

Update the Imagenet data loader to remove the checking for a meta.bin file and allow Pytorch-Vision to do the processing and checking instead.

Related links

Relevant code section
PyTorch Vision Issue #1647

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions