Skip to content

Conversation

@leaves12138
Copy link
Contributor

Purpose

Enable global index store in external path

Tests

API and Format

Documentation

private final long fileSize;
private final byte[] metadata;
@Nullable private final String externalPath;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The GlobalIndexIOMeta is intended to be a meta interface for plugin authors. However, including an external path in it might be confusing for users, as it adds complexity and is not intuitive in this context.

Would it make more sense to store only the file_path in GlobalIndexIOMeta, while leaving the external path to be handled internally by the Paimon framework? This way, the external path could be extracted and managed within the IndexFileMeta, simplifying the API for plugin authors and ensuring a clearer separation of concerns.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accepted


public Path globalIndexRootDir() {
return globalIndexExternalRootDir != null ? globalIndexExternalRootDir : indexPath();
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If globalIndexExternalRootDir is not null, the index path be external_index_path/index/ or external_index_path/? The first one maybe better? Not sure whether retaining the index/ is necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accepted, it could be like this

@leaves12138 leaves12138 changed the title [WIP] [core] Enable global index external path [core] Enable global index external path Jan 12, 2026
@lxy-9602
Copy link

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants