[core] Enable global index external path #6994
Conversation
leaves12138
force-pushed
the
index_external
branch
from
9e65491 to
d57cb9a
Compare
| private final long fileSize; | ||
| private final byte[] metadata; | ||
| @Nullable private final String externalPath; | ||
|
|
There was a problem hiding this comment.
The GlobalIndexIOMeta is intended to be a meta interface for plugin authors. However, including an external path in it might be confusing for users, as it adds complexity and is not intuitive in this context.
Would it make more sense to store only the file_path in GlobalIndexIOMeta, while leaving the external path to be handled internally by the Paimon framework? This way, the external path could be extracted and managed within the IndexFileMeta, simplifying the API for plugin authors and ensuring a clearer separation of concerns.
|
|
||
| public Path globalIndexRootDir() { | ||
| return globalIndexExternalRootDir != null ? globalIndexExternalRootDir : indexPath(); | ||
| } |
There was a problem hiding this comment.
If globalIndexExternalRootDir is not null, the index path be external_index_path/index/ or external_index_path/? The first one maybe better? Not sure whether retaining the index/ is necessary.
There was a problem hiding this comment.
Accepted, it could be like this
leaves12138
changed the title
leaves12138
force-pushed
the
index_external
branch
from
6a61e59 to
9330ef8
Compare
|
+1 |
2 participants
Purpose
Enable global index store in external path
Tests
API and Format
Documentation