feat: puffin Reader and Writer #676
Conversation
Signed-off-by: Shreyas220 <shreyas.ny@gmail.com>
Signed-off-by: Shreyas220 <shreyas.ny@gmail.com>
Signed-off-by: Shreyas220 <shreyas.ny@gmail.com>
Signed-off-by: Shreyas220 <shreyas.ny@gmail.com>
|
Hey 👋 @zeroshade , @twuebi would appreciate a review |
zeroshade
left a comment
zeroshade
left a comment
There was a problem hiding this comment.
Thanks for starting this up! I've been wanting to get a puffin reader/writer in here for a while now. I've gone through with a bunch of questions / requests etc. But this also needs to have some tests and potentially examples if possible!
Thanks again!
puffin/puffin_writer.go
Outdated
|
|
||
| // SetProperties merges the provided properties into the file-level properties | ||
| // written to the footer. Can be called multiple times before Finish. | ||
| func (w *PuffinWriter) SetProperties(props map[string]string) error { |
There was a problem hiding this comment.
should we have a way to remove properties also? Or a way to clear the current properties?
There was a problem hiding this comment.
Since this is cumulative, maybe it should be renamed as "AddProperties" ?
There was a problem hiding this comment.
Added ClearProperties() method .
is there a requirement for removing a specific property ?
puffin/puffin_reader.go
Outdated
| // Read footer JSON payload | ||
| payload := make([]byte, payloadSize) | ||
| if _, err := r.r.ReadAt(payload, footerStart+MagicSize); err != nil { | ||
| return nil, fmt.Errorf("puffin: read footer payload: %w", err) | ||
| } | ||
|
|
||
| // Parse footer JSON | ||
| var footer Footer | ||
| if err := json.Unmarshal(payload, &footer); err != nil { | ||
| return nil, fmt.Errorf("puffin: decode footer JSON: %w", err) | ||
| } |
There was a problem hiding this comment.
we don't need to copy the bytes, use a io.LimitedReader and a json.Decoder and just decode directly from the reader instead. Given that JSON can be large, avoiding the copy before we unmarshal could be a potential significant savings
puffin/puffin_reader.go
Outdated
| // ReadRange reads a raw byte range from the blob data region. | ||
| // The footer must be read first to establish bounds checking. | ||
| func (r *PuffinReader) ReadRange(offset, length int64) ([]byte, error) { |
There was a problem hiding this comment.
what is the actual use of this function? When would a user want to use this?
There was a problem hiding this comment.
Deletion vector in manifests include offset and length into Puffin file. can directly read that range
Signed-off-by: Shreyas220 <shreyas.ny@gmail.com>
Signed-off-by: Shreyas220 <shreyas.ny@gmail.com>
2 participants
for #589