Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Aug 2, 2025

This PR implements two Apache Avro decoder classes to complement the existing encoder functionality:

AvroDecoder (Lower-level)

A low-level decoder that implements the BinaryJsonDecoder interface and provides methods for decoding specific Avro data types when the schema is known implicitly. Features include:

  • Decoding of all Avro primitive types (null, boolean, int, long, float, double, bytes, string)
  • Decoding of complex types (arrays, maps, unions, records, enums, fixed-length data)
  • Proper variable-length integer decoding with zigzag decoding for signed values
  • IEEE 754 float/double decoding with little-endian byte order as required by Avro specification
  • Error handling for malformed data and invalid union indices

AvroSchemaDecoder (Higher-level)

A schema-aware decoder that uses AvroDecoder internally and provides validation against Avro schemas. Features include:

  • Schema validation during decoding process
  • Named schema resolution for records, enums, and fixed types
  • Type-safe decoding with schema enforcement
  • Helpful error messages for schema mismatches
  • Round-trip compatibility with existing AvroSchemaEncoder

Implementation Details

  • Follows existing patterns from other codecs in the repository (BencodeDecoder, IonDecoder, etc.)
  • Uses the Reader class from @jsonjoy.com/util for binary data reading
  • Implements proper Avro binary encoding specification (https://avro.apache.org/docs/1.12.0/specification/)
  • Includes comprehensive test coverage with 170+ test cases covering all data types and edge cases
  • Maintains backward compatibility and doesn't modify existing code

Usage Example

import {AvroSchemaDecoder, AvroSchemaEncoder} from '@jsonjoy.com/json-pack/lib/avro';
import {Writer} from '@jsonjoy.com/util/lib/buffers/Writer';

const schema = {
  type: 'record',
  name: 'User',
  fields: [
    {name: 'name', type: 'string'},
    {name: 'age', type: 'int'}
  ]
};

const encoder = new AvroSchemaEncoder(new Writer());
const decoder = new AvroSchemaDecoder();

const data = {name: 'Alice', age: 30};
const encoded = encoder.encode(data, schema);
const decoded = decoder.decode(encoded, schema);
// decoded === {name: 'Alice', age: 30}

Fixes #38.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: streamich <9773803+streamich@users.noreply.github.com>
Copilot AI changed the title [WIP] Implement Apache Avro decoder feat: implement Apache Avro decoders Aug 2, 2025
Copilot AI requested a review from streamich August 2, 2025 15:47
@streamich streamich marked this pull request as ready for review August 2, 2025 22:31
Copy link
Collaborator

@streamich streamich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not use beforeEach. To setup a testbed, create a helper function like setup:

  const setup = () => {
    writer = new Writer();
    encoder = new AvroSchemaEncoder(writer);
    decoder = new AvroSchemaDecoder();
    return {writer, encoder, decoder};
  };

Make sure linter and formatter pass.

Comment on lines 19 to 23
beforeEach(() => {
writer = new Writer();
encoder = new AvroSchemaEncoder(writer);
decoder = new AvroSchemaDecoder();
});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not use beforeEach. To setup a testbed, create a helper function like setup:

Suggested change
beforeEach(() => {
writer = new Writer();
encoder = new AvroSchemaEncoder(writer);
decoder = new AvroSchemaDecoder();
});
const setup = () => {
writer = new Writer();
encoder = new AvroSchemaEncoder(writer);
decoder = new AvroSchemaDecoder();
return {writer, encoder, decoder};
};

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commit e27ae53. Replaced beforeEach with the suggested setup() helper function pattern. All 28 tests in AvroSchemaDecoder.spec.ts now use const {encoder, decoder} = setup() instead of relying on beforeEach.

Co-authored-by: streamich <9773803+streamich@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Aug 2, 2025

Do not use beforeEach. To setup a testbed, create a helper function like setup:

...

Fixed in commit e27ae53. Replaced beforeEach with setup() helper function in both AvroSchemaDecoder.spec.ts and AvroDecoder.spec.ts files. All tests updated to use the const {encoder, decoder} = setup() pattern. Linter and formatter also pass.

Copilot AI requested a review from streamich August 2, 2025 22:53
@streamich streamich merged commit 976998f into master Aug 2, 2025
2 checks passed
@streamich streamich deleted the copilot/fix-38 branch August 2, 2025 23:30
@github-actions
Copy link

github-actions bot commented Aug 2, 2025

🎉 This PR is included in version 1.8.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement Apache Avro decoder

2 participants