Skip to content

Conversation

@VeskeR
Copy link
Contributor

@VeskeR VeskeR commented Jun 13, 2025

This is the spec equivalent of the encoding/decoding logic implemented in ably-js for ObjectMessage

@github-actions github-actions bot temporarily deployed to staging/pull/335 June 13, 2025 02:06 Inactive
@VeskeR VeskeR added the live-objects Related to LiveObjects functionality. label Jun 13, 2025
* @(OD4)@ @ObjectData@ encoding:
** @(OD4a)@ Payloads must be booleans, binary, numbers, strings, or objects capable of JSON representation. Any other data type must not be permitted and result in an error with code 40013.
** @(OD4b)@ When the MessagePack protocol is used:
*** @(OD4b1)@ A boolean payload is encoded as a MessagePack boolean type, and the result is set on the @ObjectData.boolean@ attribute.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
*** @(OD4b1)@ A boolean payload is encoded as a MessagePack boolean type, and the result is set on the @ObjectData.boolean@ attribute.
*** @(OD4b1)@ A boolean payload is encoded as a MessagePack boolean type, and the result is set on the @ObjectData.boolean@ attribute. The @ObjectData.encoding@ attribute is then set to "msgpack"

or you can add separate spec point for the same

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The @ObjectData.encoding@ attribute is then set to "msgpack"

It must not be set in this case. Currently the ObjectData.encoding is only set when the payload is being JSON-stringified, in OD4b5 and OD4c5

Copy link
Collaborator

@sacOO7 sacOO7 Jun 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, then we should explicitly specify that it should not be set. Otherwise, it could easily be misinterpreted as to be set to either msgpack or json.
i.e. we set encoding field for OOP5a, OOP5b

Copy link
Contributor Author

@VeskeR VeskeR Jun 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, then we should explicitly specify that it should not be set

I don't think it's reasonable to list all the things the client library should not do. ObjectData encoding/decoding spec should not be compared to encoding for ObjectOperation

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course, it shouldn't be compared directly. However, from a reader's perspective, it's reasonable to expect that the encoding field would have a specific value. It's always better to define things explicitly in the spec to avoid any implicit misunderstandings. In this case, the spec doesn't clearly state whether a value should or shouldn't be set—it's rather ambiguous

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that the encoding field would have a specific value

I do not agree with this. The spec is explicit in its phrasing that ObjectOperation.initialValueEncoding OOP3i must be set by the client library when sending create ops to the server, while the ObjectData.encoding OD2b may be set to indicate that one of the value fields has an additional encoding (an argument can be made that it shouldn't mention bytes field at the moment, as there is a no procedure for additionally encoding anything that goes into bytes).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not currently comparing this with initialValueEncoding; rather, I'm trying to share my understanding from a reader's perspective. Specifically, this spec doesn't clearly state whether a value should or should not be set.

For example, in the case of OD4b5, it explicitly says to set the encoding to json. In other cases, we describe how the value is encoded—for instance, "A string payload is encoded as a MessagePack string type"—but we neither state that the encoding field must be set, nor that it should be left unset.

If we were to feed this spec into a context-driven LLM model, it would likely infer or assume some default value. That’s why I’m suggesting we make this aspect explicit in the spec to avoid ambiguity.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current encoding spec for ObjectMessage was written in the same format as the encoding spec for regular messages in RSL4, where a spec only specifies when to set the encoding field.
That being said, I don't see a huge issue with adding a single spec point that states that encoding field is left unset by default, unless specified by the encoding procedure. Added it in https://github.com/ably/specification/compare/826e4bddf506d3923dc746edc893591ce78d245d..7f30177752945dae89891c9146109fbfa1dcea08

we describe how the value is encoded—for instance, "A string payload is encoded as a MessagePack string type"—but we neither state that the encoding field must be set, nor that it should be left unset.

I should note that ObjectData.encoding (just like Message.encoding from RSL4) is not meant to tell the client library that the value at the ObjectData.string attribute is string-encoded. It exists to inform the client library that the value represents something else and should be further processed. For example, if client received ObjectData.string with ObjectData.encoding set to json, then the string is actually a JSON string - so if you JSON-parse it, you'll get the original object.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ObjectData.encoding is no longer used and will be removed altogether in #360

@VeskeR VeskeR force-pushed the objects-encoding branch from 6155ce4 to 826e4bd Compare June 13, 2025 13:19
@github-actions github-actions bot temporarily deployed to staging/pull/335 June 13, 2025 13:19 Inactive
*** @(OD4c1)@ A boolean payload is represented as a JSON boolean and set on the @ObjectData.boolean@ attribute.
*** @(OD4c2)@ A binary payload is Base64-encoded and represented as a JSON string; the result is set on the @ObjectData.bytes@ attribute.
*** @(OD4c3)@ A number payload is represented as a JSON number and set on the @ObjectData.number@ attribute.
*** @(OD4c4)@ A string payload is represented as a JSON string and set on the @ObjectData.string@ attribute.
Copy link
Collaborator

@sacOO7 sacOO7 Jun 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally ObjectData.encoding should be called ObjectData.isJsonString or ObjectData.isJson that is either set to true or false.
If it's true, we construct JsonObject or JsonArray from the @objectdata.string@ value.
Is it possible to be changed as a part of https://github.com/ably/specification/pull/335/files#r2144849490 changes
wdyt @lmars

Copy link
Collaborator

@sacOO7 sacOO7 Jun 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ObjectData.encoding in itself feels misleading. When we apply msgpack encoding, ObjectData.encoding is set to either json or none

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH if we make a change to the fields here, I'd suggest adding a separate json field which is explicitly used to send a JSON encoded value, rather than using the string field with encoding=json.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, so field ObjectData.json set to true or false ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No sorry, I mean ObjectData.json would be set to the JSON encoded value, for example ObjectData.json = '{"this":"is some json"}'), rather than what would currently be string = '{"this":"is some json"}', encoding = 'json'

Copy link
Collaborator

@sacOO7 sacOO7 Jun 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, understood, so new ObjectData will look like

class ObjectData // OD*, internal
  objectId: String? // OD2a
  boolean: Boolean? // OD2c
  bytes: Binary? // OD2d
  number: Number? // OD2e
  string: String? // OD2f
  json: String? // OD2g

This also makes logical sense—since the encoding field can be misleading. It implies that the encoding applies to the entire ObjectData( related discussion ), when in fact, it only refers to the encoding of a specific field.

Copy link
Collaborator

@sacOO7 sacOO7 Jun 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lmars, I guess this can be addressed as a part of getting rid of initialValueEncoding => #335 (comment)

Copy link
Contributor Author

@VeskeR VeskeR Sep 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spec for ObjectData.encoding and .json will be updated in #360

lawrence-forooghian added a commit to ably/ably-liveobjects-swift-plugin that referenced this pull request Jun 18, 2025
@VeskeR VeskeR force-pushed the objects-encoding branch from 826e4bd to 7f30177 Compare June 19, 2025 03:53
@github-actions github-actions bot temporarily deployed to staging/pull/335 June 19, 2025 03:53 Inactive
@VeskeR VeskeR force-pushed the objects-encoding branch from 7f30177 to f4cff45 Compare June 19, 2025 04:08
@github-actions github-actions bot temporarily deployed to staging/pull/335 June 19, 2025 04:08 Inactive
@VeskeR VeskeR force-pushed the objects-encoding branch 2 times, most recently from 2619ee2 to 21dda52 Compare June 20, 2025 01:39
@github-actions github-actions bot temporarily deployed to staging/pull/335 June 20, 2025 01:39 Inactive
@VeskeR VeskeR force-pushed the objects-encoding branch from 21dda52 to 2e975cb Compare June 20, 2025 01:44
@github-actions github-actions bot temporarily deployed to staging/pull/335 June 20, 2025 01:45 Inactive
VeskeR added a commit to ably/ably-js that referenced this pull request Jun 20, 2025
sacOO7
sacOO7 previously requested changes Jun 22, 2025
Copy link
Collaborator

@sacOO7 sacOO7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw, just noticed, both of my msgpack and json were using enum string to send and receive data. Instead it should use code starting from 0, can you add separate spec point for the same?

@lawrence-forooghian
Copy link
Collaborator

Instead it should use code starting from 0

Why?

@sacOO7
Copy link
Collaborator

sacOO7 commented Jun 23, 2025

Instead it should use code starting from 0

Why?

Apologies for the confusion — what I meant is that it should use the underlying code values. Currently, the enums have code values starting from 0, but in my case, the serializer was defaulting to serializing them as strings. Instead, it should encode the enum values using their underlying code representation.

@lawrence-forooghian
Copy link
Collaborator

Sorry, I don't think I know what you're referring to. What property are you talking about exactly, and what is the enum that you're referring to?

@lawrence-forooghian
Copy link
Collaborator

lawrence-forooghian commented Jun 23, 2025

Sorry, I don't think I know what you're referring to. What property are you talking about exactly, and what is the enum that you're referring to?

Asked about this in standup — @sacOO7 just meant that the spec should be explicit that when sending an enum value (e.g. a MapSemantics) then the client should encode it as the numerical enum value. I don't think we need to do this here — we've been treating that as implicit elsewhere in the spec (e.g. when sending a PresenceAction or ProtocolMessageAction). Might not be good to be explicit about it in general but I don't think you need to let it distract you on this PR.

lawrence-forooghian added a commit to ably/ably-liveobjects-swift-plugin that referenced this pull request Jun 24, 2025
lawrence-forooghian added a commit to ably/ably-liveobjects-swift-plugin that referenced this pull request Jun 24, 2025
Based on [1] at 2e975cb.

This was implemented by updating the code to reflect the internal API
that I wanted to exist, and then asking Cursor to implement the rules of
the spec and to write tests. I then edited the generated code to
simplify it a bit and add things like @SPEC annotations and other
explanatory comments.

I wanted some round-trip tests that go through the Realtime backend but
decided to leave them until later once I have a bit more knowledge of
the LiveObjects protocol; have created #17.

[1] ably/specification#335
VeskeR added a commit to ably/ably-js that referenced this pull request Jun 26, 2025
lmars
lmars previously requested changes Jun 30, 2025
lawrence-forooghian added a commit to ably/ably-cocoa that referenced this pull request Jul 23, 2025
The plugin will use this to apply the spec's rules (see [1]) regarding
the encoding and decoding of binary data.

This was initially generated by Cursor at my instructions, but I ended
up rewriting most of the documentation it generated because it didn't
have enough context to understand how the plugin would use the format.

[1] ably/specification#335
Also improves the phrasing used for some of the related
@VeskeR VeskeR requested a review from lmars September 5, 2025 07:38
@VeskeR VeskeR dismissed stale reviews from lmars and sacOO7 September 5, 2025 07:41
@VeskeR VeskeR merged commit 36460be into main Sep 5, 2025
2 checks passed
@VeskeR VeskeR deleted the objects-encoding branch September 5, 2025 07:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

live-objects Related to LiveObjects functionality.

Development

Successfully merging this pull request may close these issues.

5 participants