[PUB-1041] OBJECT_SYNC spec #333

VeskeR · 2025-06-12T08:45:41Z

This is the spec equivalent of the object sync sequence handling in ably-js https://github.com/ably/ably-js/blob/main/src/plugins/objects/objects.ts#L215

Resolves PUB-1041

textile/objects-features.textile

Based on [1] at dd25dca. The development approach here was roughly the following: I decided the internal types and interfaces that I wanted to exist, and some of the implementation details (e.g. the mutations that DefaultRealtimeObjects performs during a sync sequence). I then asked Cursor to fill in the most of the implementation and tests (providing it with the relevant spec text). I have done some tweaking of the code that Cursor generated, but that doesn't mean that all of the code that's here is exactly the same as code that I would write, and I'm going to leave it that way. I have looked through all (and changed some) of the tests that Cursor generated, and in doing so also fixed the spec conformance tags (because it got those very wrong), so I am _fairly_ confident that the tests are testing what they claim to test. But I have not looked at them in minute detail (in particular, the exact data created by the canned data factories that it created). I also think that tests are one of those things where there's not always one right way to write them, so I'm not going to get hung up on "could it have done this test differently?" or "are the tests consistent with each other?" at the moment. There are some outstanding questions on the spec PR at the moment; have implemented to best of my understanding, and will update later once those are answered. Not yet implemented: - Checking of channel modes before performing operations (RTO2 and the points that refer to it); there's an outstanding question about this at [2] [1] ably/specification#333 [2] https://github.com/ably/specification/pull/333/files#r2152297442

Based on [1] at dd25dca. The development approach here was roughly the following: I decided the internal types and interfaces that I wanted to exist, and some of the implementation details (e.g. the mutations that DefaultRealtimeObjects performs during a sync sequence). I then asked Cursor to fill in the most of the implementation and tests (providing it with the relevant spec text). I have done some tweaking of the code that Cursor generated, but that doesn't mean that all of the code that's here is exactly the same as code that I would write, and I'm going to leave it that way. I have looked through all (and changed some) of the tests that Cursor generated, and in doing so also fixed the spec conformance tags (because it got those very wrong), so I am _fairly_ confident that the tests are testing what they claim to test. But I have not looked at them in minute detail (in particular, the exact data created by the canned data factories that it created). I also think that tests are one of those things where there's not always one right way to write them, so I'm not going to get hung up on "could it have done this test differently?" or "are the tests consistent with each other?" at the moment. There are some outstanding questions on the spec PR at the moment; have implemented to best of my understanding, and will update later once those are answered. Not yet implemented: - Checking of channel modes before performing operations (RTO2 and the points that refer to it); there's an outstanding question about this at [2] A note re the MutableState pattern that I've used here — done this so that the class can essentially have mutating instance methods which can call each other without having to worry about whose responsibility it is to acquire the mutex. [1] ably/specification#333 [2] https://github.com/ably/specification/pull/333/files#r2152297442

textile/objects-features.textile

sacOO7 · 2025-07-10T09:13:09Z

textile/objects-features.textile

+**** @(RTO5c1a)@ If an object with @ObjectState.objectId@ exists in the internal @ObjectsPool@:
+***** @(RTO5c1a1)@ Override the internal data for the object as per "RTLC6":#RTLC6, "RTLM6":#RTLM6
+**** @(RTO5c1b)@ If an object with @ObjectState.objectId@ does not exist in the internal @ObjectsPool@:
+***** @(RTO5c1b1)@ Create a new @LiveObject@ using the data from @ObjectState@ and add it to the internal @ObjectsPool@:


Since we already have objectState.objectId, why can't we create zero-value liveobject from the same?

This feels like an alternative and duplicate of RTO6

At the same time, RTO6 allow us to validate ObjectId properly, which is missing in current case

RTO5c1b1 makes sense as the object creation here depends on ObjectState to have a correct field (ObjectState.counter for a counter and ObjectState.map for a map) to decide what object to create. If SDK can't find a recognizable field, the SDK can't process such ObjectState message and logs a warning instead (RTO5c1b1c).

Also, since ObjectState.objectId and ObjectState.counter/ObjectState.map are part of the same ObjectState object, the spec expects that realtime has sent a correct objectId that matches data from ObjectState.counter/ObjectState.map.

textile/objects-features.textile

sacOO7 · 2025-07-11T06:41:52Z

textile/objects-features.textile

+** @(RTO3a)@ @ObjectsPool@ is a @Dict<String, LiveObject>@ - a map of @LiveObject@s keyed by "@objectId@":../features#OST2a string
+** @(RTO3b)@ It must always contain a @LiveMap@ object with id @root@
+* @(RTO4)@ When a channel @ATTACHED@ @ProtocolMessage@ is received, the @ProtocolMessage@ may contain a @HAS_OBJECTS@ bit flag indicating that it will perform an objects sync, see "TR3":../features#TR3 . Note that this does not imply that objects are definitely present on the channel, only that there may be; the @OBJECT_SYNC@ message may be empty
+** @(RTO4a)@ If the @HAS_OBJECTS@ flag is 1, the server will shortly perform an @OBJECT_SYNC@ sequence as described in "RTO5":#RTO5


In ably-js, we check if objects state is INITIALIZED, so irrespective of hasObjects, we always call startNewSync in such a case, can we include this case here?

The check in ably-js you mentioned is only required so that library emits "syncing" -> "synced" events (in that order) for lifecycle events even if HAS_OBJECTS flag was false.
The lifecycle events are not part of this spec and will be added in the later PR, which will also address this spec point. Will work on the lifecycle events PR as soon as we finish addressing the outstanding comments on existing PRs.

In relation to this PR, it's fine for the spec to only check HAS_OBJECTS flag

textile/objects-features.textile

I have observed it being absent for a tombstoned map entry received in an OBJECT_SYNC. Also RTLM8a2a of #333 at dd25dca tells us to set it to null.

…ECT_SYNC spec

… phrasing in RTLM7*, RTLM8*, RTLM6d1a and RTLM6d1b This uses phrasing initially added in 652daac commit in #343 PR. It makes sense to include those changes in this #333 PR to make the OBJECT sync sequence and related spec items more clear.

sacOO7

LGTM ( You can recheck all review comments in case )

VeskeR requested review from lawrence-forooghian and sacOO7 June 12, 2025 08:45

github-actions bot temporarily deployed to staging/pull/333 June 12, 2025 08:46 Inactive

VeskeR changed the base branch from main to objects-plugin June 12, 2025 08:46

sacOO7 reviewed Jun 12, 2025

View reviewed changes

textile/objects-features.textile Show resolved Hide resolved

VeskeR force-pushed the PUB-1041/object-sync branch from d10e385 to 3336d98 Compare June 12, 2025 09:14

github-actions bot temporarily deployed to staging/pull/333 June 12, 2025 09:15 Inactive

lawrence-forooghian added the live-objects Related to LiveObjects functionality. label Jun 12, 2025

VeskeR force-pushed the PUB-1041/object-sync branch 2 times, most recently from 36792eb to cdb9d58 Compare June 12, 2025 23:37

github-actions bot temporarily deployed to staging/pull/333 June 12, 2025 23:37 Inactive

VeskeR force-pushed the PUB-1041/object-sync branch from cdb9d58 to f4dd5d4 Compare June 13, 2025 00:15

github-actions bot temporarily deployed to staging/pull/333 June 13, 2025 00:15 Inactive

VeskeR force-pushed the PUB-1041/object-sync branch from f4dd5d4 to b4d8e4d Compare June 13, 2025 02:04

github-actions bot temporarily deployed to staging/pull/333 June 13, 2025 02:05 Inactive

VeskeR force-pushed the PUB-1041/object-sync branch from b4d8e4d to 948ad9b Compare June 13, 2025 08:40

github-actions bot temporarily deployed to staging/pull/333 June 13, 2025 08:40 Inactive

lawrence-forooghian reviewed Jun 17, 2025

View reviewed changes

textile/objects-features.textile Outdated Show resolved Hide resolved

VeskeR force-pushed the objects-plugin branch 3 times, most recently from bebe5fa to 5a6893a Compare June 19, 2025 03:00

VeskeR force-pushed the PUB-1041/object-sync branch from 948ad9b to 39111b8 Compare June 19, 2025 03:05

github-actions bot temporarily deployed to staging/pull/333 June 19, 2025 03:06 Inactive

VeskeR force-pushed the PUB-1041/object-sync branch from 39111b8 to 8a7c38e Compare June 19, 2025 03:06

github-actions bot temporarily deployed to staging/pull/333 June 19, 2025 03:07 Inactive

VeskeR mentioned this pull request Jun 19, 2025

Add spec for ObjectMessage encoding and decoding #335

Merged

VeskeR force-pushed the objects-plugin branch from 5a6893a to 2266a50 Compare June 19, 2025 04:29

VeskeR force-pushed the PUB-1041/object-sync branch from 8a7c38e to 2335a1d Compare June 19, 2025 04:30

github-actions bot temporarily deployed to staging/pull/333 June 19, 2025 04:30 Inactive

sacOO7 reviewed Jul 8, 2025

View reviewed changes

textile/objects-features.textile Show resolved Hide resolved

sacOO7 reviewed Jul 10, 2025

View reviewed changes

sacOO7 reviewed Jul 11, 2025

View reviewed changes

textile/objects-features.textile Show resolved Hide resolved

sacOO7 reviewed Jul 11, 2025

View reviewed changes

lawrence-forooghian mentioned this pull request Jul 11, 2025

[PUB-1827] Add Objects spec for subscriptions #346

Merged

lawrence-forooghian reviewed Jul 30, 2025

View reviewed changes

textile/objects-features.textile Show resolved Hide resolved

lawrence-forooghian added a commit that referenced this pull request Aug 4, 2025

Fix nullability of ObjectsMapEntry.data

97c43cb

I have observed it being absent for a tombstoned map entry received in an OBJECT_SYNC. Also RTLM8a2a of #333 at dd25dca tells us to set it to null.

lawrence-forooghian mentioned this pull request Aug 4, 2025

Fix nullability of ObjectsMapEntry.data #355

Merged

VeskeR force-pushed the PUB-1041/object-sync branch from dd25dca to fc46aed Compare September 3, 2025 08:45

This was referenced Sep 4, 2025

Change RTO1 to have an implicit channel attach and clarify how it reacts to sync sequence changes #369

Open

Update RTO2 to check only against the modes granted by the server and give languages a choice of loggin a warning instead of throwing an error #371

Open

VeskeR requested a review from lawrence-forooghian September 4, 2025 20:11

VeskeR added 2 commits September 5, 2025 08:48

Add separate page for Objects spec, RealtimeChannel.objects and OBJ…

ba9d924

…ECT_SYNC spec

Improve phrasing and clarify a couple of spec points

b3e06dc

VeskeR force-pushed the PUB-1041/object-sync branch from bfcf05b to b3e06dc Compare September 5, 2025 07:48

github-actions bot temporarily deployed to staging/pull/333 September 5, 2025 07:48 Inactive

github-actions bot temporarily deployed to staging/pull/333 September 5, 2025 11:56 Inactive

lawrence-forooghian approved these changes Sep 5, 2025

View reviewed changes

VeskeR requested review from owenpearson and sacOO7 September 5, 2025 13:52

Clarify RTO5 expects ObjectMessage.object to be non-null

18cbb9c

github-actions bot temporarily deployed to staging/pull/333 September 5, 2025 13:55 Inactive

Clarify RTO6

a70dc1a

github-actions bot deployed to staging/pull/333 September 5, 2025 14:03 View deployment

sacOO7 approved these changes Sep 5, 2025

View reviewed changes

VeskeR merged commit 6d1dabd into main Sep 8, 2025
2 checks passed

VeskeR deleted the PUB-1041/object-sync branch September 8, 2025 08:22

[PUB-1041] OBJECT_SYNC spec #333

[PUB-1041] OBJECT_SYNC spec #333

Uh oh!

Conversation

VeskeR commented Jun 12, 2025 • edited by atlassian bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sacOO7 Jul 10, 2025

Choose a reason for hiding this comment

Uh oh!

sacOO7 Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sacOO7 Jul 10, 2025

Choose a reason for hiding this comment

Uh oh!

VeskeR Sep 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sacOO7 Jul 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

VeskeR Sep 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sacOO7 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

5 participants

VeskeR commented Jun 12, 2025 •

edited by atlassian bot

Loading

sacOO7 Jul 10, 2025 •

edited

Loading

VeskeR Sep 5, 2025 •

edited

Loading

sacOO7 Jul 11, 2025 •

edited

Loading