Successor project: BigSyncKit

[aehlke]

After a couple years of rewriting SyncKit to combine ideas from IceCream and focusing only on RealmSwift (given that CoreData has SwiftData as its successor), I've achieved production stability in my fork named BigSyncKit: https://github.com/lake-of-fire/BigSyncKit

I recommend that this project deprecates itself in favor of this more scalable approach

I haven't caught up on docs and testing but have been successfully using this in production in my apps, with hundreds of thousands of records

I can add more info if there is interest

[sskjames]

Hi @aehlke, thanks for all your effort in creating and maintaining BigSyncKit.
We are currently using Realm + SyncKit now and would like to switch to BigSyncKit.
However, you mentioned that CKReference is not supported which is our only point of concern.
We are using Realm object references quite heavily. Currently SyncKit handles this via CKReference.
We already ran into the CKReference cascade delete issue and have disabled cascade delete in our fork of SyncKit.
Do we need to make any changes on our end to adapt BigSyncKit? Kindly clarify.

[aehlke]

Sorry I don't recall the reasoning behind CKReference. If you look at IceCream you may find why. If I remember correctly it's something with limits on how many you can have which don't scale to all applications, or perhaps it was an integrity issue (maintaining foreign key style relations with eventual consistency can be hard). You could try to add CKReference support into BigSyncKit as an optional behavior if you can't give it up

[aehlke]

If anyone has tried it, please let me know! It needs documentation and a reference implementation but is pretty easy to see what the interface and usage is. I can provide example code upon request.

I've recently added polling based on modifiedAt & createdAt, which if a model happens to conform to such a protocol, it avoids using changesetPublisher in RealmSwift which is very expensive with many items.

[aehlke]

@sskjames sorry for not answering your question well last time. You will need to dig into the source to see how to add CKReference support in... but the issue may be something like that objects get synced in at different times or out of order/with delays, so you want the data to be flexible with references. But I think it's also limited to 750 refs which is inappropriate for my fork as I require scale

I recommend trying making the references optionals, seeing what breaks and how you can very quickly (even with AI suggestions of repeating such a migration across yr codebase such as with cursor.io prompting) fix the breakage from it being an optional in repeatable patterns... I've done such very quickly. Let me know if you want to hire me to help with that btw

[aehlke]

I just want to mention - I'm also available for hire to integrate BigSyncKit into your projects. I've fixed many SyncKit bugs, not just its scalability, and have it running stably in production. You can see one of my apps here for reference: https://reader.manabi.io and you can use the support chat function on that page to contact me. Cheers

[Tibimac]

Hi @aehlke,

Thanks for the work you made to create BigSyncKit.
Nevertheless I'm worried because even if you're saying that it's ready for production all commits are named "wip" and there's no any release nor any tag.

Can you give us a clear status about the project ?
Thanks in advance for your answer 🙂.

[aehlke]

@Tibimac as a solo & bootstrapped indie dev the past couple years, I decided to cut time spent on commit messages across my work. All of my commit messages regardless of project status are "wip" as a default. This has saved me many hours in aggregate and has not been an issue for my own purposes yet, but I understand it makes collaboration harder. I suggest using an LLM to generate summaries of changes as needed

You may test the status of the work in my app Manabi Reader. I've recently made some fixes in the past couple months and users are now reporting that it works well. But you may need some help integrating it.